This is the third post in a series about source code management. You can start here if you haven’t read the others in the series.

There we were, happy as the proverbial Larry, checking our code into TFVC, requesting code reviews, branching, merging, viewing file history, comparing versions, annotating and writing a lot of PowerShell to automate tasks with the VSTS API. We were feeling pretty great about our source code management.

What could possibly induce us to move away from this system?

Developer Isolation

Our standard practice had always been for developers to work in the same development database. We rarely needed to have multiple developers working on the same project at the same time or not on the same objects at least.

As we invested in our products and grew the team we found ourselves overlapping a lot more than had been the case with customer specific development. Dynamics NAV isn’t equipped to handle this particularly well:

• If two developers have the same object open at the same time, both make changes and then save, whoever saves last wins. There is potential for development to be lost as soon as it has been written
• NAV does allow you to lock objects – sort of like checking them out in source code management – but we weren’t in the habit of doing it. We wouldn’t consistently lock them when we were working on them or unlock them when we had finished working on them
  • Hardly a fair criticism of NAV you might say – we just didn’t have a discipline for using the lock functionality properly. You may well be right
  • Even so, locking an object prevents anyone else from designing it. What is the point of me carefully splitting some development into discrete tasks so that different developers can work on them in parallel if they are going to trip over each other because they happen to be modifying the same objects?
• Only one developer can debug on a service tier at any time
  • So add more service tiers to the development server. You could, but it was already becoming a headache trying to manage all the development databases and service tiers that we’d got on our dev server. I didn’t fancy throwing any more fuel on that particular fire
• When you export the objects from NAV to check-in to source code management you increase the likelihood that mistakes will be made
  • I export objects that I’ve changed which may include changes that you’ve made. If I’m not careful then I’ll check-in your changes as part of my changeset
  • That can be complicated to fix and then merge into another branch. Or I’ll have to revert the changeset and have a redundant pair of changesets etched into the history of the project

The solution to all of these problems was to isolate the developers from each other. We’d each create a local development environment where we could work, test and debug safe in the knowledge that no one else is monkeying with the objects.

We invested time in PowerShell scripts to automate the creation and management of these environments. Once again, the VSTS API and tf.exe were our friends. As a side benefit we’d also limited our reliance on our development server and the single point of failure danger that it had posed.

Life was good again. We could work on separate features and share the code through shelvesets and code review before checking-in. We could create a new environment of a given product for development or testing in a few minutes with our automation.

Branching

Once we’d isolated developers I was more confident defining separate tasks for different developers to work on in parallel. So I did, but as we were still sharing the same branch in TFVC we started to run into a different set of problems.

• What if the same developer wanted to work on multiple work items at the same time?
  • This was particularly true when they’d finish the first work item and put it in for review. While they were waiting for review they’d want to crack on with the next task
  • Managing their local development environment became difficult – when they start the second task ideally they should work in a database that doesn’t include the changes from the first task
  • Creating an environment per work item – while feasible – isn’t very attractive.
• Having several code reviews open against the same branch becomes difficult to follow.
  • While we’d try to review and give feedback/approve as quickly as possible there are inevitably some that stick around for a while
  • The reviewing developer wants an environment with the latest stable code and the code under review. When there is an update to the code under review the shelveset must be replaced and downloaded and applied to the database again (a challenge in TFVC in general)

A sensible step to take is to introduce a branch per work item in TFVC. This allows unrelated changes to be isolated from each other and merged into a production branch once the code has been reviewed. I wasn’t thrilled at this prospect.

Branching in TFVC is expensive – in the sense that it is a full copy of the folder that is has been branched from. Even if you’ve got an entire branch downloaded into your workspace when you create another branch from it the new branch is created on the server and you must download it separately. If you want to delete the branch – which we’d want to do once the work item was finished – you need to download the entire branch, then delete your local folder to tell the server to delete the branch.

I now know that we’d stumbled over two of the most compelling reasons to use Git rather than TFVC (Or other distributed version control systems – but as we were already using VSTS Git was the natural alternative).

In Git:

• Isolated developers are a given. Everyone has their own copy of the whole repository
• Branching is cheap and easy. So cheap and easy that you are encouraged to do it all the time. It is very simple to isolate changes from each other and merge them back together again at a later date

Git

It is beyond the scope of this post to compare TFVC and Git in any depth – although there will be more in the final post – but these are the key points that led us to trial it and ultimately move all our code to Git repositories.

• Now that we were working in separate NAV databases our source code was effectively distributed among us – as opposed to centralised in a single development database.
  • This fits the ethos of Git (as a distributed version control system) much better than TFVC (as a centralised version control system)
  • Without realising it at the time we had effectively already moved to distributed version control
• Git maintains a single working directory with the contents of the current branch – as opposed to a separate folder with a copy of all objects per branch
  • This principle is what makes branching so cheap in Git. Creating a new branch requires no more than the creation of a single text file with a pointer to the contents of that branch
  • This is a far more attractive proposition when it comes to maintaining your local development database. Don’t create a database per branch or confuse yourself trying to work in multiple branches on the same database. Instead create a single database whose objects are updated to reflect the current contents of Git’s working directory (see below). PowerShell
• Code is shared through branches which are pushed to the central repository
  • Rather than through shelvesets which are necessarily a single, self-contained set of changes which are difficult to update and re-share.
  • Code reviews (pull requests) compare the changes between two branches rather than the changes contained in a single shelveset. I am constantly delighted with the power of this concept and the tools that Azure DevOps (VSTS) provides to support it. Perhaps more of that in another post one day. Pull requests are awesome

Synchronising with Git’s Working Directory

The most important jigsaw piece in our puzzle of adopting Git was to find a smooth way to keep the NAV database and Git’s working directory in sync with one another. If not, we were going to see some unexpected differences when exporting from NAV.

PowerShell came to the rescue and I added a bunch of functions to the module that we develop and use internally. We also use some functions from Cloud Ready Software’s modules – mainly their wrappers for standard functions that make them a little easier to call by just supplying the NAV service tier. The main functions are:

• Build-DevEnvironmentFromGit – to create the NAV database and service tier. I won’t go into the details of how that works now. Typically we’d do this once per product as we’re going to reuse the same database from now on rather than constantly building and deleting them
• Start-GitDev – to start the NAV service tier and import the NAV PowerShell modules from the correct build
• Export-ModifiedObjectsToWorkingTree
  • Export objects (Modified = true) from the NAV database to individual text files in the Git directory
  • Set Modified to false and DateTime to 1st January of the current year (to minimise conflicts on the DateTime)
• Apply-CommitsToServiceTier (-Top / -Since) – find top X commits or commits since a point in the log and apply (using Apply-CommitToServiceTier) them to the service tier
• Apply-CommitToServiceTier – identify the objects modified by this commit and import to / delete from the NAV database as appropriate
• Checkout-GitBranch
  • Pop a list of branches (including remote) using Out-GridView for the developer to select the target branch
  • Identify the objects that are different between the current branch/commit and the branch/commit to checkout. Import to / delete from the NAV database as appropriate

Using an appropriate combination of the above we can always keep the NAV database and the Git working directory in sync. This provides some really powerful flexibility:

• If I want to test the code someone is including in a pull request I can just checkout that remote branch and test in my local database. I can then easily switch back to what I was doing on my own branch
• I can create as many local branches as I like for separate tasks and easily flick between them knowing that the NAV database doesn’t have any unrelated changes hanging around

Dynamics 365 Business Central

A lot of the above has now been rendered obsolete with Dynamics 365 Business Central as we are moving away from working with NAV databases.

The learning curve has, however, been invaluable as we continue to rely heavily on Git and Azure DevOps in our development of v2 apps for Business Central.

We’ll wrap up this series with some concluding thoughts in the next post.