Managing Business Central Development with Git: Rebasing

This is part two of a series about using Git to manage your Business Central development. This time – rebasing. It seems that rebasing can be something of a daunting subject. It needn’t be. Let’s start with identifying the base of a branch before worrying about rebasing.

Example Repo

Imagine this repository where I’ve created a new branch feature/new-field-on-sales-docs to do some development.

* 445e3e1 (HEAD -> feature/new-field-on-sales-docs) Add field to Order Confirmation
* fddf9fb Set DataClassification
* 176af2d Set field on customer validation
* 3cd4889 Add field to Sales Header
* 3894d1a (origin/master, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

We can consider that the base of the feature branch is where it diverges from the master branch. In this example commit 3894d1a is the base (“Correct typo in caption”). Simple. Now a more complex example:

* 412ce8f (HEAD -> bug/some-bug-fix) Fixing a bug in the bug fix
* 7df90bf Fixing a bug
| * d88a322 (feature/another-feature) And some more development
| * 0d16a39 More development
|/
| * 445e3e1 (feature/new-field-on-sales-docs) Add field to Order Confirmation
| * fddf9fb Set DataClassification
| * 176af2d Set field on customer validation
| * 3cd4889 Add field to Sales Header
|/
* 3894d1a (origin/master, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

I’ve got three branches on the go for a couple of new features and a bug fix. Follow the lines on the graph (created with git log –oneline –all –graph) and notice that they all diverge from master at the same commit as before. That is the base of each of the branches.

Now imagine that the bug fix is merged into the master branch – it was some urgent fix that we needed to push out to customers. I’ve merged the bug fix branch, deleted it and pushed the master branch to the server.

git checkout master
git merge bug/some-bug-fix
git push
git branch bug/some-bug-fix -d

The graph now looks like this:

* 412ce8f (HEAD -> master, origin/master) Fixing a bug in the bug fix
* 7df90bf Fixing a bug
| * d88a322 (feature/another-feature) And some more development
| * 0d16a39 More development
|/
| * 445e3e1 (feature/new-field-on-sales-docs) Add field to Order Confirmation
| * fddf9fb Set DataClassification
| * 176af2d Set field on customer validation
| * 3cd4889 Add field to Sales Header
|/
* 3894d1a Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Merge Commit vs Rebase Example

Now for an example of what rebasing is and why you might want to use it.

Despite what was happened to the master branch notice that the feature branches still diverge from the master branch at the same commit. They still have the same base. This is one reason you might want to consider rebasing. If I was to merge the feature/another-feature branch into master now I would create a merge commit. Like this:

* 44c19a0 (HEAD -> master) Merge branch 'feature/another-feature'
|\
| * d88a322 (feature/another-feature) And some more development
| * 0d16a39 More development
* | 412ce8f (origin/master) Fixing a bug in the bug fix
* | 7df90bf Fixing a bug
|/
| * 445e3e1 (feature/new-field-on-sales-docs) Add field to Order Confirmation
| * fddf9fb Set DataClassification
| * 176af2d Set field on customer validation
| * 3cd4889 Add field to Sales Header
|/
* 3894d1a Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

The graph illustrates that the master branch and the feature branch diverged before being merged back together. An alternative solution would be to rebase the feature branch onto the master branch. What does that mean?

Git will identify the point at which the feature branch and the target branch (master in this case) diverged. This is commit 3894d1a as noted above. It will then rewind the changes that have been made since that point and replay them on top of the target branch.

git checkout feature/another-feature
git rebase master

First, rewinding head to replay your work on top of it…
Applying: More development
Applying: And some more development

And now the graph shows this

* ac25a75 (HEAD -> feature/another-feature) And some more development
* 8db81ff More development
* 412ce8f (origin/master, master) Fixing a bug in the bug fix
* 7df90bf Fixing a bug
| * 445e3e1 (feature/new-field-on-sales-docs) Add field to Order Confirmation
| * fddf9fb Set DataClassification
| * 176af2d Set field on customer validation
| * 3cd4889 Add field to Sales Header
|/
* 3894d1a Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

The two commits that comprised the feature branch have been moved to sit on top of the master branch. It’s as if the development on the feature had been started after the bug fix changes had been made.

Notice that the commit ids are different – due to how Git works internally – but the effect is the same in both cases. The version of the code at the top of the log contains the changes for the both the bug fix and the new feature.

I won’t discuss the pros and cons of either approach. Rebasing keeps the history neater – all the commits line up in a straight line. Merge commits reflect what actually happened and the order in which changes were made. There are plenty of proponents of both approaches if you want to follow the subject up elsewhere.

Interactive Rebasing

In the previous post I was discussing the value of amending commits so that they tell the story of your development. With git amend we can edit the contents and/or commit message of the previous commit.

Remember that rebasing identifies a series of commits and replays them onto another commit. That’s useful for moving commits around. It is also very useful in helping to create the story of your development. Let me simplify the example again to show you what I mean.

* 445e3e1 (feature/new-field-on-sales-docs) Add field to Order Confirmation
* fddf9fb Set DataClassification
* 176af2d Set field on customer validation
* 3cd4889 Add field to Sales Header
* 3894d1a (HEAD -> master, origin/master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Look at commit fddf9fb Set DataClassification. I added a new field to the Sales Header table but forgot to set the DataClassification property so I’ve gone back and added it separately. That kinda sucks. Other developers don’t need to know that. It’s an unnecessary commit that will only make the history harder to read when we come back to it in the future.

But there’s a problem. I can’t amend the commit because I’ve committed another change since then. Enter interactive rebasing.

git checkout feature/new-field-on-sales-docs
git rebase master -i

This tells Git to identify the commits from the point at which the feature diverges from master, rewind them and then apply them on top of master again. In itself, the command will have no effect as we’re replaying the changes on top of the branch they are already on.

Adding the -i switch runs the command in interactive mode. You’ll see something like this in your text editor.

pick 3cd4889 Add field to Sales Header
pick 176af2d Set field on customer validation
pick fddf9fb Set DataClassification
pick 445e3e1 Add field to Order Confirmation

# Rebase 3894d1a..445e3e1 onto 445e3e1 (4 commands)
#
# Commands:
# p, pick <commit> = use commit
# r, reword <commit> = use commit, but edit the commit message
# e, edit <commit> = use commit, but stop for amending
# s, squash <commit> = use commit, but meld into previous commit
# f, fixup <commit> = like "squash", but discard this commit's log message
# x, exec <command> = run command (the rest of the line) using shell
# b, break = stop here (continue rebase later with 'git rebase --continue')
# d, drop <commit> = remove commit
# l, label <label> = label current HEAD with a name
# t, reset <label> = reset HEAD to a label
# m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
# .       create a merge commit using the original merge commit's
# .       message (or the oneline, if no original merge commit was
# .       specified). Use -c <commit> to reword the commit message.
#
# These lines can be re-ordered; they are executed from top to bottom.
#
# If you remove a line here THAT COMMIT WILL BE LOST.
#
# However, if you remove everything, the rebase will be aborted.
#
# Note that empty commits are commented out

You can think of this as a script that Git will follow to apply the changes on top of the target branch. Read it from top to bottom (unlike the Git log which is read bottom to top).

The script contains the four commits that exist in the feature branch but not in the master branch. By default each of those commits will be “picked” to play onto the target branch.

You can read the command help and see that you can manipulate this script. Reorder the lines to play the commits in a different order. Remove lines altogether to remove the commit. “Squash” one or more commits into a previous line. This is what we want.

I want to squash the “Add field to Sales Header” and “Set DataClassification” commits together. In future it will look as if I’ve made both changes at the same time. Great for hiding my ineptitude from my colleagues but also for making the history more readable. I’ll change the script to this:

pick 3cd4889 Add field to Sales Header
fixup fddf9fb Set DataClassification
pick 176af2d Set field on customer validation
pick 445e3e1 Add field to Order Confirmation

and close the text editor. Git does the rest and now my graph looks like this:

* 5025f76 (HEAD -> feature/new-field-on-sales-docs) Add field to Order Confirmation
* 367faab Set field on customer validation
* 91a9252 Add field to Sales Header
* 3894d1a (origin/master, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Panic Button

Rebasing takes a little practice to get used to. You might want to include the -i switch every time you rebase to start with to check which commits you are moving around.

It isn’t uncommon to run into some merge commits when playing a commit in a rebase. You’ll get something like this alarming looking message

First, rewinding head to replay your work on top of it…
Applying: Add field to Sales Header
Applying: Set field on customer validation
Applying: Add field to Order Confirmation
Using index info to reconstruct a base tree…
Falling back to patching base and 3-way merge…
CONFLICT (add/add): Merge conflict in 173626564
Auto-merging 173626564
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch' to see the failed patch
Patch failed at 0003 Add field to Order Confirmation
Resolve all conflicts manually, mark them as resolved with
"git add/rm ", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

I’m not a fan of anything that has the word CONFLICT in caps…

VS Code does a nice job of presenting the content of the target branch the “current change” and the changes that are being played as part of the rebase the “incoming change”. Resolve the conflict i.e. edit the conflicted file to correctly merge the current and incoming changes and stage the file.

Once you’ve done that you can continue the rebase with git rebase –continue

If all hell breaks loose and you need to smash the emergency glass you can always run git rebase –abort, breath into a brown paper bag and the repo will be returned to the state it was in before the rebase.

Managing Business Central Development with Git: Amending History

Preamble

This is the start of a series of posts about managing AL development with Git. I don’t profess to be a Git expert and much of what I write about will not exclusively apply to Business Central development. This is a collection of approaches I’ve found to be useful when it comes to managing our source code. Take what you will, discard the rest, vociferously argue with me if you feel strongly enough about it.

Preamble over. Let’s get on with it.

(Re)Writing History

My introduction to source control was using TFVC (more here). As a centralised source control system when you check code in it is immediately pushed to the server. All the changes that anyone pushes make a nice, neat, straight line. Check-ins are given a changeset number. Those numbers are unique, always increase and can never be changed. History has been written.

Some changesets in the history of a branch in TFVC

Stands to reason. We can’t go back and change the past. But what if we could…?

You can use Git like this if you want. Make a change, commit the change, make a change, commit the change. Keep committing in a straight line and keep your history really simple.

* cd03362 (HEAD -> master) Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Unlike TFVC you have to push those commits to the server before anyone can see them. Do that on a regular basis and make sure your colleagues are pulling your changes before they commit theirs and not much can go wrong.

That’s fine as far as it goes, but it’s not particularly elegant. What about when you make another commit correcting a typo in the caption? (Reading the history from bottom to top)

* 1ee22a6 (HEAD -> master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Now we’ve got two commits in the history of the project just to add a caption and get the caption correct. With TFVC you’re stuck with it, but with Git, we’ve got complete control over the history of the project.

Tell a Story

Having control over the history of the project ought to make us think differently about it. What is the history for anyway? It’s to help other developers, including our future selves, understand what changes have been made to the code and why they have been made. The best way I’ve heard this described is that we can use the commits to tell the story of the changes that we’ve made.

When you were working on this feature what changes did you make? What code did you add or remove?

The reality might be something like:

  1. Added a field to the customer table
  2. Added an OnInsert trigger to populate the new field
  3. Added a missing caption
  4. Corrected a typo in the caption
  5. Added the field to the customer card
  6. Realised the field was in the wrong group, moved it
  7. Added a missing application area
  8. Realised I should have included a suffix to the field name, renamed the field

Development can be messy. We make mistakes and fix them as we go. But is that history easy to read? Do other developers care about all the steps in the process? Does future you need to reminded of all those mistakes? No. We can do better than that.

Terminal

From here on in we’re going to use a terminal – command prompt / bash / PowerShell to manipulate the history of the repository. Don’t be intimidated – it’s fine with a little practice. I’d recommend a combination of PowerShell and posh-git module – its tab completion and status in the prompt makes life easier.

Incidentally, to show the graphs of the history in this post I’ve used:

git log --graph --oneline --all

i.e. show the log (history) of the branch as a graph with each commit on a single line.

git commit –amend

The first tool we’ve got to put some of this mess right is the –amend switch to the commit command. Perfect for when you realise you’d made a mistake with the latest commit. You’ve found a typo or forgotten to include some changes that should have been made with it.

Stage the changes that you want to include with the previous commit (using git add or VS Code or some other UI tool). Rather than committing them in the UI switch to a terminal and type git commit –amend

Amending a commit

Git will open a text file with the commit comment at the top and details of the changes which are included in the commit underneath. Change the commit comment if you want and close the file. You’ll have selected the text editor you want to use when installing Git. If you can’t remember doing that then you’ll find out what you chose now. You can change the editor in Git’s config if you like.

Congratulations. You just rewrote the history of the repo. You can do that perfectly safely on commits that are only in your local copy of the repository.

Only Share Your Changes When You’re Ready

This is one of the big benefits of a distributed source control system like Git. It’s your copy of the repo. You can do whatever you like to it without affecting anyone else until you are ready. Make mistakes. Muck about with different ideas. Start again. Redesign. Whatever.

When you are happy with the changes that you’ve made and the story that the commits tell – push those changes to the server and let your colleagues get their hands on them.

Different Versions of History

Before going on to describe other methods for manipulating history it is probably responsible to briefly discuss the consequences of rewriting commits that have been already been pushed to the server.

If this is a commit that has already been pushed to the server you should know that your history no longer matches the history on the remote.

The graph will end up looking something like this. My local copy of the commit has a different commit hash (c1152b2) to the remote copy (aea8ffa) – usually, but not necessary, called “origin”. Notice the posh-git prompt indicates this with the up and down arrows. 1 commit ahead of master, 1 commit behind master.

* c1152b2 (HEAD -> master) Correct typo in caption
| * aea8ffa (origin/master) Correct typo in caption
|/
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card
C:\Users\james.pearson.TECMAN\Desktop\GitDemo [master ↓1 ↑1]>

While this is the case I won’t be able to push my changes to the remote. This is what happens when I run git push

! [rejected] master -> master (non-fast-forward)
error: failed to push some refs to 'C:\users\james.pearson.TECMAN\Desktop\GitDemo-Origin.git'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'git pull …') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

Updates were rejected. There is a danger that some commits on the server will be lost if my copy of the master branch is pulled as is.

The advice is to pull the commits that are on the server and incorporate them into my local copy before I push my changes again. Usually good advice. Only, in this case I want that change to be lost. The commit that is in the server’s copy but not mine is the commit that I want to overwrite. In which case, I can safely force my changes onto the server with git push -f

Before forcing your changes make sure that you know which changes are going to be lost i.e. everything from the point at which the graph diverges.

If that all sounds a little daunting, don’t do it. Practice amending local commits first and getting them into shape before you push them to the server. Being able to confidently manipulate the history of the repo with a few key commands will prove an invaluable tool in your own work and especially as you collaborate on the same code with others.

Next up, interactive rebasing.

Source Code Management: Conclusions

I stated in the first post in this series that I wasn’t going to offer any advice. I will, however, attempt to draw some conclusions from our experiences and hope that you’ll find them helpful, or at least interesting.

(Not) Migrating to Git

A few months before we trialled Git in earnest as a team I tried it out for myself. I had a look because I’d heard various reasons that we should migrate:

  • “It’s faster” – yes, in my experience all the key operations are faster in Git than TFVC (committing vs. checking-in, cloning vs. getting latest version, viewing differences between versions)
    • Is that a compelling reason in itself to migrate? You can be the judge of that
  • “Microsoft are moving to it themselves” – who cares? Do you have the same requirements as Microsoft?
    • This would only be a valid argument if they stopped supporting TFVC. As far as I can see they are adding support for more version control systems not removing them (the Build system can now retrieve code from Subversion)
  • “VS Code has built in support for Git” – true, which is great.
    • You can add support for TFVC through a VS Code extension published by Microsoft
    • Again, you can decide how important the convenience of having support for Git in your IDE is weighed against other factors
    • Having tried a few GUIs for Git, VS Code is not my personal favourite – Git Extensions is

I decided at the time that we didn’t need to migrate to Git. The benefits didn’t outweigh the challenges in having the team learn a new system and migrate to it in my estimation. This was during the days of us developing in a central NAV development database (more on that here).

Moving to a distributed version control system while we were working in a single development database didn’t seem to make a lot of sense and I figured that our development practice should drive our choice of version control – not the other way round.

Migrating to Git

All of that said, now that we have migrated to Git I can’t imagine going back to TFVC. Some of our key experiences and learning points:

  • Git has a steeper learning curve. Getting your head round cloning the entire repository, how branches work, pushing, fetching and pulling changes – it’s all a little more involved than TFVC
  • It’s worth investing the time to understand the core concepts. I watched many hours of YouTube videos about Git and read lots of blogs – you can use Git as centralised version control (by pushing to the remote every time you commit) but you’re missing most of the power if you do
  • Personally, I forced myself to use the command line rather than a GUI for common tasks – this helped me grasp what the commands were actually doing and how they can be manipulated in different situations. That knowledge will come in handy when someone in your team asks why their rebase has resulted in a conflict and how to fix it
  • We create a lot of branches now, because it’s so easy and because we use pull requests (see below)
  • Git gives you much finer control over the repositories and your commits than TFVC: interactive rebasing, resetting, reverting, cherry-picking, squashing, fixing, amending commits – with a little practice and research (see above) there aren’t really any ways to screw up the repository so badly that you can’t clean it up again
  • The complexity that makes Git harder to get started with also makes it very flexible and powerful
  • Did I mention that pull requests are awesome? The tools to collaborate on code in Azure DevOps have revolutionised our development workflow. We got started with code review in TFVC but moving to Git has allowed us to move on to another level

At the End of the Day

At the end of the day, source code management is a tool to help us turn out working software for our customers. Different dev teams use different systems in different ways. That’s because they have different development practices and procedures.

However, I think it’s fair to say that we’ve found source code management is not a substitute for good process. Implementing it was initially difficult because we weren’t following a consistent, disciplined development process. It was clear that we weren’t going to be able to extract much value from our system until we were.

As we have changed, and sought to improve our process over the years we have changed our system to suit, which feels like the right way round to me.

Source Code Management: Migrating to Git

This is the third post in a series about source code management. You can start here if you haven’t read the others in the series.

There we were, happy as the proverbial Larry, checking our code into TFVC, requesting code reviews, branching, merging, viewing file history, comparing versions, annotating and writing a lot of PowerShell to automate tasks with the VSTS API. We were feeling pretty great about our source code management.

What could possibly induce us to move away from this system?

Developer Isolation

Our standard practice had always been for developers to work in the same development database. We rarely needed to have multiple developers working on the same project at the same time or not on the same objects at least.

As we invested in our products and grew the team we found ourselves overlapping a lot more than had been the case with customer specific development. Dynamics NAV isn’t equipped to handle this particularly well:

  • If two developers have the same object open at the same time, both make changes and then save, whoever saves last wins. There is potential for development to be lost as soon as it has been written
  • NAV does allow you to lock objects – sort of like checking them out in source code management – but we weren’t in the habit of doing it. We wouldn’t consistently lock them when we were working on them or unlock them when we had finished working on them
    • Hardly a fair criticism of NAV you might say – we just didn’t have a discipline for using the lock functionality properly. You may well be right
    • Even so, locking an object prevents anyone else from designing it. What is the point of me carefully splitting some development into discrete tasks so that different developers can work on them in parallel if they are going to trip over each other because they happen to be modifying the same objects?
  • Only one developer can debug on a service tier at any time
    • So add more service tiers to the development server. You could, but it was already becoming a headache trying to manage all the development databases and service tiers that we’d got on our dev server. I didn’t fancy throwing any more fuel on that particular fire
  • When you export the objects from NAV to check-in to source code management you increase the likelihood that mistakes will be made
    • I export objects that I’ve changed which may include changes that you’ve made. If I’m not careful then I’ll check-in your changes as part of my changeset
    • That can be complicated to fix and then merge into another branch. Or I’ll have to revert the changeset and have a redundant pair of changesets etched into the history of the project

The solution to all of these problems was to isolate the developers from each other. We’d each create a local development environment where we could work, test and debug safe in the knowledge that no one else is monkeying with the objects.

We invested time in PowerShell scripts to automate the creation and management of these environments. Once again, the VSTS API and tf.exe were our friends. As a side benefit we’d also limited our reliance on our development server and the single point of failure danger that it had posed.

Life was good again. We could work on separate features and share the code through shelvesets and code review before checking-in. We could create a new environment of a given product for development or testing in a few minutes with our automation.

Branching

Once we’d isolated developers I was more confident defining separate tasks for different developers to work on in parallel. So I did, but as we were still sharing the same branch in TFVC we started to run into a different set of problems.

  • What if the same developer wanted to work on multiple work items at the same time?
    • This was particularly true when they’d finish the first work item and put it in for review. While they were waiting for review they’d want to crack on with the next task
    • Managing their local development environment became difficult – when they start the second task ideally they should work in a database that doesn’t include the changes from the first task
    • Creating an environment per work item – while feasible – isn’t very attractive.
  • Having several code reviews open against the same branch becomes difficult to follow.
    • While we’d try to review and give feedback/approve as quickly as possible there are inevitably some that stick around for a while
    • The reviewing developer wants an environment with the latest stable code and the code under review. When there is an update to the code under review the shelveset must be replaced and downloaded and applied to the database again (a challenge in TFVC in general)

A sensible step to take is to introduce a branch per work item in TFVC. This allows unrelated changes to be isolated from each other and merged into a production branch once the code has been reviewed. I wasn’t thrilled at this prospect.

Branching in TFVC is expensive – in the sense that it is a full copy of the folder that is has been branched from. Even if you’ve got an entire branch downloaded into your workspace when you create another branch from it the new branch is created on the server and you must download it separately. If you want to delete the branch – which we’d want to do once the work item was finished – you need to download the entire branch, then delete your local folder to tell the server to delete the branch.

I now know that we’d stumbled over two of the most compelling reasons to use Git rather than TFVC (Or other distributed version control systems – but as we were already using VSTS Git was the natural alternative).

In Git:

  • Isolated developers are a given. Everyone has their own copy of the whole repository
  • Branching is cheap and easy. So cheap and easy that you are encouraged to do it all the time. It is very simple to isolate changes from each other and merge them back together again at a later date

Git

It is beyond the scope of this post to compare TFVC and Git in any depth – although there will be more in the final post – but these are the key points that led us to trial it and ultimately move all our code to Git repositories.

  • Now that we were working in separate NAV databases our source code was effectively distributed among us – as opposed to centralised in a single development database.
    • This fits the ethos of Git (as a distributed version control system) much better than TFVC (as a centralised version control system)
    • Without realising it at the time we had effectively already moved to distributed version control
  • Git maintains a single working directory with the contents of the current branch – as opposed to a separate folder with a copy of all objects per branch
    • This principle is what makes branching so cheap in Git. Creating a new branch requires no more than the creation of a single text file with a pointer to the contents of that branch
    • This is a far more attractive proposition when it comes to maintaining your local development database. Don’t create a database per branch or confuse yourself trying to work in multiple branches on the same database. Instead create a single database whose objects are updated to reflect the current contents of Git’s working directory (see below). PowerShell
  • Code is shared through branches which are pushed to the central repository
    • Rather than through shelvesets which are necessarily a single, self-contained set of changes which are difficult to update and re-share.
    • Code reviews (pull requests) compare the changes between two branches rather than the changes contained in a single shelveset. I am constantly delighted with the power of this concept and the tools that Azure DevOps (VSTS) provides to support it. Perhaps more of that in another post one day. Pull requests are awesome

Synchronising with Git’s Working Directory

The most important jigsaw piece in our puzzle of adopting Git was to find a smooth way to keep the NAV database and Git’s working directory in sync with one another. If not, we were going to see some unexpected differences when exporting from NAV.

PowerShell came to the rescue and I added a bunch of functions to the module that we develop and use internally. We also use some functions from Cloud Ready Software’s modules – mainly their wrappers for standard functions that make them a little easier to call by just supplying the NAV service tier. The main functions are:

  • Build-DevEnvironmentFromGit – to create the NAV database and service tier. I won’t go into the details of how that works now. Typically we’d do this once per product as we’re going to reuse the same database from now on rather than constantly building and deleting them
  • Start-GitDev – to start the NAV service tier and import the NAV PowerShell modules from the correct build
  • Export-ModifiedObjectsToWorkingTree
    • Export objects (Modified = true) from the NAV database to individual text files in the Git directory
    • Set Modified to false and DateTime to 1st January of the current year (to minimise conflicts on the DateTime)
  • Apply-CommitsToServiceTier (-Top / -Since) – find top X commits or commits since a point in the log and apply (using Apply-CommitToServiceTier) them to the service tier
  • Apply-CommitToServiceTier – identify the objects modified by this commit and import to / delete from the NAV database as appropriate
  • Checkout-GitBranch
    • Pop a list of branches (including remote) using Out-GridView for the developer to select the target branch
    • Identify the objects that are different between the current branch/commit and the branch/commit to checkout. Import to / delete from the NAV database as appropriate

Using an appropriate combination of the above we can always keep the NAV database and the Git working directory in sync. This provides some really powerful flexibility:

  • If I want to test the code someone is including in a pull request I can just checkout that remote branch and test in my local database. I can then easily switch back to what I was doing on my own branch
  • I can create as many local branches as I like for separate tasks and easily flick between them knowing that the NAV database doesn’t have any unrelated changes hanging around

Dynamics 365 Business Central

A lot of the above has now been rendered obsolete with Dynamics 365 Business Central as we are moving away from working with NAV databases.

The learning curve has, however, been invaluable as we continue to rely heavily on Git and Azure DevOps in our development of v2 apps for Business Central.

We’ll wrap up this series with some concluding thoughts in the next post.

Source Code Management: Adopting TFVC

This is the second post in a series about source code management. If you haven’t already read the beginning of the story you can find it here.

We’d realised that we’d outgrown our system of one-developer-per-customer and ad-hoc communication between teams about ongoing changes to objects. We needed some more structure and somewhere safe to keep previous versions of objects.

Selecting a System

We evaluated a couple of systems and decided to go with Team Foundation Version Control (TFVC), hosted in Visual Studio Team Services (recently renamed Azure DevOps). The key factors:

  • It’s hosted in the cloud and therefore always available – including for developers who were working on site (something we tend not to do now but happened a lot at the time)
  • Familiar tools – developers were already using Visual Studio for RDLC report development and occasional C# development
  • Source code management is independent of NAV development
    • It doesn’t matter what version of NAV you are developing in as they all support export to/import from text files
    • It doesn’t matter where the NAV database is, you only need to be able to bring the text files back to the local workspace on your laptop
    • We did evaluate a product where previous versions of objects are saved in the NAV development database – but the thought of recreating a development database from a SQL backup and losing the history made it a non-starter
  • We could see there was a lot of scope for future improvements with work item tracking, testing and a comprehensive API
  • We’d already got some in-house experience with Team Foundation Server in our .Net development team

Our philosophy was:

  • We’d have a VSTS project for each customer project
  • A branch in that project to represent each NAV database
  • We’d check-in all vanilla objects at the start of the project and all modifications thereafter

Implementing TFVC

It was pretty hard.

The benefits are an easy sell – we’ll be able to rollback objects, we’ll know what was changed, when, by whom and link to the change request that also tell us why. We could even go live with some changes and not others, in the same file. What’s not to love about that? The practice is a little trickier.

The mindset change was the key thing to try and get right. We were so used to being able to make quick changes to objects. Some consultants were comfortable making minor changes – adding fields to forms and pages for example. Support would also make minor changes and put bug fixes in. Some customers would make their own changes as well.

For source code management to be of any use you have to be able to trust the data. For the data to be trustworthy all code changes must be checked-in, no matter how small and seemingly insignificant.

We didn’t want to put too many obstacles in the way of non-developers customising objects for our customers, after all, most likely the customer bought NAV on the strength of our ability to quickly customise it for them in the first place. However, one way or another, all code changes must make their way to TFVC.

The obvious place to start then was to incorporate TFVC into the development process as smoothly as possible:

  1. Write code in NAV test database
  2. Export object(s) to text
  3. Copy into local workspace and check-in to development branch
  4. Repeat 1-3 as necessary
  5. Merge changeset(s) from test branch to live branch
  6. Download merge changeset(s) from live branch
  7. Import into NAV live database, compile, synchronise (comments about how we ought to have been building fobs and importing them instead can be posted below if you really feel the need)

The main challenge were NAV is concerned is keeping the objects in the database and the contents of the workspace in sync. Fortunately, this was also the time that PowerShell support was being added and we were realising its power for other applications.

For step 2 we allow the developers to export all the relevant objects to a single text file and split them into separate files, one per object, named appropriately into a new folder. Identical to Split-NAVApplicationObjectFile but as a custom function so that we can support forms and dataports.

We also have a PowerShell function for step 6. The developer enters the required changeset ID and PowerShell uses the API to collect all the files modified in that changeset, with contents as at that changeset version, and join them into a single text file to import into the target database. Deleted files are included with a Version List of DELETEME and a message is popped to the developer asking them to manually delete those objects from Object Designer.

Ongoing

Some of the concepts and terms were new to most of us and took a while to get the hang of. Branching, merging, changesets, conflicts, server version, workspace version etc. but nothing that the team can’t get used to with some patience and practice.

Once we were comfortable with the development process and used to crafting sensible, self-contained changesets we had the foundation to start reviewing some code. What a good day that was.

Over time we’ve used the VSTS API and tf command line commands to automate many tasks:

  • Building an environment (database and service tier) for a given TFVC branch
  • Creating deltas of changes between objects in one branch and applying those deltas to objects in another branch
  • Getting the latest version from a given branch of objects that exist in a given folder
  • Creating deltas of the changes between two branches and packaging them into a (v1) extension
  • …a bunch of other cool stuff that isn’t really the point of this post

Suffice to say I can’t imagine us not using source code management now. All of the above only becomes possible once you’re code is checked-in and you can trust its content.

Git

Eventually we reached a point where we started coming up against short-comings of TFVC. We were struggling to use TFVC to accommodate changes in the way that we wanted to work as a team and stared to learn about Git.

I’ll discuss why we decided to move to Git and why it might (or might not) be right for you in the next post…