Managing Business Central Development with Git: Branches

Obligatory Preamble

I wasn’t really intending to write this post. If you want training materials for learning the basic concepts of Git then there is tonnes of great free content around on blogs and YouTube channels. I was going to share some thoughts about our branching strategy but thought I’d write a little about manipulating branches first.

Amble

When I was new to Git and trying to establish whether it was worth the time and effort migrating from TFVC I heard a lot about branches. “Migrate to Git” they said. “Branching is so cheap”. What on earth does that mean?

In TVFC, creating a branch of your source code involves creating an entire copy of the working folder, with all of its contents. Conceptually this is nice and easy. If you want to work in another branch then you work in another folder. Changes in different branches are isolated from each other in separate folders.

In performance terms though, not so great i.e. “expensive”. Especially when we were working with 5K+ CAL object text files. Creating a copy of all those files and downloading from the server took some time. Visual Studio would complain that I had more than 100K files under source control – which should be OK…but do you really need all of these it would complain?

Git’s approach to branches is very different. You can think of them as just labels that point at a given commit in the graph. Creating a new branch is just a case of creating a new label, a tiny new text file in the .git directory.

* 825e0ac (HEAD -> feature/some-great-new-feature) Vehicles
* 45494f5 Bookings
* d80589c Ranger table and page
* 440e851 Animal table and page
| * 1225ee5 (bug/some-bug-fix) Fix sales order posting bug
|/
| * 5025f76 (feature/new-field-on-sales-docs) Add field to Order Confirmation
| * 367faab Set field on customer validation
| * 91a9252 Add field to Sales Header
|/
* 3894d1a (origin/master, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

There are 12 commits in this repository. Each commit is a snapshot of the status of the entire source code – the accumulation of all the changes up to that point. The four (local) branches are just pointers to different places in the graph. HEAD just indicates the point at which a new commit will be added i.e. this commit will become the parent of the next commit you make.

Conceptually a little harder to get your head around – but so much more elegant and powerful. You have a single working folder, the contents of which reflect the branch that is currently checked-out. You can quickly and easily create, delete, merge and move branches around. It’s “cheap”.

Cheap Branching

Being able to create branches so easily allows you to change the way you work. Need to fix a bug? Create a branch. Working on a new feature? Create a branch. Want to experiment with some proof of concept? Create a branch. Because you can – and once you have you know that your changes are safely isolated from each other.

I can’t say it better than the man himself. If you haven’t seen this video of Linus Torvalds presenting Git at Google then I recommend it. You’ll need to see past his somewhat sarcastic demeanour in this talk – but I’m British, I’ve had a lot of practice.

Branches are the building blocks of pull requests (merge code from this source branch into this target branch). If you’re already in the habit of creating local branches then pushing those branches and creating pull requests is an easy extension to your process. I’ve said it before – pull requests have been the best improvement in our development process and was the most compelling reason for us to migrate to Git in the first place.

Manipulating Branches

Seeing that branches are just labels pointing at different commits in the history of the repository we can easily move them around.

Let’s walk through an example. Say I’ve got a feature branch in progress and I find and fix a bug. I’m in the middle of developing the new feature so just commit the bug fix alongside whatever else I’m working on. That becomes the latest commit in the graph.

* c123f06 (HEAD -> feature/some-great-new-feature) The bug fix
* 825e0ac Vehicles
* 45494f5 Bookings
* d80589c Ranger table and page
* 440e851 Animal table and page
* 3894d1a (origin/master, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

The problem is we need to get that bug fix out to a customer. The feature isn’t ready to be merged into master and we can’t wait. With hindsight I should have started a new branch to commit the bug fix to.

We can sort that out with cherry-pick and reset like this.

git branch bug/bug-fix master
git checkout bug/bug-fix
git cherry-pick c123f06

Create a new branch called “bug/bug-fix” pointing at the same commit that the master branch currently points to. Checkout that branch and cherry-pick the commit with hash c123f06. That isolates the bug fix into its own branch and I can create a pull request and merge it separately to the feature development. Great. Except, the bug fix is still in the feature branch. Here’s the graph:

* cef4f31 (HEAD -> bug/bug-fix) The bug fix
| * c123f06 (feature/some-great-new-feature) The bug fix
| * 825e0ac Vehicles
| * 45494f5 Bookings
| * d80589c Ranger table and page
| * 440e851 Animal table and page
|/
* 3894d1a (origin/master, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

We learnt last time that one solution to the problem would be to interactively rebase the feature branch on top of master and remove the bug fix commit from the rebase script.

pick 440e851 Animal table and page
pick d80589c Ranger table and page
pick 45494f5 Bookings
pick 825e0ac Vehicles
pick c123f06 The bug fix <-- YOU COULD REMOVE THIS LINE FROM THE SCRIPT

Reset

Alternatively you could use reset. Resetting a branch allows you to force it to point at a different commit. Remember, a branch is just pointing to a commit. It can point somewhere else if you want.

git checkout feature/some-great-new-feature
git reset 825e0ac

This will check out the feature branch and force it to point to commit 82530ac. You’ll notice that it leaves the changes between the commit it has come from and its new commit in the working folder. If you don’t want that you can add –hard to the command. That will tell Git to force the branch to point to the new commit and to hell with any consequences.

Resetting to a Forced Push

Another scenario you might want to reset is when a colleague has force pushed some changes to a branch. Perhaps they’ve rebased the branch and force pushed the changes to the server. Now you’ve got a local copy of the branch that no longer matches the remote copy. Here’s an example:

* f48d506 (origin/feature/some-great-new-feature) Animal table and page
* 9e96653 Vehicles
* d6dbbaf Bookings
* 473781f Ranger table and page
| * 825e0ac (HEAD -> feature/some-great-new-feature) Vehicles
| * 45494f5 Bookings
| * d80589c Ranger table and page
| * 440e851 Animal table and page
|/
* 3894d1a (origin/master, origin/HEAD, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

My copy of feature/some-great-new-feature is shown in the middle of the graph (commit 825e0ac). Meanwhile a colleague has reordered the commits and force pushed. The remote branch origin/feature/some-great-new-feature is now pointing at commit f48d506. I’m not going to be able to push any changes to the branch while my repository is in this state.

This kind of disruption is why you should be careful force pushing your changes to the server – but is sometimes necessary. If I’m confident that I’m not going to lose any work, all I want to do is force my local branch to point to the same commit as the server. We’ve just learnt that reset will do that.

If you want to preserve your changes locally – just in case you do have something locally that isn’t on the server – you can just create a new branch at the same point. When you’re sure you don’t need those commits you can delete that branch.

git checkout feature/some-great-new-feature
git branch backup feature/some-great-new-feature
git reset origin/feature/some-great-new-feature --hard

After running those commands the graph looks like this. My local feature branch now matches the server and I’ve got a new backup local branch which I can always check out if I need to.

* f48d506 (HEAD -> feature/some-great-new-feature, origin/feature/some-great-new-feature) Animal table and page
* 9e96653 Vehicles
* d6dbbaf Bookings
* 473781f Ranger table and page
| * 825e0ac (backup) Vehicles
| * 45494f5 Bookings
| * d80589c Ranger table and page
| * 440e851 Animal table and page
|/
* 3894d1a (origin/master, origin/HEAD, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Next Time

That’s a few concepts about managing branches, amending history, rebasing and cherry-picking. Next time we’ll combine some of these concepts to discuss managing a Business Central app in all its different flavours, for different versions of BC and the branching strategy that we currently use and why.

Managing Business Central Development with Git: Rebasing

This is part two of a series about using Git to manage your Business Central development. This time – rebasing. It seems that rebasing can be something of a daunting subject. It needn’t be. Let’s start with identifying the base of a branch before worrying about rebasing.

Example Repo

Imagine this repository where I’ve created a new branch feature/new-field-on-sales-docs to do some development.

* 445e3e1 (HEAD -> feature/new-field-on-sales-docs) Add field to Order Confirmation
* fddf9fb Set DataClassification
* 176af2d Set field on customer validation
* 3cd4889 Add field to Sales Header
* 3894d1a (origin/master, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

We can consider that the base of the feature branch is where it diverges from the master branch. In this example commit 3894d1a is the base (“Correct typo in caption”). Simple. Now a more complex example:

* 412ce8f (HEAD -> bug/some-bug-fix) Fixing a bug in the bug fix
* 7df90bf Fixing a bug
| * d88a322 (feature/another-feature) And some more development
| * 0d16a39 More development
|/
| * 445e3e1 (feature/new-field-on-sales-docs) Add field to Order Confirmation
| * fddf9fb Set DataClassification
| * 176af2d Set field on customer validation
| * 3cd4889 Add field to Sales Header
|/
* 3894d1a (origin/master, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

I’ve got three branches on the go for a couple of new features and a bug fix. Follow the lines on the graph (created with git log –oneline –all –graph) and notice that they all diverge from master at the same commit as before. That is the base of each of the branches.

Now imagine that the bug fix is merged into the master branch – it was some urgent fix that we needed to push out to customers. I’ve merged the bug fix branch, deleted it and pushed the master branch to the server.

git checkout master
git merge bug/some-bug-fix
git push
git branch bug/some-bug-fix -d

The graph now looks like this:

* 412ce8f (HEAD -> master, origin/master) Fixing a bug in the bug fix
* 7df90bf Fixing a bug
| * d88a322 (feature/another-feature) And some more development
| * 0d16a39 More development
|/
| * 445e3e1 (feature/new-field-on-sales-docs) Add field to Order Confirmation
| * fddf9fb Set DataClassification
| * 176af2d Set field on customer validation
| * 3cd4889 Add field to Sales Header
|/
* 3894d1a Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Merge Commit vs Rebase Example

Now for an example of what rebasing is and why you might want to use it.

Despite what was happened to the master branch notice that the feature branches still diverge from the master branch at the same commit. They still have the same base. This is one reason you might want to consider rebasing. If I was to merge the feature/another-feature branch into master now I would create a merge commit. Like this:

* 44c19a0 (HEAD -> master) Merge branch 'feature/another-feature'
|\
| * d88a322 (feature/another-feature) And some more development
| * 0d16a39 More development
* | 412ce8f (origin/master) Fixing a bug in the bug fix
* | 7df90bf Fixing a bug
|/
| * 445e3e1 (feature/new-field-on-sales-docs) Add field to Order Confirmation
| * fddf9fb Set DataClassification
| * 176af2d Set field on customer validation
| * 3cd4889 Add field to Sales Header
|/
* 3894d1a Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

The graph illustrates that the master branch and the feature branch diverged before being merged back together. An alternative solution would be to rebase the feature branch onto the master branch. What does that mean?

Git will identify the point at which the feature branch and the target branch (master in this case) diverged. This is commit 3894d1a as noted above. It will then rewind the changes that have been made since that point and replay them on top of the target branch.

git checkout feature/another-feature
git rebase master

First, rewinding head to replay your work on top of it…
Applying: More development
Applying: And some more development

And now the graph shows this

* ac25a75 (HEAD -> feature/another-feature) And some more development
* 8db81ff More development
* 412ce8f (origin/master, master) Fixing a bug in the bug fix
* 7df90bf Fixing a bug
| * 445e3e1 (feature/new-field-on-sales-docs) Add field to Order Confirmation
| * fddf9fb Set DataClassification
| * 176af2d Set field on customer validation
| * 3cd4889 Add field to Sales Header
|/
* 3894d1a Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

The two commits that comprised the feature branch have been moved to sit on top of the master branch. It’s as if the development on the feature had been started after the bug fix changes had been made.

Notice that the commit ids are different – due to how Git works internally – but the effect is the same in both cases. The version of the code at the top of the log contains the changes for the both the bug fix and the new feature.

I won’t discuss the pros and cons of either approach. Rebasing keeps the history neater – all the commits line up in a straight line. Merge commits reflect what actually happened and the order in which changes were made. There are plenty of proponents of both approaches if you want to follow the subject up elsewhere.

Interactive Rebasing

In the previous post I was discussing the value of amending commits so that they tell the story of your development. With git amend we can edit the contents and/or commit message of the previous commit.

Remember that rebasing identifies a series of commits and replays them onto another commit. That’s useful for moving commits around. It is also very useful in helping to create the story of your development. Let me simplify the example again to show you what I mean.

* 445e3e1 (feature/new-field-on-sales-docs) Add field to Order Confirmation
* fddf9fb Set DataClassification
* 176af2d Set field on customer validation
* 3cd4889 Add field to Sales Header
* 3894d1a (HEAD -> master, origin/master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Look at commit fddf9fb Set DataClassification. I added a new field to the Sales Header table but forgot to set the DataClassification property so I’ve gone back and added it separately. That kinda sucks. Other developers don’t need to know that. It’s an unnecessary commit that will only make the history harder to read when we come back to it in the future.

But there’s a problem. I can’t amend the commit because I’ve committed another change since then. Enter interactive rebasing.

git checkout feature/new-field-on-sales-docs
git rebase master -i

This tells Git to identify the commits from the point at which the feature diverges from master, rewind them and then apply them on top of master again. In itself, the command will have no effect as we’re replaying the changes on top of the branch they are already on.

Adding the -i switch runs the command in interactive mode. You’ll see something like this in your text editor.

pick 3cd4889 Add field to Sales Header
pick 176af2d Set field on customer validation
pick fddf9fb Set DataClassification
pick 445e3e1 Add field to Order Confirmation

# Rebase 3894d1a..445e3e1 onto 445e3e1 (4 commands)
#
# Commands:
# p, pick <commit> = use commit
# r, reword <commit> = use commit, but edit the commit message
# e, edit <commit> = use commit, but stop for amending
# s, squash <commit> = use commit, but meld into previous commit
# f, fixup <commit> = like "squash", but discard this commit's log message
# x, exec <command> = run command (the rest of the line) using shell
# b, break = stop here (continue rebase later with 'git rebase --continue')
# d, drop <commit> = remove commit
# l, label <label> = label current HEAD with a name
# t, reset <label> = reset HEAD to a label
# m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
# .       create a merge commit using the original merge commit's
# .       message (or the oneline, if no original merge commit was
# .       specified). Use -c <commit> to reword the commit message.
#
# These lines can be re-ordered; they are executed from top to bottom.
#
# If you remove a line here THAT COMMIT WILL BE LOST.
#
# However, if you remove everything, the rebase will be aborted.
#
# Note that empty commits are commented out

You can think of this as a script that Git will follow to apply the changes on top of the target branch. Read it from top to bottom (unlike the Git log which is read bottom to top).

The script contains the four commits that exist in the feature branch but not in the master branch. By default each of those commits will be “picked” to play onto the target branch.

You can read the command help and see that you can manipulate this script. Reorder the lines to play the commits in a different order. Remove lines altogether to remove the commit. “Squash” one or more commits into a previous line. This is what we want.

I want to squash the “Add field to Sales Header” and “Set DataClassification” commits together. In future it will look as if I’ve made both changes at the same time. Great for hiding my ineptitude from my colleagues but also for making the history more readable. I’ll change the script to this:

pick 3cd4889 Add field to Sales Header
fixup fddf9fb Set DataClassification
pick 176af2d Set field on customer validation
pick 445e3e1 Add field to Order Confirmation

and close the text editor. Git does the rest and now my graph looks like this:

* 5025f76 (HEAD -> feature/new-field-on-sales-docs) Add field to Order Confirmation
* 367faab Set field on customer validation
* 91a9252 Add field to Sales Header
* 3894d1a (origin/master, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Panic Button

Rebasing takes a little practice to get used to. You might want to include the -i switch every time you rebase to start with to check which commits you are moving around.

It isn’t uncommon to run into some merge commits when playing a commit in a rebase. You’ll get something like this alarming looking message

First, rewinding head to replay your work on top of it…
Applying: Add field to Sales Header
Applying: Set field on customer validation
Applying: Add field to Order Confirmation
Using index info to reconstruct a base tree…
Falling back to patching base and 3-way merge…
CONFLICT (add/add): Merge conflict in 173626564
Auto-merging 173626564
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch' to see the failed patch
Patch failed at 0003 Add field to Order Confirmation
Resolve all conflicts manually, mark them as resolved with
"git add/rm ", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

I’m not a fan of anything that has the word CONFLICT in caps…

VS Code does a nice job of presenting the content of the target branch the “current change” and the changes that are being played as part of the rebase the “incoming change”. Resolve the conflict i.e. edit the conflicted file to correctly merge the current and incoming changes and stage the file.

Once you’ve done that you can continue the rebase with git rebase –continue

If all hell breaks loose and you need to smash the emergency glass you can always run git rebase –abort, breath into a brown paper bag and the repo will be returned to the state it was in before the rebase.

Managing Business Central Development with Git: Amending History

Preamble

This is the start of a series of posts about managing AL development with Git. I don’t profess to be a Git expert and much of what I write about will not exclusively apply to Business Central development. This is a collection of approaches I’ve found to be useful when it comes to managing our source code. Take what you will, discard the rest, vociferously argue with me if you feel strongly enough about it.

Preamble over. Let’s get on with it.

(Re)Writing History

My introduction to source control was using TFVC (more here). As a centralised source control system when you check code in it is immediately pushed to the server. All the changes that anyone pushes make a nice, neat, straight line. Check-ins are given a changeset number. Those numbers are unique, always increase and can never be changed. History has been written.

Some changesets in the history of a branch in TFVC

Stands to reason. We can’t go back and change the past. But what if we could…?

You can use Git like this if you want. Make a change, commit the change, make a change, commit the change. Keep committing in a straight line and keep your history really simple.

* cd03362 (HEAD -> master) Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Unlike TFVC you have to push those commits to the server before anyone can see them. Do that on a regular basis and make sure your colleagues are pulling your changes before they commit theirs and not much can go wrong.

That’s fine as far as it goes, but it’s not particularly elegant. What about when you make another commit correcting a typo in the caption? (Reading the history from bottom to top)

* 1ee22a6 (HEAD -> master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Now we’ve got two commits in the history of the project just to add a caption and get the caption correct. With TFVC you’re stuck with it, but with Git, we’ve got complete control over the history of the project.

Tell a Story

Having control over the history of the project ought to make us think differently about it. What is the history for anyway? It’s to help other developers, including our future selves, understand what changes have been made to the code and why they have been made. The best way I’ve heard this described is that we can use the commits to tell the story of the changes that we’ve made.

When you were working on this feature what changes did you make? What code did you add or remove?

The reality might be something like:

  1. Added a field to the customer table
  2. Added an OnInsert trigger to populate the new field
  3. Added a missing caption
  4. Corrected a typo in the caption
  5. Added the field to the customer card
  6. Realised the field was in the wrong group, moved it
  7. Added a missing application area
  8. Realised I should have included a suffix to the field name, renamed the field

Development can be messy. We make mistakes and fix them as we go. But is that history easy to read? Do other developers care about all the steps in the process? Does future you need to reminded of all those mistakes? No. We can do better than that.

Terminal

From here on in we’re going to use a terminal – command prompt / bash / PowerShell to manipulate the history of the repository. Don’t be intimidated – it’s fine with a little practice. I’d recommend a combination of PowerShell and posh-git module – its tab completion and status in the prompt makes life easier.

Incidentally, to show the graphs of the history in this post I’ve used:

git log --graph --oneline --all

i.e. show the log (history) of the branch as a graph with each commit on a single line.

git commit –amend

The first tool we’ve got to put some of this mess right is the –amend switch to the commit command. Perfect for when you realise you’d made a mistake with the latest commit. You’ve found a typo or forgotten to include some changes that should have been made with it.

Stage the changes that you want to include with the previous commit (using git add or VS Code or some other UI tool). Rather than committing them in the UI switch to a terminal and type git commit –amend

Amending a commit

Git will open a text file with the commit comment at the top and details of the changes which are included in the commit underneath. Change the commit comment if you want and close the file. You’ll have selected the text editor you want to use when installing Git. If you can’t remember doing that then you’ll find out what you chose now. You can change the editor in Git’s config if you like.

Congratulations. You just rewrote the history of the repo. You can do that perfectly safely on commits that are only in your local copy of the repository.

Only Share Your Changes When You’re Ready

This is one of the big benefits of a distributed source control system like Git. It’s your copy of the repo. You can do whatever you like to it without affecting anyone else until you are ready. Make mistakes. Muck about with different ideas. Start again. Redesign. Whatever.

When you are happy with the changes that you’ve made and the story that the commits tell – push those changes to the server and let your colleagues get their hands on them.

Different Versions of History

Before going on to describe other methods for manipulating history it is probably responsible to briefly discuss the consequences of rewriting commits that have been already been pushed to the server.

If this is a commit that has already been pushed to the server you should know that your history no longer matches the history on the remote.

The graph will end up looking something like this. My local copy of the commit has a different commit hash (c1152b2) to the remote copy (aea8ffa) – usually, but not necessary, called “origin”. Notice the posh-git prompt indicates this with the up and down arrows. 1 commit ahead of master, 1 commit behind master.

* c1152b2 (HEAD -> master) Correct typo in caption
| * aea8ffa (origin/master) Correct typo in caption
|/
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card
C:\Users\james.pearson.TECMAN\Desktop\GitDemo [master ↓1 ↑1]>

While this is the case I won’t be able to push my changes to the remote. This is what happens when I run git push

! [rejected] master -> master (non-fast-forward)
error: failed to push some refs to 'C:\users\james.pearson.TECMAN\Desktop\GitDemo-Origin.git'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'git pull …') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

Updates were rejected. There is a danger that some commits on the server will be lost if my copy of the master branch is pulled as is.

The advice is to pull the commits that are on the server and incorporate them into my local copy before I push my changes again. Usually good advice. Only, in this case I want that change to be lost. The commit that is in the server’s copy but not mine is the commit that I want to overwrite. In which case, I can safely force my changes onto the server with git push -f

Before forcing your changes make sure that you know which changes are going to be lost i.e. everything from the point at which the graph diverges.

If that all sounds a little daunting, don’t do it. Practice amending local commits first and getting them into shape before you push them to the server. Being able to confidently manipulate the history of the repo with a few key commands will prove an invaluable tool in your own work and especially as you collaborate on the same code with others.

Next up, interactive rebasing.

An Approach to Package Management in Dynamics 365 Business Central

TL;DL

We use PowerShell to call the Azure DevOps API and retrieve Build Artefacts from the last successful build of the repository/repositories that we’re dependent on.

Background

Over the last few years I’ve moved into a role where I’m managing a development team more than I’m writing code myself. I’ve spent a lot of that time looking at tools and practices in the broader software development community. After all, whether you’re writing C/AL, AL, PowerShell or JavaScript it’s all code and it’s unlikely that we’ll face any challenges that haven’t already been faced in one way or another in a different setting.

In that time we’ve introduced:

Package Management

The next thing to talk about is package management. I’ve written about the benefits of trying to avoid dependencies between your apps before (see here). However, if app A relies on app B and you cannot foresee ever deploying A without B then you have a dependency. There is no point trying to code your way round the problems that avoiding the dependency will create.

Accepting that your app has one or more dependencies – and most of our apps have at least one – opens up a bunch of questions and presents some interesting challenges.

Most obviously you need to know, where can I get the .app files for the apps that I am dependent on? Is it at least the minimum version required by my app? Is this the correct app for the version of the Dynamics NAV / Dynamics 365 Business Central that I am developing against? Are the apps that I depend on themselves dependent on other apps? If so, where do I get those from? Is there another layer of dependencies below that? Is it really turtles all the way down?

These are the sorts of questions that you don’t want to have to worry about when you are setting up an environment to develop in. Docker gives us a slick way to quickly create disposable development and testing environments. We don’t want to burn all the time that Docker saves us searching for, publishing and installing app files before we can start work.

This is what a package manager is for. The developer just needs to declare what their app depends on and leave the package manager to retrieve and install the appropriate packages.

The Goal

Why are we talking about this? What are we trying to achieve?

We want to keep the maintenance of all apps separate. When writing app A I shouldn’t need to know or care about the development of app B beyond my use of its API. I just need to know:

  • The minimum version that includes the functionality that I need – this will go into my app.json file
  • I can acquire that, or a later, version of the app from somewhere as and when I need it

I want to be able to specify my dependencies and with the minimum of fuss download and install those apps into my Docker container.

We’ve got a PowerShell command to do just that.

Get-ALDependencies -Container BCOnPrem -Install

There are a few jigsaw pieces we need to gather before we can start putting it all together.

Locating the Apps

We need somewhere to store the latest version of the apps that we might depend upon. There is usually some central, public repository where the packages are hosted – think of the PowerShell Gallery or Docker Hub for example.

We don’t have an equivalent repository for AL apps. AppSource performs that function for Business Central SaaS but that’s not much use to us while we are developing or if the apps we need aren’t on AppSource. We’re going to need to set something up ourselves.

You could just use a network folder. Or maybe SharePoint. Or some custom web service that you created. Our choice is Azure DevOps build artefacts. For a few reasons:

  • We’ve already got all of our AL code going through build pipelines anyway. The build creates the .app files, digitally signs them and stores them as build artefacts
  • The artefacts are only stored if all the tests ran successfully which ought to give us more confidence relying on them
  • The build automatically increments the app version so it should always be clear which version of the app is later and we shouldn’t get caught in app version purgatory when upgrading an app that we’re dependent on
  • We’re already making use of Azure DevOp’s REST API for loads of other stuff – it was easy to add some commands to retrieve the build artefacts (hence my earlier post on getting started with the API)

Identifying the Repository

There is a challenge here. In the app.json file we identify dependencies by app name, id and publisher. To find a build – and its artefacts – we need to know the project and repository name in Azure DevOps.

Seeing as we can’t add extra details into the app.json file itself we hold these details in a separate json file – environment.json. This file can have an array of dependency objects with a:

  • name – which should match the name of the dependency in the app.json file
  • project – the Azure DevOps project to to find this app in
  • repo – the Git repository in that project to find this app in

Once we know the right repository we can use the Azure DevOps API to find the most recent successful build and download its artefacts.

I’m aware that we could use Azure DevOps to create proper releases, rather than downloading apps that are still in development. We probably should – maybe I’ll come back and update this post some day. For now, we find that using the artefacts from builds is fine for the two main purposes we use them: creating local development environments and creating a Docker container as part of a build. We have a separate, manual process for uploading new released versions to SharePoint for now.

The Code

So much for the theory, let’s look at some code. In brief we:

  1. Read app.json and iterate through the dependencies
  2. For each dependency, find the corresponding entry in the environment.json file and read the project and repo for that dependency
  3. Download the app from the last successful build for that repo
  4. Acquire the app.json of the dependency
  5. Repeat steps 2-5 recursively for each branch of the dependency tree
  6. Optionally publish and install the apps that have been found (starting at the bottom of the tree and working up)

A few notes about the code:

  • It’s not all here – particularly the definition of Invoke-TFSAPI. That is just a wrapper for the Invoke-WebRequest command which adds the authentication headers (as previously described)
  • These functions are split across different files and grouped into a module, I’ve bundled them into a single file here for ease

(The PowerShell is hosted here if you can’t see it embedded below: https://gist.github.com/jimmymcp/37c6f9a9981b6f503a6fecb905b03672)

function Get-ALDependencies {
Param(
[Parameter(Mandatory=$false)]
[string]$SourcePath = (Get-Location),
[Parameter(MAndatory=$false)]
[string]$ContainerName = (Split-Path (Get-Location) Leaf),
[Parameter(Mandatory=$false)]
[switch]$Install
)
if (!([IO.Directory]::Exists((Join-Path $SourcePath '.alpackages')))) {
CreateEmptyDirectory (Join-Path $SourcePath '.alpackages')
}
$AppJson = ConvertFrom-Json (Get-Content (Join-Path $SourcePath 'app.json') Raw)
Get-ALDependenciesFromAppJson AppJson $AppJson SourcePath $SourcePath ContainerName $ContainerName Install:$Install
}
function Get-ALDependenciesFromAppJson {
Param(
[Parameter(Mandatory=$true)]
$AppJson,
[Parameter(Mandatory=$false)]
[string]$SourcePath = (Get-Location),
[Parameter(Mandatory=$false)]
[string]$RepositoryName,
[Parameter(Mandatory=$false)]
[string]$ContainerName,
[Parameter(Mandatory=$false)]
[switch]$Install
)
foreach ($Dependency in $AppJson.dependencies) {
$EnvDependency = Get-DependencyFromEnvironment SourcePath $SourcePath Name $Dependency.name
$Apps = Get-AppFromLastSuccessfulBuild ProjectName $EnvDependency.project RepositoryName $EnvDependency.repo
$DependencyAppJson = Get-AppJsonForProjectAndRepo ProjectName $EnvDependency.project RepositoryName $EnvDependency.repo
Get-ALDependenciesFromAppJson AppJson $DependencyAppJson SourcePath $SourcePath RepositoryName $RepositoryName ContainerName $ContainerName Install:$Install
foreach ($App in $Apps) {
if (!$App.FullName.Contains('Tests')) {
Copy-Item $App.FullName (Join-Path (Join-Path $SourcePath '.alpackages') $App.Name)
if ($Install.IsPresent) {
try {
Publish-NavContainerApp containerName $ContainerName appFile $App.FullName sync install
}
catch {
if (!($_.Exception.Message.Contains('already published'))) {
throw $_.Exception.Message
}
}
}
}
}
}
}
function Get-AppJsonForProjectAndRepo {
Param(
[Parameter(Mandatory=$true)]
[string]$ProjectName,
[Parameter(Mandatory=$false)]
[string]$RepositoryName
)
$VSTSProjectName = (Get-VSTSProjects | where name -like ('*{0}*' -f $ProjectName)).name
$AppContent = Invoke-TFSAPI ('{0}{1}/_apis/git/repositories/{2}/items?path=app.json' -f (Get-TFSCollectionURL), $VSTSProjectName, (Get-RepositoryId ProjectName $VSTSProjectName RepositoryName $RepositoryName)) GetContents
$AppJson = ConvertFrom-Json $AppContent
$AppJson
}
function Get-DependencyFromEnvironment {
Param(
[Parameter(Mandatory=$true)]
[string]$SourcePath,
[Parameter(Mandatory=$true)]
[string]$Name
)
Get-EnvironmentKeyValue SourcePath $SourcePath KeyName 'dependencies' | where name -eq $Name
}
function Get-EnvironmentKeyValue {
Param(
[Parameter(Mandatory=$false)]
[string]$SourcePath = (Get-Location),
[Parameter(Mandatory=$true)]
[string]$KeyName
)
if (!(Test-Path (Join-Path $SourcePath 'environment.json'))) {
return ''
}
$JsonContent = Get-Content (Join-Path $SourcePath 'environment.json') Raw
$Json = ConvertFrom-Json $JsonContent
$Json.PSObject.Properties.Item($KeyName).Value
}
function Get-VSTSProjects {
(Invoke-TFSAPI Url ('{0}_apis/projects?$top=1000' -f (Get-TFSCollectionURL))).value
}
function Get-RepositoryId {
Param(
[Parameter(Mandatory=$true)]
[string]$ProjectName,
[Parameter(Mandatory=$false)]
[string]$RepositoryName
)
$Repos = Invoke-TFSAPI ('{0}{1}/_apis/git/repositories' -f (Get-TFSCollectionURL), $ProjectName)
if ($RepositoryName -ne '') {
$Id = ($Repos.value | where name -like ('*{0}*' -f $RepositoryName)).id
}
else {
$Id = $Repos.value.item(0).id
}
if ($Id -eq '' -or $Id -eq $null) {
$Id = Get-RepositoryId ProjectName $ProjectName RepositoryName ''
}
$Id
}

An Introduction to Pull Requests in Azure DevOps

An Intro to the Intro

I’ve previously written about our experience with source control and our eventual migration to Git. I said that pull requests in Azure DevOps are awesome and are one of the biggest reasons to consider the switch to Git. In this post we’ll dig a little more into the details of why they are so good and how to use them.

What Are You Trying to Achieve?

Before we start, don’t forget that code review (i.e. pull requests in Git) and source control are tools. They are a means to an end and not an end in themselves.

I get it. We’re developers and typically we love the latest tools and gadgets. We go to a conference and we hear “You should be using… Docker / PowerShell / Agile / Azure DevOps / pair programming / test-driven development / insert some other tech or best practice here…” That’s great, as long as we don’t lose sight of why we should be using them. What are you trying to achieve? What problem do you have that this new tool or practice will alleviate? What will its introduction make more efficient?

Think about how you’d answer those questions. Write them down. Discuss with colleagues. Leave yourself a voice memo. Whatever works. Just make sure you’ve got some idea of how introducing this tool is going to help achieve your team’s goals.

The Goal

OK, let’s start with the goal. Better quality software, delivered faster.

  • Better quality means the code is clear, easy to read and maintain, does what it is supposed to do and doesn’t do more than it is supposed to do
  • Delivered faster means we are able to take a requirement or bug, make the code changes and get them out to our users in a shorter space of time

One of the ways we will work towards that goal is by reviewing code before it is shipped. You might query how adding a review step allows us to deliver faster but consider time that is sometimes wasted going back and forth with a consultant or customer fixing bugs that could have been found during  a code review.

The Process

Before we get stuck into the specifics of pull requests in Azure DevOps, take a minute to think about how you’d want this process to work. Consider the requirements of both the reviewers and the author. This is my list.

  • Clearly identify the code changes that are under review
  • Select one or more colleagues to review the code
  • Allow the reviewers to add comments. It must be clear which line(s) of code the comments are about. Comments must be visible to all reviewers
  • Allow for discussion of particular issues. The author may need to answer questions, reviewers may need to add clarifications to their comments
  • The author must be able to make further code changes to create a new version of the code under review. Reviewers should be able to see the changes that have been made between versions
  • Send notifications to reviewers when a change is made to a review that they are involved in
  • Record when reviewers are satisfied that the changes can be shipped
  • Keep a record of the review after it has been completed so that it can be referred back to, if necessary

Beyond the scope of this post, but related:

  • Run automated tests against the code under review and record the test results
  • Prevent a review from being completed if any associated tests have failed
  • Mandate that code can only be shipped after it has been through a code review

Do you agree with those requirements? What does your current process look like? How many of those points can you tick off? Would you see value in adopting a process that would allow you to tick more, or all, of those points of the list?

Pull Requests

On to the topic at hand. A pull request is the process of merging code changes between branches in Git repositories – or in our scenario between two branches in the same repository.

Pull Request.gif

  • Developer clones the repository to their local machine
  • Create a new local branch to start some new feature e.g. the branch might be called feature/some-new-feature
  • Start developing and committing their changes to that local branch
  • Push local branch to create a copy on the server (usually referred to as origin)
  • Create a pull request to merge the changes from the feature/some-new-feature branch to the master branch
  • Reviewers and author discuss the changes. Author (or another developer) pushes new commits to create an update to the pull request. Repeat as necessary
  • Complete the pull request to merge the changes into the master branch
    • While completing, optionally squash the commits into a new single commit (as shown in the gif)

Creating the Pull Request

You’ve done some work in a new branch in your local repository and have pushed that branch to the server. When you view the branches in Azure DevOps in the browser portal it prompts you to create a pull request for this new branch.

Typically you will be prompted to create a pull request from your new branch (referred to as the “source branch”) into the master branch (the “target branch”). If you follow some workflow that merges your changes into a development / release / some other branch first you can change the target branch and the request will update accordingly.

You will see the code differences between the source and target branches – these are the changes that are under review. If you have already associated the commit(s) in the source branch with work items they will be automatically associated with the pull request. You can manually add or remove work items as well. This provides useful context for the reviewers. Also some might ask, if you don’t have a work item describing the changes you’ve made…why have you changed anything?

Add individual or groups of reviewers and they will receive email notifications that their expertise and opinions are required.

Identifying Changes

PR Identifying Changes.jpg

The pull request shows a tree of folders/files that have been modified. The changes for each file are highlighted on the right. It’s nice and easy for everyone to see the code changes that are included in this pull request. You can also see the work item(s) that are associated with this pull request for a description of the requirements that these changes are designed to meet.

Updates

By default you’ll be looking at the changes that have been made across all updates made to the pull request i.e. all pushes to the source branch since the request has been opened. You can, however, just view changes made in a given update. Imagine you’ve already reviewed the code and given some feedback and the author has made a small change to address your comments. You can select the latest update to only see the latest changes.

PR Update Selection.jpg

Comments

The most impressive thing about the pull request flow is the comments. Highlighting the code that the comment relates to and posting your message creates a new thread which supports:

  • Others posting new messages in context to that thread
  • Tracking the status of the comment (active, resolved, won’t fix)
  • @mentioning colleagues to alert them to something
  • Linking to work items with #work item no.
  • Pasting images and emoji, liking comments
  • Seeing which update the comment refers to
  • Tracking how the code in question has changed between updates

If you have a requirement to get your team reviewing each other’s work and collaborating on code (and if you don’t…really?) then this is a lovely tool to help you do it.

The last point is especially good. If I arrive late to a review and some comments and updates have already been made I am easily able to catch up. I can see the comments that have already been made and the code changes that were made to resolve them.

PR View Original Diff.gif

Notifications

Azure DevOps provides a lot of flexibility to configure how and when you want to be notified about pull requests. You can receive an email when:

  • You are included as a reviewer on a new pull request
  • A new update is created i.e. new commits are pushed to the source branch
  • The request is completed or abandoned
  • A reply is posted to a comment thread that you opened
  • You are @mentioned

In addition to notifications the _pulls view (https://dev.azure.com/organisation/_pulls) provides an overview of the pull requests that you have created or are a reviewer for and their status.

Voting

When you’ve reviewed the code changes you cast your vote on the pull request. The options are: Approve, Approve with suggestions, Wait for author, Reject.

Completing

Once the comments have been commented upon and the votes voted on you can hit the big Complete button. This marks the pull request as being complete and merges its code changes from the source branch into the target branch. With the following options:

  • Complete linked worked items
  • Delete source branch
  • Squash changes into a single, new commit on the target branch

We tend to have all three ticked. If there are a bunch of tiny changes in the source branch e.g. fixing typos then I don’t particularly want to see those in the target branch. Generally we’re happy with all the changes related to the request being grouped into a single commit.

The request, complete with comments, commits and votes is archived and remains on Azure DevOps if you need to refer back to it. Like most things in Azure DevOps you can access them through the REST API as well – as I did the other day to get some stats on how many requests we had completed in 2018.

More

And there is a load more than that as well. Beyond this post, but maybe a topic for another day. I hope the above has been enough to whet your code review appetite to try it out and investigate further.

  • Protecting branches to only allow changes from a pull request (as opposed to pushing commits directly to the branch)
  • Enforcing a minimum number of reviewers and preventing users from reviewing their own changes
  • Enforcing that a build must run – and succeed – before the request can be completed
  • Enforcing that all comments are resolved before completing the request
  • Automatically include certain users or groups as reviewers on specified branches