Tip: Remove-BranchesWithUpstreamGone

Wait, I Thought I Deleted That Branch?

One of the things that I found counter-intuitive when I was getting started with Git is that when branches are deleted on the server they are still present in your local repository, even after you have fetched from the server.

We typically delete the source branch when completing a pull request, so this happens a lot. Usually, once the PR has been completed I want to:

  1. remove the reference to the deleted remote branch
  2. remove the corresponding local branch

Removing Remote Reference

The reference to the remote branch is removed when you run git fetch with the prune switch.

git fetch --prune
Fetching origin
From <remote url>
- [deleted] (none) -> origin/test

Removing Local Branches

Local branches can be removed with the git branch command. Adding -d first checks for unmerged commits and will not delete the branch if there are any commits which are only in the branch that is being deleted. Adding -D overrides the check and deletes the branch anyway.

git branch -d test
Deleted branch test (was <commit hash>)

Remove-BranchesWithUpstreamGone

I’ve added a couple of PowerShell functions to my profile file – which means they are always available in my terminal. If I’m working on an app and I know that some PR’s have been merged I can clean up my workspace running Remove-BranchesWithUpstreamGone in VS Code’s terminal.

As a rule, I don’t need to keep any branches which used to have a copy of the server, but don’t any more (indicated by [gone] in the list of branches). Obviously, local branches which have never been pushed to the server won’t be deleted.

function Remove-BranchesWithUpstreamGone {
  (Get-BranchesWithUpstreamGone) | ForEach-Object {
    Write-Host "Removing branch $_"
    git branch $_ -D
  }
}

function Get-BranchesWithUpstreamGone {
  git fetch --all --prune | Out-Null
  $Branches = git branch -v | Where-Object { $_.Contains('[gone]') }
  $BranchNames += $Branches | ForEach-Object {
    if ($_.Split(' ').Item(1) -ne '') {
      $_.Split(' ').Item(1)
    }
    else {
      $_.Split(' ').Item(2)
    }
  }

  $BranchNames
}

Tip: Octopus Merges in Git

If you occasionally glance at my blog you might have noticed that I am a big fan of pull requests as served up by Azure DevOps (exhibit A). I briefly described our typical branching strategy here, including how and why we use pull requests.

I love it. Writing, testing and reviewing discrete pieces of development independently of one another helps spread the work around the team and prevent work getting blocked by some unrelated development in the same project.

However, occasionally we’ll have several pull requests open for the same project which I want to publish to some testing environment (either a local Docker container or SaaS sandbox) all together. Maybe it made sense to develop those work items separately but it’s hard to test them in isolation.

I tend to create a new local branch called testing or something equally banal, merge all the changes into that branch and publish. Here’s where the octopus comes in. We usually use git merge to merge a single other branch into the current branch e.g.

git checkout master
git merge release/1.2.0 --ff-only

To merge the commits in the release branch into the master branch (but only if it is possible to fast-forward merge).

Git doesn’t restrict you to a single other branch though. You can add as many as you like. Say we’ve got 3 feature branches on the go which I want to push to a SaaS sandbox for a consultant to test.

git checkout -b testing
git merge feature/one feature/two feature/three

This will create and checkout a new testing branch and then merge the three feature branches into it. Octopuses for the win.

Getting Started with Git Hooks

Git hooks have been on my “to mess about with and learn a little some day” list for a while. It’s the old conundrum: I might use them if I knew what they could do, but I’m not going to learn about them until I’ve got a use for them. Chickens-and-eggs for developers.

The Problem

Until recently, that is. I wondered:

What is the best way to save a different launch configuration in VS Code for each branch in my repo?

Why would I want to do that? We have branches for versions of our apps compatible with BC14, 15 and 16. I’ve got separate Docker containers of each of those versions to develop and test against. I need to change the launch config when I want to test in another container. Not a great hardship admittedly, but it would be nice to automate somehow.

Source Control?

I could of course just source-control the launch.json file and commit the relevant changes to each branch. I don’t like that though. We ignore the .vscode folder in .gitignore. To my mind anything that happens in that folder is for the benefit of the local development environment. No one else cares what my launch config is – they’ll be pushing code to their own Docker containers or to SaaS sandboxes.

I suppose you could have some local commits that include the launch.json file that you don’t push to the server. Sounds horrible to manage though. Don’t do that.

So we want something that is related to source control – we want a launch config per branch – but doesn’t involve actually committing the config file. Enter hooks.

Hooks

Hooks are points in the Git workflow at which you can execute a custom script. Some are executed server-side, some client-side. You can read the documentation at https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks

When a Git repo is initialised a .git folder is created which contains all the version history of the repo and Git’s guts. If you’re curious to learn more about how Git actually works have a browse through these folders. You might want to load up a presentation on the subject as well – like this: https://www.youtube.com/watch?v=ig5E8CcdM9g

You’ll notice that one of the folders is called hooks. This contains some sample hook files which you can take a look through. The .sample file type stops them from actually doing anything but you get a flavour of what’s possible and how you can use them.

Post-Checkout Example

For our example we’re going to want to use the post-checkout hook. This allows us to create a script that Git will execute after a checkout command. (Side note: I tend to use the integrated terminal in VS Code to execute Git commands, including checking out branches but this hook is also executed if you switch branches with the UI).

First, post-checkout isn’t one of the samples that is created when you init a new repo. You can copy the post-update.sample file instead and rename it to post-checkout (no file extension).

Writing the Hook

Okay…so what are we supposed to put in this file? Can we just start typing PowerShell? Erm…no, we can’t. The file looks distinctly Linuxy…*shudder* I’ve got nothing against Linux, I just have next to no idea what I’m doing.

The top line of the file indicates the language that you are going to script in. It’s possible to use Perl, Python, Ruby and a bunch of other stuff, including PowerShell. I won’t pretend to really know what I’m talking about but there is an useful thread on Stack Overflow on the topic.

#!/bin/sh

It is possible to write PowerShell directly into the hook file, but from what I’ve read it seems you need PowerShell Core to run it. I’ve settled for this:

#!/bin/sh

c:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe -File '.git\hooks\post-checkout.ps1' $1 $2 $3

Call powershell.exe to execute the .git\post-checkout.ps1 file and pass it the three parameters that have been received. Great. How do we know that there are three parameters? And what are they? You can read the documentation here: https://git-scm.com/docs/githooks#_post_checkout

The parameters are:

  • The previous HEAD (the commit that we’ve come from)
  • The new HEAD (the commit that we’ve just checked out)
  • A flag to indicate whether a branch was checked out (it has a value of 1 if it was)

Writing the Script

Now we’re getting somewhere.

The plan is:

  • Save the launch configuration as it stands for the branch that we are coming from
  • Find a saved launch configuration for the branch that we have moved to and overwrite launch.json

What Branch Have We Come From?

This is a subtler question than we might have expected. The parameter that Git passes is the hexadecimal hash of the commit that was previously checked out. Some jumble of numbers and letters that is integral to how Git works but probably meaningless to the developer. We need to convert that hash into a branch. Or branches – potentially more than one branch might be pointing at that commit.

git branch --points-at <commit hash>

The above command will give us zero or more branches that are pointing at a given commit. Note that the current branch is denoted with an asterisk which we’ll need to trim off the front of the name.

Which Branch Have We Moved To?

We can find the name of the new, current branch with:

git branch --show-current

Finally, branch names can contain characters that cannot be used in filenames. Specifically we use forward-slashes a lot in our branch names. I’ve got a Convert-BranchToFileName function to clean it up.

Putting all the jigsaw pieces together this is the content of my post-checkout.ps1 file (which is in the .git\hooks directory).

[CmdletBinding()]
param (
    $PreviousHead,
    $NewHead,
    $BranchCheckout
)

function Convert-BranchToFileName() {
    param(
        $branch
    )

    $filename = $branch
    if ($branch.StartsWith('*')) {
        $filename = $branch.Substring(2)
    }

    $filename = $filename.Split([IO.Path]::GetInvalidPathChars()) -join ''
    return $filename.Trim()
}

if (!(Test-Path '.vscode\launch')) {
    New-Item '.vscode\launch' -ItemType Directory | Out-Null
}

$branches = git branch --points-at $PreviousHead
foreach ($branch in $branches) {
    if (Test-Path '.vscode\launch.json') {
        $filename = Convert-BranchToFileName $branch
        Copy-Item '.vscode\launch.json' ".vscode\launch\$filename.json" -Force
    }
}

if ($BranchCheckout -eq 1) {
    $newBranch = git branch --show-current
    if (Test-Path ".vscode\launch\$(Convert-BranchToFileName $newBranch).json") {
        Copy-Item ".vscode\launch\$(Convert-BranchToFileName $newBranch).json" '.vscode\launch.json' -Force
    }
}

Result

The result is something like this. When I checkout a new branch launch.json is copied to .vscode\launch\<branchname>.json and the saved json file matching the new branch name overwrites launch.json

A pretty trivial example you might argue – and you’d be right – but hopefully enough to demonstrate some of the possibilities. More useful examples might include:

  • Cleaning up the source directory between checkouts e.g. deleting .app files from previous builds or removing duplicate apps in the .alpackages folder
  • Checking Git configuration before committing e.g. is the email address that will be associated with the commit in an acceptable format e.g. does it make a particular regex pattern?
  • Enforcing some policy about compile errors or warnings before a new commit is made
  • Use your imagination…

Managing Business Central Development with Git: Platforms

Another post about Business Central development and Git. Maybe the last one. Who knows?

Whatever your precise circumstances, if you are developing apps for Business Central you have to be mindful of the differences between BC versions and how it affects your app. If you are only developing for SaaS you might only care about the current and next version.

If you are developing and maintaining apps for the on-prem/PaaS market then likely you need to concern yourself with a wider range of BC versions. Even a we-only-support-Business-Central-and-not-NAV stance means we are now supporting four major versions – 13, 14, 15 and 16. I refuse to call the versions “Business Central <Year> <Spring Release/Fall Release/Wave One/Wave Two> – a number makes much more sense to me. Also, I’m British – “fall” is an accident, not a season.

Nomenclature aside, all of this does present us with a challenge. How do we maintain the source code for our apps, for different Business Central versions, in an efficient manner?

Changes Between Versions

For the uninitiated, what sorts of changes are we talking about between platform versions? There are various things to think about:

  • Runtime differences e.g. new mandatory properties in app.json
    • contextSensitiveHelp
    • target – using “Cloud” instead of “Extension”
    • dependencies – using “id” instead of “appId”
    • depending on the “System Application” and “Base Application” apps rather than using the application property
  • Standard fields that have recently been converted to enums
  • Standard functionality that has been moved, methods that have new signatures

And of course, many of you will have experienced the pain of the BC14 -> BC15 upgrade. TempBlob, Base64, Languages, Tenant Mgt. / Environment Info, Calendar Management – all breaking changes. Microsoft were criticised, rightly so, for breaking BC14 compatible apps so badly in BC15.

To their credit, however, Microsoft said that they would minimise future breaking changes, instead marking code as becoming obsolete for at least 12 months until it is removed. That has been borne out with the release of BC16. All but one of our BC15 apps works without any changes in BC16. The exception was an app that was using the Sorting Method on Warehouse Activity Header which has been converted from an option to an enum and now has different values. Microsoft sent me an email with the details of the compilation error.

Strategy

How to manage this then? When we first switched from developing AL apps on NAV2018 (don’t – it’s more trouble than it’s worth) to BC13 we created a new Git repo for each app. It became obvious that we don’t want to keep doing that. We don’t want as many repos as number of apps * number of supported BC versions. We need something smarter than that.

We’ve settled on something like this instead:

  • the master branch has the stable code for the current release of BC (as of this week, BC16), app.json has a platform value to match the latest version (16.0.0.0) and is built against the current Docker image (mcr.microsoft.com/businesscentral/sandbox)
  • new development is done against the current BC (worldwide) version in release, bug, and feature branches (as described here)
  • the code for each version of BC that we are supporting is in a BC13, BC14 or BC15 branch – this branch has an appropriate platform value in app.json and is built against a sandbox Docker image of that version

Imagine a repo like this:

* 3b6ba3c (HEAD -> BC14) Env. Info changes for BC14
* 5e3fda6 TempBlob changes for BC14
* 0f85829 Update Docker image for BC14
* e4fe665 app.json for BC14
| * 517615b (BC15) Update Docker image for BC15
| * e893e39 app.json for BC15
|/
* 60fe758 (tag: 1.1.0, master) Some changes for v1.1.0
* 8bd6f26 (tag: 1.0.0) Initial version

The current version of our app is 1.1.0 and we are supporting the current version of BC (BC16, in the master branch) and BC15 and BC14 in their respective branches. Revisiting an earlier idea, I like to think of these branches as telling a story, answering the question – “what changes do you have to make to the current version of the code to make it compatible with this version of BC?” For BC15 the answer is “not much” – just change app.json and the Docker image. For BC14 the answer is likely to be somewhat longer.

Now we are going to work on the next version of our app, v1.2.0. These changes would go through feature branches, pull requests, a release branch and eventually into master. I’ll skip all that and just show a new commit in the master branch.

* cd7b2ff (HEAD -> master, tag: 1.2.0) Changes for v1.2.0
| * 3b6ba3c (BC14) Env. Info changes for BC14
| * 5e3fda6 TempBlob changes for BC14
| * 0f85829 Update Docker image for BC14
| * e4fe665 app.json for BC14
|/
| * 517615b (BC15) Update Docker image for BC15
| * e893e39 app.json for BC15
|/
* 60fe758 (tag: 1.1.0) Some changes for v1.1.0
* 8bd6f26 (tag: 1.0.0) Initial version

Pushing those changes to the master branch triggers a build against BC16. Now, we want to include the 1.2.0 changes in the BC15 and BC14 versions of our app. We can simply rebase the BC15 and BC14 branches back on top of the master branch.

* f4a7d9b (HEAD -> BC14) Env. Info changes for BC14
* dd072ea TempBlob changes for BC14
* ad8905b Update Docker image for BC14
* 184001e app.json for BC14
| * 71140f4 (BC15) Update Docker image for BC15
| * 2981c4d app.json for BC15
|/
* cd7b2ff (tag: 1.2.0, master) Changes for v1.2.0
* 60fe758 (tag: 1.1.0) Some changes for v1.1.0
* 8bd6f26 (tag: 1.0.0) Initial version

(Force) Pushing the changes to the BC15 and BC14 branches will trigger new builds of the app against their respective Docker images.

Depending on what the v1.2.0 changes actually were, we may need to do some more work in the BC14 branch to make the new code compatible e.g. if the new code included some use of the TempBlob codeunit.

* ff1455b (HEAD -> BC14) More TempBlob changes
* f4a7d9b Env. Info changes for BC14
* dd072ea TempBlob changes for BC14
* ad8905b Update Docker image for BC14
* 184001e app.json for BC14
| * 71140f4 (BC15) Update Docker image for BC15
| * 2981c4d app.json for BC15
|/
* cd7b2ff (tag: 1.2.0, master) Changes for v1.2.0
* 60fe758 (tag: 1.1.0) Some changes for v1.1.0
* 8bd6f26 (tag: 1.0.0) Initial version

Going back to the idea of the BC14 branch telling a coherent story of making the app compatible with BC14, does it make much sense to have two commits of TempBlob changes? No. It doesn’t add anything for a developer looking at the repo in the future. We can sort that with an interactive rebase: git rebase -i master

pick 184001e app.json for BC14
pick ad8905b Update Docker image for BC14
pick dd072ea TempBlob changes for BC14
pick f4a7d9b Env. Info changes for BC14
pick ff1455b More TempBlob changes

Change the rebase script to tell Git to “fixup” the second TempBlob change into the first.

pick 184001e app.json for BC14
pick ad8905b Update Docker image for BC14
pick dd072ea TempBlob changes for BC14
fixup ff1455b More TempBlob changes
pick f4a7d9b Env. Info changes for BC14

Those changes will be melded together and keep the history of the repo neat and readable.

Managing Business Central Development with Git: Branching Strategy

The last few posts have been about manipulating the history of your Git repository, getting comfortable tools like rebase, reset, cherry-pick and commit –amend. That’s all geared towards trying to create a history which is more than just a record of stuff that happened but tells a story of the development of your app that is useful for your colleagues and your future self.

This post is on the same theme but we’re talking about your branching strategy. Remember one of the strengths of Git is how easy it is to create branches to isolate pieces of development from each other. That’s an awesome tool – but how do we make best use of it?

When is it useful to separate pieces of development from each other in different branches? How and when do you stick the pieces of the jigsaw back together again?

Options

As you’d expect there are a lot of different approaches and no shortage of people online supporting each one. Here are some popular options. I won’t attempt to critique them because we haven’t tried them all and because you can read, try them out for yourselves and form your own opinions.

Git Flow

https://nvie.com/posts/a-successful-git-branching-model/

This approach has a “develop” branch alongside master and feature branches which are used to manage the work in progress before they are merged back to master only when they are ready to be released.

GitHub Flow

https://guides.github.com/introduction/flow/

As with Git Flow, work in progress changes are isolated in their own branches. Unlike Git Flow they are merged directly back into master once they have been reviewed and are ready to go.

Trunk Based Development

https://trunkbaseddevelopment.com/

The key idea is to avoid having long-lived branches other than the trunk (master) branch. Development can be done against other branches but only to facilitate code review and discussion. Changes should be committed to master at least every 24 hours.

Considerations

As before adopting any tool or practice we need to think about our particular circumstances and needs. What are we actually trying to achieve? By all means read about what other people are doing. If you keep reading I’ll share what we’re (currently) doing but you should think about your own requirements, decide on something that makes sense for you and be prepared to improve it in future.

I think there is something to learn from each of the strategies I’ve linked to.

App Development

We are developing apps for Business Central either to be deployed via AppSource or installed through our partners on-premise to their customers. Either way, making a new version of our app available to our customers is not a trivial exercise.

When we submit a new version of our app it is typically at least 3 or 4 working days until it is available in AppSource. For on-prem customers we are reliant on our partners to upgrade the apps manually. Neither of these scenarios exactly falls into the ideal “continuous deployment” category. Some branching strategies are geared towards getting code into master as soon as possible so it can be pushed to the production environment each day, or even multiple times a day.

However attractive that might sound that is just isn’t reality for us – at least not yet. We’re due to be getting an API for pushing updates to AppSource, which is great, but as long as it is backed by a manual certification process I can’t see Microsoft thanking us for pushing multiple updates each day.

Given the lead time to getting a release live we should be quite careful about what is going to go into each one. We don’t really have the luxury of pushing an update immediately after another because we forgot to include something.

#1 Create a Release Branch

We start by creating a release branch. This is where we are going to collect all the changes that should be included in the next release before they are merged into the master branch. We do occasionally bundle in last minute changes and fixes to a release but we ought to have a pretty clear idea of what the release will include before we start.

Imagine we’ve got this repo. All of the commits are merged into the master branch which is tagged with 1.0.0. Tags are useful additional pointers to particular places in the history of the repo. In future if we want to see the code as it was in v1.0.0 we can just run git checkout 1.0.0

* 3894d1a (HEAD -> master, tag: 1.0.0, origin/master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Now create a new branch to use as our release branch. For now this just points to the same commit as master.

git branch release/1.1.0

#2 Create Individual Feature and Bug Branches

Now we’ll create separate branches for each feature or bug fix that we’ve decided to include in release 1.1.0. Why not just do all the changes we need in the release branch? Because we want to be able to develop and test them separately from each other.

* 381c83d (HEAD -> bug/commission-calc) Fix rounding error in commission calc
| * e9d31b4 (feature/sales-report) Action to open sales report from customer
| * 78102dd Sales report
|/
| * c450814 (feature/sales-price-calc) Prices in non-base UOM
| * dd5f6c0 Prices in additional currencies
| * 02fa619 Pricing elements per item
|/
* 3894d1a (tag: 1.0.0, origin/master, release/1.1.0, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

The graph might look something like this now. Separate branches with one or more commits in each. Incidentally, naming the branches feature/* and bug/* is just a convention – it doesn’t have any affect on how they are managed.

#3 Create Pull Requests and Complete Quickly

When each feature or bug fix is ready for review and testing we create a pull request targeting the release branch. Pull requests in Azure DevOps are great. However, in my experience there are two main things that make pull requests less great, or even bad.

  1. Bundling too many changes in a single pull request
  2. Leaving them open for too long

Having lots of changes makes it difficult to review and test those changes. Which means no one is enthusiastic to do it. Which means it gets left open for a long time.

Leaving pull requests open for a long time means people forget what the changes were for and whether they have already been tested. It becomes a burden that no one wants to take responsibility for. Eventually someone completes it because we’re all sick of seeing it on the list. Not an ideal reason to complete it.

We’ve got a couple of measures on our team dashboard – number of open pull requests and average age of those requests in days. If the average age is creeping over 7, say, then we’re likely doing something wrong.

We squash the commits when the pull request is completed. Like it sounds, that squashes all of the changes that are in the feature or bug branch into a single commit which is added to the release branch. We lose some of the history doing this but I think it makes it more readable later on. We are rarely interested in the details of how we wrote a certain feature – just that we did, and these were the changes that we made.

* 35cf673 (HEAD -> release/1.1.0) Merged PR 03: Commission Calc
* b23b8c5 Merged PR 02: Sales Report
* 8007dcf Merged PR 01: Sales Price Calc
| * 381c83d (bug/commission-calc) Fix rounding error in commission calc
|/
| * e9d31b4 (feature/sales-report) Action to open sales report from customer
| * 78102dd Sales report
|/
| * c450814 (feature/sales-price-calc) Prices in non-base UOM
| * dd5f6c0 Prices in additional currencies
| * 02fa619 Pricing elements per item
|/
* 3894d1a (tag: 1.0.0, origin/master, master) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

Here is the graph now. I’ve removed the remote branches to keep it simpler. Notice the “Merged PR” commits which have been created by completing the pull requests. I’ve still got local branches with the individual changes. These can now safely be deleted now that those changes have been squashed into the release branch.

#4 Merge into Master and Tag

Each push to the server triggers a pipeline to compile the code and run the tests. Assuming those builds are passing and with the manual testing that we’ve done we ought to be confident that the changes work as expected. Each time we complete a pull request it runs a build incorporating the other completed changes. If that passes as well then we’re ready to merge the changes into master, delete the release branch and tag the new version as 1.1.0

* 35cf673 (HEAD -> master, origin/master, tag: 1.1.0) Merged PR 03: Commission Calc
* b23b8c5 Merged PR 02: Sales Report
* 8007dcf Merged PR 01: Sales Price Calc
* 3894d1a (tag: 1.0.0) Correct typo in caption
* cd03362 Add missing caption for new field
* 94388de Populate new Customer field OnInsert
* c49b9c9 Add new field to Customer card

The end result – at least what we’re aiming for – is a neat summary of the changes that have been made between the two versions. We can see the changes which we made for each feature or bug fix in those commits. If we want more detail we can always go back and view the completed pull request on Azure DevOps.

In a future post we’ll think about how to manage different versions of the code for different versions of Business Central.

Further Reading

Check out Michael Megel’s post on the same topic here: https://never-stop-learning.de/branching-workflow-ci-cd-part-6/