Tip: Octopus Merges in Git

If you occasionally glance at my blog you might have noticed that I am a big fan of pull requests as served up by Azure DevOps (exhibit A). I briefly described our typical branching strategy here, including how and why we use pull requests.

I love it. Writing, testing and reviewing discrete pieces of development independently of one another helps spread the work around the team and prevent work getting blocked by some unrelated development in the same project.

However, occasionally we’ll have several pull requests open for the same project which I want to publish to some testing environment (either a local Docker container or SaaS sandbox) all together. Maybe it made sense to develop those work items separately but it’s hard to test them in isolation.

I tend to create a new local branch called testing or something equally banal, merge all the changes into that branch and publish. Here’s where the octopus comes in. We usually use git merge to merge a single other branch into the current branch e.g.

git checkout master
git merge release/1.2.0 --ff-only

To merge the commits in the release branch into the master branch (but only if it is possible to fast-forward merge).

Git doesn’t restrict you to a single other branch though. You can add as many as you like. Say we’ve got 3 feature branches on the go which I want to push to a SaaS sandbox for a consultant to test.

git checkout -b testing
git merge feature/one feature/two feature/three

This will create and checkout a new testing branch and then merge the three feature branches into it. Octopuses for the win.

Testing Internal Functionality

Internal Access Modifier

We’ve had access modifiers in Business Central for a little while now. You can use them to protect tables, fields, codeunits and queries that shouldn’t be accessible to code outside your app.

For example, you might have a table that contains some sensitive data. Perhaps some part of a licensing mechanism or internal workings of your app that no one else should have access to. Mark the table as:

Access = Internal;

and only code in your app will be able to access it. Even if someone develops an app that depends on your app they will receive a compile error if they create a variable to the table: “<table> is inaccessible due to its protection level.” Before you ask about RecordRefs – I don’t know, I haven’t tested. I assume that Microsoft have thought of that and prevent another app from opening a RecordRef to an internal table belonging to another app.

Alternatively you might have a function in a codeunit that shouldn’t be called from outside your app. The function needs to be public so that other objects in your app can call it, but you can mark it as internal to prevent anyone else calling it:

internal procedure SomeSensitiveMethod()
begin
  //some sensitive code that shouldn't be accessible from outside this app
end;

Testing

Cool.

But wait…how do we test this functionality? We develop our tests alongside the app code but split the test codeunits out into a separate app in our build pipeline – because that’s how Microsoft like it for AppSource submissions.

The result is that the tests run fine against the local Docker container that I am developing and testing against. I push my changes to Azure DevOps to create a pull request and…the build fails. My (separate) test app is now trying to access the internal objects of the production app and fails to compile.

The solution is to use the internalsVisibleTo key in app.json of the production app. List one or more apps (by id, name and publisher) that are allowed to access the internals of the production app. More about that here.

Maybe you already develop your tests as a separate app and so can copy the app id from app.json of the test app.

In our case we usually generate a new guid for the test app as part of the build process – because we don’t usually care what id it has. For times we do want to specify the id of the test app we have an environment.json file that holds some settings for the build – Docker image, credentials, translations to test etc. We can set a testappid in that file and include it in the internalsVisibleTo key in app.json.

Now the build splits the apps into two and creates a test app with the id specified by testappid which compiles and can access internal objects and functions of the production app.

Getting Started with Git Hooks

Git hooks have been on my “to mess about with and learn a little some day” list for a while. It’s the old conundrum: I might use them if I knew what they could do, but I’m not going to learn about them until I’ve got a use for them. Chickens-and-eggs for developers.

The Problem

Until recently, that is. I wondered:

What is the best way to save a different launch configuration in VS Code for each branch in my repo?

Why would I want to do that? We have branches for versions of our apps compatible with BC14, 15 and 16. I’ve got separate Docker containers of each of those versions to develop and test against. I need to change the launch config when I want to test in another container. Not a great hardship admittedly, but it would be nice to automate somehow.

Source Control?

I could of course just source-control the launch.json file and commit the relevant changes to each branch. I don’t like that though. We ignore the .vscode folder in .gitignore. To my mind anything that happens in that folder is for the benefit of the local development environment. No one else cares what my launch config is – they’ll be pushing code to their own Docker containers or to SaaS sandboxes.

I suppose you could have some local commits that include the launch.json file that you don’t push to the server. Sounds horrible to manage though. Don’t do that.

So we want something that is related to source control – we want a launch config per branch – but doesn’t involve actually committing the config file. Enter hooks.

Hooks

Hooks are points in the Git workflow at which you can execute a custom script. Some are executed server-side, some client-side. You can read the documentation at https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks

When a Git repo is initialised a .git folder is created which contains all the version history of the repo and Git’s guts. If you’re curious to learn more about how Git actually works have a browse through these folders. You might want to load up a presentation on the subject as well – like this: https://www.youtube.com/watch?v=ig5E8CcdM9g

You’ll notice that one of the folders is called hooks. This contains some sample hook files which you can take a look through. The .sample file type stops them from actually doing anything but you get a flavour of what’s possible and how you can use them.

Post-Checkout Example

For our example we’re going to want to use the post-checkout hook. This allows us to create a script that Git will execute after a checkout command. (Side note: I tend to use the integrated terminal in VS Code to execute Git commands, including checking out branches but this hook is also executed if you switch branches with the UI).

First, post-checkout isn’t one of the samples that is created when you init a new repo. You can copy the post-update.sample file instead and rename it to post-checkout (no file extension).

Writing the Hook

Okay…so what are we supposed to put in this file? Can we just start typing PowerShell? Erm…no, we can’t. The file looks distinctly Linuxy…*shudder* I’ve got nothing against Linux, I just have next to no idea what I’m doing.

The top line of the file indicates the language that you are going to script in. It’s possible to use Perl, Python, Ruby and a bunch of other stuff, including PowerShell. I won’t pretend to really know what I’m talking about but there is an useful thread on Stack Overflow on the topic.

#!/bin/sh

It is possible to write PowerShell directly into the hook file, but from what I’ve read it seems you need PowerShell Core to run it. I’ve settled for this:

#!/bin/sh

c:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe -File '.git\hooks\post-checkout.ps1' $1 $2 $3

Call powershell.exe to execute the .git\post-checkout.ps1 file and pass it the three parameters that have been received. Great. How do we know that there are three parameters? And what are they? You can read the documentation here: https://git-scm.com/docs/githooks#_post_checkout

The parameters are:

  • The previous HEAD (the commit that we’ve come from)
  • The new HEAD (the commit that we’ve just checked out)
  • A flag to indicate whether a branch was checked out (it has a value of 1 if it was)

Writing the Script

Now we’re getting somewhere.

The plan is:

  • Save the launch configuration as it stands for the branch that we are coming from
  • Find a saved launch configuration for the branch that we have moved to and overwrite launch.json

What Branch Have We Come From?

This is a subtler question than we might have expected. The parameter that Git passes is the hexadecimal hash of the commit that was previously checked out. Some jumble of numbers and letters that is integral to how Git works but probably meaningless to the developer. We need to convert that hash into a branch. Or branches – potentially more than one branch might be pointing at that commit.

git branch --points-at <commit hash>

The above command will give us zero or more branches that are pointing at a given commit. Note that the current branch is denoted with an asterisk which we’ll need to trim off the front of the name.

Which Branch Have We Moved To?

We can find the name of the new, current branch with:

git branch --show-current

Finally, branch names can contain characters that cannot be used in filenames. Specifically we use forward-slashes a lot in our branch names. I’ve got a Convert-BranchToFileName function to clean it up.

Putting all the jigsaw pieces together this is the content of my post-checkout.ps1 file (which is in the .git\hooks directory).

[CmdletBinding()]
param (
    $PreviousHead,
    $NewHead,
    $BranchCheckout
)

function Convert-BranchToFileName() {
    param(
        $branch
    )

    $filename = $branch
    if ($branch.StartsWith('*')) {
        $filename = $branch.Substring(2)
    }

    $filename = $filename.Split([IO.Path]::GetInvalidPathChars()) -join ''
    return $filename.Trim()
}

if (!(Test-Path '.vscode\launch')) {
    New-Item '.vscode\launch' -ItemType Directory | Out-Null
}

$branches = git branch --points-at $PreviousHead
foreach ($branch in $branches) {
    if (Test-Path '.vscode\launch.json') {
        $filename = Convert-BranchToFileName $branch
        Copy-Item '.vscode\launch.json' ".vscode\launch\$filename.json" -Force
    }
}

if ($BranchCheckout -eq 1) {
    $newBranch = git branch --show-current
    if (Test-Path ".vscode\launch\$(Convert-BranchToFileName $newBranch).json") {
        Copy-Item ".vscode\launch\$(Convert-BranchToFileName $newBranch).json" '.vscode\launch.json' -Force
    }
}

Result

The result is something like this. When I checkout a new branch launch.json is copied to .vscode\launch\<branchname>.json and the saved json file matching the new branch name overwrites launch.json

A pretty trivial example you might argue – and you’d be right – but hopefully enough to demonstrate some of the possibilities. More useful examples might include:

  • Cleaning up the source directory between checkouts e.g. deleting .app files from previous builds or removing duplicate apps in the .alpackages folder
  • Checking Git configuration before committing e.g. is the email address that will be associated with the commit in an acceptable format e.g. does it make a particular regex pattern?
  • Enforcing some policy about compile errors or warnings before a new commit is made
  • Use your imagination…

“Discrete Business Logic” implements Interface

If you’ve read anything about what is new in BC16 for developers then you’ll have seen that AL now supports interfaces (*fanfare*). No more convoluted workarounds with events – at least once our customers have upgraded.

I won’t explain what interfaces are and how to use them in AL. You can read the release notes here or blog posts like this and this. Instead I’m going to share some thoughts about where we might make use of interfaces. The obvious disclaimer is that we’ve only just started working with them and I’ve no doubt that we’ll learn more about how to use them properly as we go.

TL;DR

Look for discrete blocks of business logic that can be abstracted to interfaces rather than entities. Identify which business logic you, or other devs dependent on your app, would like to be able to replace. Identify the essential methods that would need to be implemented to replace that logic.

Entities

If you do a quick search for interface examples in other languages you’ll find that most of them are based on entities i.e. real world things or concepts that you are modelling in your software. The common actions those entities can perform or properties that they have are grouped into an interface.

The favourite examples seem to be animals and vehicles e.g. animals can make a sound, eat, sleep etc. You end up with:

interface IAnimal
{
  int noOfLegs;
  procedure MakeSound();
  procedure Eat();
  procedure Sleep();
}

or

interface IVehicle
{
  decimal: maxSpeed;
  int: noOfWheels;
  procedure Start();
  procedure Drive();
  procedure Stop();
  procedure GetMaxSpeed();
}

In other languages you might model your entities with classes that implement those interfaces.

class TreeFrog implements IAnimal
...

class Segway implements IVehicle
...

The trouble is, that doesn’t map particularly well into AL. We model our entities in tables and tables don’t support interfaces, at least not yet. You could image for instance IPerson and IAddress interfaces which describes the properties and methods of people and addresses.

interface IPerson
{
  field("First Name"; Text[50]);
  field(Surname; Text[50]);
}

interface IAddress
{
  field(Address; Text[50]);
  field("Address 2"; Text[50]);
  field(City; Text[50];
  field(County; Text[50]);
  field("Post Code"; Text[50]);
  field("Country/Region Code"; Code[10]);
  procedure Format(AddrArray: Array[8] of Text[50]);
}

Those interfaces could be implemented by various entities in Business Central.

table Customer implements IAddress;
table Employee implements IAddress, IPerson;
table "Salesperson/Purchaser" implements IPerson;

You could imagine that – but that’s all you can do for now. AL interfaces don’t support properties and can’t be implemented by tables.

That’s a long way of saying that it might be useful not to focus on entities when we are looking at candidates for abstracting into an interface.

Discrete Business Logic

Instead of thinking in terms of entities, try thinking in terms of discrete business logic. What self-contained blocks of business logic do you have in your app that it might be useful to replace with a different implementation? Or, what business logic do you have several implementations of that you need to choose between?

Consider that Microsoft’s first major use of an interface in the app wasn’t to do with customer, vendors, sales orders, G/L accounts or any other major entity. It was the sales price calculation logic. From BC16 you can select between different standard implementations or implement your own. More abstract than IVehicle.GetMaxSpeed() but probably far more useful in real life.

It occurs to me that there are probably several benefits to looking for existing and new blocks of business logic that are candidates for implementing via an interface.

Separation of Concerns

Hopefully obvious, but worth a mention. Anything that helps us to consider what our discrete block of business logic are and how to separate them from one another is a good thing. It’s easier to develop – especially in a team and easier to understand and maintain later on.

The process of designing the interface and deciding exactly what is the required to implement some business logic is valuable in itself.

Extensibility

Other developers want to build on top of our logic – our replace parts of our logic with their own. We try to make that as easy as possible with OnBefore/OnAfter events, Handled parameters and developer documentation. It is still a relatively complex task though and easy to subscribe to the wrong event, not respect the Handled flag or make some other mistake that breaks stuff.

An event? For me?

If you are able to just tell a 3rd party developer they can extend an enum and implement an interface life becomes a little easier. The interface, and the compiler, tell them the methods they need to implement and your app can call their implementation rather than having them subscribe to events and fire up an instance of their codeunit every time – like an excitable chihuahua barking every time post comes through the letter box only to find that the mail isn’t addressed to them.

Speaking of which I imagine there must be some performance benefit – probably negligible in most cases – but there must be one.

Obsolescence

We’ve got the Obsolete properties now and can move methods around and use the ObsoleteReason to tell dependent developers how they need to change their code. That’s all great. What if you are having a much bigger restructuring of your code though?

There might be an opportunity to follow Microsoft’s lead. Introduce an interface and have the existing business logic implement that interface. That frees you up to create a new implementation and switch customers and custom development over in a controlled fashion.

For example, we’ve got a case at the moment where existing customisation has been done through event subscriptions. In future we’re going to want dependent developers to extend an enum and implement an interface.

We’re introducing an interface and a default implementation. The default implementation calls the existing events so custom development will continue to work. Then we and dependent developers can migrate over to the interface over time.

Performance of Test Code

Let’s talk about the performance of the test code that we write for Business Central. What do I mean by “performance” and how can we improve it?

Defining “Performance”

Obviously, before we set out to improve something we need to have an idea of what it is we’re trying to optimise for. I’m coming to think of the performance of test code in a couple of key ways:

  1. How easy/quick is it to write test code?
  2. How quickly do the tests run?

Performance of Writing Tests

I suppose none of the below points are specific to test code. They are relevant to any sort of code that we are writing but we can be more inclined to neglect them for test code than production code. If you embrace any sort of automated tested discipline you’re going to spend a significant proportion of your time reading and writing test code – perhaps even as much as you do on production code. It is well worth investing a little time to clean up the code and making it easier to read and maintain.

Comments

Say what you like about comments in production code – variable names should declare the intent, comments are evil blah blah – I do find a few comments valuable in a test.

In fact, I write them first. Given some set of circumstances, when this happens then this is the expected result. Writing those sentences first helps to be clear about what I’m trying to test and what the desired behaviour actually is.

Of course the code in between those comments should be readable and easy to follow – but if you are diligent with a few comments per test you can describe the expected behaviour of that part of the system without having to read any code.

Grouping Tests Logically

I appreciate the situation is different for VARs but as an ISV we have large object ranges to do our development in. There is no reason for us to bundle unrelated code into the same codeunit. If we are starting work on separate from existing business logic then it belongs in its own codeunit. In which case, why wouldn’t the corresponding tests also go in their own codeunit?

Ideally I want to be able to glance down the list of test codeunits that we have and see logical grouping of tests that correspond to recognisable entities or concepts in the production code. If I want to see how our app changes the behaviour of warehouse receipts I can look in WhseReceiptTests.Codeunit.al.

Refactoring Into Library Codeunits

As soon as you start writing tests you’ll start working with the suite of library codeunits provided by Microsoft. You’ll notice that they are separated into different areas of the system e.g. Library – Sales, Library – Warehouse, Library – Manufacturing and so on.

Very likely you’ll want to create your own library codeunit to:

  • Initialise some setup, perhaps create your app’s setup record, create some No. Series etc.
  • Create new records required by your tests – it is useful to follow the convention LibraryCodeunit.CreateXYZ(var XYZ: Record XYZ);
  • Consider having those Create functions returning the primary key of the new record as well so the result can be used inline in the test e.g.
    • LibraryCodeunit.CreateXYZ(var XYZ: Record XYZ): Code[20]
    • LibraryCodeunit.CreateXYZNo(): Code[20]
  • Use method overloads for the create functions – have an overload with more parameters when you need to specify extra field values but keep a simple overload for when you don’t
  • Identify blocks of code that are often required in tests and consider moving them to a library method e.g. creating an item with some bespoke fields populated in a certain way

Having a comprehensive library codeunit brings two benefits:

  1. Tests are easier and faster to write if you already have library methods to implement the Given part of the test
  2. Less code in the tests, making them easier to read

Performance of Running Tests

First, why do we care about how long tests take to run? Does it really matter if your test suite takes an extra minute or two to run?

Obviously, we want our builds to complete as quickly as possible, while still performing all the checks and steps that we want to include. The longer a build takes the more likely another one is going to be queued at the same time and eventually someone is going to end up having to wait. We’ve got a finite number of agents to run simultaneous builds (we host our own – more on that here if you’re curious).

But that isn’t the biggest incentive.

I’m a big fan of running tests while I’m developing – both new tests that I’m writing to cover my new code and existing tests (more on that here). I usually run:

  • the test that I’m working on very frequently (at least every few minutes – see it fail, see it pass)
  • all the tests in that codeunit frequently (maybe each time I’ve finished with a new test)
  • the whole test suite every so often (at least 2-3 times before pushing my work and creating a pull request)

After all, if you’ve got the tests, why not run them? I should know sooner rather than later if some code that I’ve changed has broken something. If the whole test suite takes 60 seconds to run that’s fine. If it takes 10 minutes that’s more of a problem.

In that case I’ll be more inclined to push my changes to the server without waiting, keep a build agent busy for half an hour, start working on something else and then get an email saying the build has failed. Something I could have realised and fixed if I’d run the test suite before I pushed my changes.

So, how to make them faster?

Minimal Setup

First, only create the scenario that is sufficient for your test. For example, we work with warehousing functionality a lot. If we’re testing something to do with warehouse shipments do we need a location with advanced warehousing, zones, bin types, bins, warehouse employees…?

Probably not. Likely I can create a location without Bin Mandatory or Requires Pick and still create a sufficient test.

If you need ledger entries to test with you may be able to create and post the relevant journals rather than creating documents and posting them. Creating an item ledger entry by posting an item journal line is faster than posting a sales order.

Or, you probably want to prevent negative inventory in real life – but does that matter for your test? Save yourself the trouble of having to post some inventory before shipping an item and just allow it to go negative.

Try to restrict the setup of your tests to what is actually essential for the scenario that you are testing. Answering that question is, in itself, a useful thought process.

Shared Fixture

Better yet, set something up once per test codeunit and reuse it in each of the tests. This what Luc van Vugt refers to as a “shared fixture”. You should check out his blog for more about that.

I feel a little mixed about this. I like the idea that each test is entirely responsible for creating its own given scenario and isn’t dependent on anything else but this is denying that this is faster. Finding a posted sales invoice that already exists is much faster than creating a customer, item, sales order and shipping and invoicing it.

No Setup

What is even faster than setting up some data one time? Doing it no times. Depending on what you are testing you may just be able to insert the records you need or call field validation on a record without inserting it.

If I’m testing that validating a field behaves in a certain way I may not need a record to actually be inserted in the table.

Alternatively if you need a sales invoice header to test with you might be able just to initialise it and call SalesInvHeader.Insert; It feels so wrong – but if your test just needs a record and doesn’t care where it came from, who cares? It will all be rolled back after the tests have run anyway.