You Can Ditch Our Build Helper for Dynamics 365 Business Central

I’m a bit of a minimalist when it comes to tooling, so I’m always happy to ditch a tool because its functionality can be provided by something else I’m already using.

In a previous post I described how we use our Build Helper AL app to prep a test suite with the test codeunits and methods that you want to run. Either as part of a CI/CD pipeline or to run from VS Code.

Freddy K has updated the navcontainerhelper PowerShell module and improved the testing capabilities – see this post for full details.

The new extensionId parameter for the Run-TestsInBCContainer function removes the need to prepare the test suite before running the tests. Happily, that means we can dispense with downloading, publishing, installing, synchronising and calling the Build Helper app.

The next version of our own PowerShell module will read the app id from app.json and use the extensionId parameter to run the tests. Shout out to Freddy for making it easier than ever to run the tests from the shell đź‘Ť

YAML Multi-Stage Pipelines in Azure DevOps, Stage 2

In the previous post I introduced you to multi-stage YAML pipelines. Build/Release pipelines vs. a multi-stage pipeline, enabling the preview feature (it’s still in preview at the time of writing) and an overview of the structure of the file.

Now we’ll take a more detailed look at an example multi-stage YAML file. This is geared at building apps for Business Central but the principles are transferable to any other application you are targeting. This is the example that I talked through in my recent webinar. If you’d rather view it as a gist you can see that here.

trigger:
- '*'

pool:
  name: Default

variables:
  image_name: mcr.microsoft.com/businesscentral/sandbox
  container_name: Build
  company_name: My Company
  user_name: admin
  password: P@ssword1
  license_file: C:\Users\james.pearson.TECMAN\Desktop\Licence.flf

stages:
- stage: build
  displayName: Build
  jobs:
  - job: Build
    pool:
      name: Default
    steps:
      - task: PowerShell@1    
        displayName: Create build container
        inputs:
          scriptType: inlineScript
          inlineScript: > 
            Import-Module navcontainerhelper;
            $Credential = [PSCredential]::new('$(user_name)',(ConvertTo-SecureString '$(password)' -AsPlainText -Force));
            New-NavContainer -accept_eula -accept_outdated -containerName '$(container_name)' -auth NavUserPassword -credential $Credential -image $(image_name) -licenseFile $(license_file) -doNotExportObjectsToText -restart no -shortcuts None -useBestContainerOS -includeTestToolkit -includeTestLibrariesOnly -updateHosts
      - task: PowerShell@1
        displayName: Copy source into container folder
        inputs:
          scriptType: inlineScript
          inlineScript: >
            $SourceDir = 'C:\ProgramData\NavContainerHelper\Extensions\$(container_name)\Source';
            New-Item $SourceDir -ItemType Directory;
            Copy-Item '$(Build.SourcesDirectory)\*' $SourceDir -Recurse -Force;
      - task: PowerShell@1
        displayName: Compile app
        inputs:
          scriptType: inlineScript
          inlineScript: >
            Import-Module navcontainerhelper;
            $SourceDir = 'C:\ProgramData\NavContainerHelper\Extensions\$(container_name)\Source';
            $Credential = [PSCredential]::new('$(user_name)',(ConvertTo-SecureString '$(password)' -AsPlainText -Force));
            Compile-AppInNavContainer -containerName '$(container_name)' -appProjectFolder $SourceDir -credential $Credential -AzureDevOps -FailOn 'error';
      - task: PowerShell@1
        displayName: Copy app into build artifacts staging folder
        inputs:
          scriptType: inlineScript
          inlineScript: >
            $SourceDir = 'C:\ProgramData\NavContainerHelper\Extensions\$(container_name)\Source';        
            Copy-Item "$SourceDir\output\*.app" '$(Build.ArtifactStagingDirectory)'
      - task: PowerShell@1
        displayName: Publish and install app into container
        inputs:
          scriptType: inlineScript
          inlineScript: >
            Import-Module navcontainerhelper;        
            Get-ChildItem '$(Build.ArtifactStagingDirectory)' | % {Publish-NavContainerApp '$(container_name)' -appFile $_.FullName -skipVerification -sync -install}
      - task: PowerShell@1
        displayName: Run tests
        inputs:
          scriptType: inlineScript
          inlineScript: >
            $Credential = [PSCredential]::new('$(user_name)',(ConvertTo-SecureString '$(password)' -AsPlainText -Force));
            $BuildHelperPath = 'C:\ProgramData\NavContainerHelper\Extensions\$(container_name)\My\BuildHelper.app';
            Download-File 'https://github.com/CleverDynamics/al-build-helper/raw/master/Clever%20Dynamics_Build%20Helper_BC14.app' $BuildHelperPath;
            Publish-NavContainerApp $(container_name) -appFile $BuildHelperPath -sync -install;
            $Url = "http://{0}:7047/NAV/WS/{1}/Codeunit/AutomatedTestMgt" -f (Get-NavContainerIpAddress -containerName '$(container_name)'), '$(company_name)';
            $AutomatedTestMgt = New-WebServiceProxy -Uri $Url -Credential $Credential;
            $AutomatedTestMgt.GetTests('DEFAULT',50100,50199);
            Import-Module navcontainerhelper;
            $ResultPath = 'C:\ProgramData\NavContainerHelper\Extensions\$(container_name)\my\Results.xml';        
            Run-TestsInBcContainer -containerName '$(container_name)' -companyName '$(company_name)' -credential $Credential -detailed -AzureDevOps warning -XUnitResultFileName $ResultPath -debugMode
      - task: PublishTestResults@2
        displayName: Upload test results    
        inputs:
          failTaskOnFailedTests: true
          testResultsFormat: XUnit
          testResultsFiles: '*.xml'
          searchFolder: C:\ProgramData\NavContainerHelper\Extensions\$(container_name)\my

      - task: PublishBuildArtifacts@1
        displayName: Publish build artifacts
        inputs:
          ArtifactName: App Package
          PathtoPublish: $(Build.ArtifactStagingDirectory)

      - task: PowerShell@1
        displayName: Remove build container
        inputs:
          scriptType: inlineScript
          inlineScript: >
            Import-Module navcontainerhelper;
            Remove-NavContainer $(container_name)
        condition: always()
- stage: Release
  displayName: Release
  condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/master'))
  jobs:
  - deployment:
    displayName: Release
    pool:
      name: Default
    environment: Release
    strategy:
      runOnce:
        deploy:
          steps:               
            - task: PowerShell@1
              displayName: Copy artifacts to release directory                  
              inputs:
                scriptType: inlineScript
                inlineScript: >
                  $Path = Split-Path '$(System.ArtifactsDirectory)' -Parent;
                  $Artifact = "$Path\App Package\*.app";
                  Copy-Item $Artifact 'C:\Release\';

Build

I’ve got a simple Business Central app. I want to use the Build stage to take my source AL code and:

  • Compile it into an .app file
  • Publish and install it into a container
  • Load the test codeunits and methods into the DEFAULT test suite
  • Run the tests
  • Upload the test results to the build
  • Upload the .app file as a build artifact

Almost all of these steps are performed with a PowerShell task. I won’t talk you through the PowerShell, you can read the code in the steps and read more about the use of the navcontainerhelper module on Freddy’s blog or dig around the PowerShell posts on my blog. I will just mention that loading the test codeunits relies on our AL Build Helper app as described previously.

The final step to run is a PowerShell script to remove the container that has been created for the build. I always want this step to run, even if another step above it has failed. See the condition: always() line that takes care of that.

Release

Space, Rocket, Travel, Science, Sky, Abstract, Planet

So far this is fairly familiar territory and stuff that we’ve been through before. The more interesting part for the purposes of this post is having a second stage to run.

First notice the condition attached to the Release stage:

and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/master'))

build.SourceBranch is one of the built-in variables that you can access in the pipeline which holds the name of the branch that triggered the build. This condition means the Release stage will only run if the pipeline has succeeded up to this point and the pipeline was triggered by the master branch.

This is useful for a CI/CD scenario where you want to trigger the pipeline for changes to any branch (and why would you not?) but only want to Release the code when it is merged back into the master branch.

My pipeline uses a specific type of deployment job which allows you to target a particular environment. The ‘Environments’ menu item is displayed when you enable the multi-stage pipelines preview feature. Including a deployment job in your pipeline will instruct Azure DevOps to automatically download the artifacts from the Build stage to the agent (see here).

The Release stage can include as many steps as you need depending on your definition of ‘releasing’ your software. It might include steps to:

  • Use PowerShell to publish the .app file to an on-prem instance of Business Central
  • Use the admin API to publish the .app file to a Business Central SaaS tenant
  • Upload the .app file to some other location: FTP, SharePoint, network path

As a very simple example my Release stage is simply going to copy the .app file that was downloaded as an artifact from the Build stage into a C:\Release folder on the build agent.

Environments

One of the good things about creating an environment and targeting it with a deployment job is that you can see at a glance which version of your software the environment is hosting.

Drill down on the environment to see details of the current and previous versions that have been deployed and all the relevant corresponding detail – the pipeline that deployed the software, the logs, commits and work items. Beautiful.

YAML Multi-Stage Pipelines in Azure DevOps, Stage 1

Let’s return to the subject of pipelines and this time let’s talk multi-stages. What is it and why might you want to implement it in your YAML file?

Builds/Releases

With the approach that Microsoft are now calling “classic” pipelines there was a definite division between a build pipeline and a release pipeline.

A build starts with a given version of your source code (a particular commit in your git repository, say) and proceeds to define the steps that should be performed on that code to “build” it.

You decide what “build” means and define the steps as you need them. In a Business Central AL extension context we’re probably talking: compile the extension into an app file, publish and install and run some tests.

A release takes artifact(s) that have been created by a particular build and/or code from a particular repository and “releases” them. Again, you define whatever “release” actually means to you. Publish an app file into a Business Central database, upload it to SharePoint, decompress the app file and send the source code to a printer – whatever you want.

Build pipelines can still be defined in the classic, visual editor or in a YAML file. The Azure DevOps interface makes it pretty obvious which way they recommend you do this. It took me a second to spot the discrete “use the classic editor” link when creating a new pipeline.

Clicking that link and successfully avoiding the top option from the following page which still creates a YAML file anyway will get you to the classic, visual editor. Select the agent that is going to run this pipeline, add one or more jobs and add one or more tasks to each job.

Even now, you’ll notice a “View YAML” link in the top right hand corner of the screen. Subtle. The term “classic” usually means something different when we’re talking about software than when we’re talking about novels. Less “masterpiece, will still be appreciated a century from now” and more “outdated, will be made obsolete and removed a few months from now”.

It’s probably a safe bet that the “classic” editor is going to go the same way as NAV’s “classic client” with its “classic reports”.

For completeness, the Release editor looks like this:

I’ve defined the build pipeline that provides the artifacts that will be released and can now define the stages and tasks involved in releasing it. At the time of writing you will still get this editor when creating a new pipelines from the Releases menu.

Multi-Stage Pipelines

Enter multi-stage pipelines. Rather than defining your build and your release tasks in separate editors you can define them in a single YAML pipeline definition.

You’ll need to enable the preview feature (from your profile menu in the top right hand corner). You’ll notice that the “Builds” option disappears from the Pipelines menu and is replaced with two new options “Pipelines” and “Environments”. Intriguing.

Now we can work with pipelines that look something like this:

trigger:
  '*'
parameters:
  image_name: a.docker.image
  container_name: my_container
stages:
- stage: build
  jobs:
  - job: Build
    pool:
      name: Default
    steps:
      (definition of the steps that are included in the build stage)
- stage: release
  condition:  and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/master'))
  jobs:
  - deployment:
    pool:
      name: Default
    environment: QA
    (further definition of the steps involved in the release stage)

We’ll go into the details of a complete multi-stage YAML pipeline in another post. For now I just want to outline the structure of the file:

stages (1) -> stage (1..*) -> jobs (1) -> job (1..*) -> steps/deployment/tasks

You can include as many stages as you need to effectively manage the build and deployment of your software. Each stage can evaluate a condition expression which decides whether the stage should be run or not. In my case I only want to run the release stage if the pipeline has succeeded up to this point and the pipeline has been triggered by the master branch.

By default, stages will be run in series and will be dependent upon the previous stage. You can mix this up by defining the dependsOn key for each stage.

Environments

You’ll also notice that my ‘deployment’ task includes the environment to which I’m going to deploy my software. This will correspond to an environment that you have created from the Environments option of the Pipelines menu in Azure DevOps.

You can control how and when code is released to a given environment with stage conditions (as above) or with manual approvals.

Open the environment and select ‘Checks’ from the menu. All approvers that are entered on the following page must approve a pipeline before the deployment to the environment will proceed. The pipeline will be paused in the meantime.

Next time…

I hope that’s enough to whet your appetite to go and investigate the possibilities for yourself and see if/how you can start making use of this in your own development team.

Next time we’ll go through a more complete example of a multi-stage YAML pipeline and how it is put together and works. Until then you might like to check out the recording of the webinar that I did for Areopa webinars. If you like it, do them a favour and subscribe to the channel, thanks.

AL Build Helper for Dynamics 365 Business Central Builds

If you’re interested in setting up a build pipeline to build apps for Business Central then you’re probably interested in running the automated tests as part of it. (I take it you are writing automated tests?)

Turns out getting your test codeunits and methods populated into a test suite ready to run isn’t straightforward. We use a separate “Build Helper” app that exposes a couple of web service methods to prep and clear a test suite. It helps us get the container ready for running Run-TestsInBCContainer (from the navcontainerhelper module).

I’ve uploaded a couple of versions of the app to a GitHub repo here: https://github.com/CleverDynamics/al-build-helper. One for BC15 and the other for BC14 and earlier.

I use it all the time for running test from VS Code as well as in our build pipelines. Our PowerShell module has an Install-BuildHelper function to download and install it. Alternatively you could slip some PowerShell like the below into your pipeline and smoke it.

$Container = 'de'
$Company = 'CRONUS DE'
$User = 'admin'
$Password = 'P@ssword1'
$TestSuite = 'DEFAULT'
$StartRange = 130000
$EndRange = 160000
$WSPort = '7047'
$BuildHelperUrl = 'https://github.com/CleverDynamics/al-build-helper/raw/master/Clever%20Dynamics_Build%20Helper_BC14.app'

$Credential = [PSCredential]::new($user, (ConvertTo-SecureString $Password -AsPlainText -Force))
$BHPath = Join-Path $env:Temp 'BH.app'
Download-File $BuildHelperUrl $BHPath
Publish-NavContainerApp $Container -appfile $BHPath -sync -install
$BH = New-WebServiceProxy ('http://{0}:{1}/NAV/WS/{2}/Codeunit/AutomatedTestMgt' -f (Get-NavContainerIpAddress $Container), $WSPort, $Company) -Credential $Credential
$BH.GetTests($TestSuite, $StartRange, $EndRange)

The above is BC14 and assumes that you’ve got the navcontainerhelper module loaded (so you can use Publish-NavContainerApp). For BC15 you’d change the script slightly to the below (different URL for Build Helper, the instance name is “BC” rather than “NAV”).

$Container = 'bc15'
$Company = 'My Company'
$User = 'admin'
$Password = 'P@ssword1'
$TestSuite = 'DEFAULT'
$StartRange = 130000
$EndRange = 160000
$WSPort = '7047'
$BuildHelperUrl = 'https://github.com/CleverDynamics/al-build-helper/raw/master/Clever%20Dynamics_Build%20Helper.app'

$Credential = [PSCredential]::new($user, (ConvertTo-SecureString $Password -AsPlainText -Force))
$BHPath = Join-Path $env:Temp 'BH.app'
Download-File $BuildHelperUrl $BHPath
Publish-NavContainerApp $Container -appfile $BHPath -sync -install
$BH = New-WebServiceProxy ('http://{0}:{1}/BC/WS/{2}/Codeunit/AutomatedTestMgt' -f (Get-NavContainerIpAddress $Container), $WSPort, $Company) -Credential $Credential
$BH.GetTests($TestSuite, $StartRange, $EndRange)

No doubt, given the rate of change in Business Central there will be a different/better way to do this by the time BC15/wave 2/Fall ’19/whatever the heck we call it is released – but this how we build against BC15 for now.

Feel free to use anything you find helpful with my blessing…but not necessarily my support. No warranties, own risk etc.

Building Microsoft Dynamics 365 Business Central Apps on Azure DevOps Hosted Agents

This is a quick follow up to this post. If you want an intro to building AL apps for Business Central you might want to check that out first.

In order to build your apps you need a build agent running somewhere which will listen for new jobs and run the scripts, create the Docker containers, run the tests or do whatever else you define in the build file.

You can install an agent on your own server somewhere and authenticate with a personal access token. You’re in charge of the hardware, install agents and scale the performance as you see fit.

Hosting

The alternative is to choose one of the hosted agents that Microsoft provide. The obvious attraction is that you don’t need to maintain any hardware. You just specify the type of machine (Ubuntu, Mac, Windows) that you want the job to run on and pay-as-you-go. Or possibly, don’t pay at all.

With the free tier of Azure DevOps you get:

  • One build job running at a time (other jobs will be queued until that has finished)
  • 1,800 minutes of build time per month

That’s cool. You can keep tabs on your usage and purchase more parallel jobs from here: https://dev.azure.com/<your organisation>/settings/buildqueue?_a=concurrentJobs

If you hit the limits of the free tier you can check out the cost of more jobs here: https://azure.microsoft.com/en-us/pricing/details/devops/azure-devops-services/. At the time of writing 40 USD gets you a second concurrent job and lifts the build minutes per month restriction to unlimited.

Self-Hosting

So…why would you not run on hosted agents? Cost is a consideration. Additional parallel jobs on self-hosted agents are only 15 USD per month. But, what’s 25 dollars per month between friends? That’s assuming you can’t live within the limits of the free tier. If you can then using hosted agents is free.

The main consideration as far as I can see is performance. If you are going to create a Docker container as part of your build (and if you aren’t then I’m not sure what you’re doing) then self-hosted agents are always going to have an advantage. You can have the right Docker image ready downloaded before the build begins but a hosted agent will always needs to download it first.

Our builds, running on a self-hosted agent, typically take between 8 and 15 minutes to complete, depending on how many tests are included in the build. Using the “Hosted Windows 2019 with VS2019” agent pool a test build (which just creates the downloads the Docker image and creates the container) takes around 18 minutes – pulling the latest production sandbox image.

NavContainerHelper is version 0.6.2.3
Host is Microsoft Windows Server 2019 Datacenter - ltsc2019
Docker Client Version is 18.09.6
Docker Server Version is 18.09.6
Pulling image mcr.microsoft.com/businesscentral/sandbox:latest-ltsc2019
latest-ltsc2019: Pulling from businesscentral/sandbox

Add in some time to actually build and publish the app, run the tests and upload the results and we’re probably looking closer to 25 minutes for the whole thing.

I’ll leave it up to you to decide whether you care enough about that performance difference to host build agents yourself. Then again, 1800 / 25 = 72 builds per month before you need to consider paying for more. Maybe that’s all you need? Especially if you’re just getting started with Azure DevOps, builds, YAML and all that jazz…