AL Test Runner for Visual Studio Code

TL;DR

I’ve written an extension for VS Code to help run your AL tests in local Docker containers. Search for “AL Test Runner” in the extension marketplace or click here. Feedback, bugs, feature suggestions all gratefully received on the GitHub repo or james@jpearson.blog

Intro

As soon as Freddy added capability to the navcontainerhelper module to execute automated tests I was excited about the potential for:

  1. Making test execution in our build pipeline simpler and more reliable
  2. Running tests from Visual Studio Code as part while developing

I’ve written about both aspects in the past, but especially #1 recently – about incorporating automated tests into your Azure DevOps pipeline.

This post is about #2 – incorporating running tests as early as possible into your development cycle.

Finding Bugs ASAP

You’ve probably heard the idea – and it’s common sense even if you haven’t – that the cost of finding a bug in your software increases the later in the development/deployment cycle you find it.

If you realise you made a silly mistake in code that you wrote 2 minutes ago – there’s likely no harm done. Realise there is a bug in software that is now live in customers’ databases and the implications could be much greater. Potentially annoyed customers, data that now needs fixing, support cases, having to rush out a hotfix etc.

We’ve all been there. It’s not a nice place to visit. I once deleted all the (hundreds of thousands of) records in the Purch. Rcpt. Line table with a Rec.DELETEALL on a temporary table…turns out it wasn’t temporary…and I was working in the live database.

Writing automated tests can help catch problems before you release them out into the wild. They force you to think about the expected behaviour of the code and then test whether it actually behaves like that. Hopefully if the code that we push to a branch in Azure DevOps has a bug it will cause a test to fail, the artifacts won’t be published, the developer will get an email and the customer won’t be the hapless recipient of our mistake. No harm done.

However, the rising cost of finding a bug over time still applies. Especially if the developer has started working on something else or gone home. Getting back your head back into the code, reproducing and finding the bug and fixing it are harder if you’ve had a break from the code than if you went looking for it straight away.

Running Tests from VS Code

That’s why I’m keen that we run tests from VS Code as we are writing them. Write a test, see it fail, write the code, see the test pass, repeat.

I’ve written about this before. You can use tasks in VS Code to execute the required PowerShell to run the tests. The task gives you access to the current file and line no. so that you can fancy stuff like running only the current test or test codeunit.

AL Test Runner

However, I was keen to improve on this and so have started work on a VS Code extension – AL Test Runner.

Running the current test with AL Test Runner and navcontainerhelper

The goals are to:

  • Make it as simple as possible to run the current test, tests in the current codeunit or all tests in the extension with commands and keyboard shortcuts
  • Cache the test results
  • Decorate test methods according to the latest test results – pass, fail or untested
  • Provide extra details e.g. error message and callstack when hovering over the test name
  • Add a snippet to make it easier to create new tests with placeholders for GIVEN, WHEN and THEN statements

Important: this is for running tests with the navcontainerhelper PowerShell module against a local Docker container. Please make sure that you are using the latest version of navcontainerhelper.

Getting Started

  • Download the extension from the extension marketplace in VS Code and reload the window.
  • Open a folder containing an AL project
  • Open a test codeunit, you should notice that the names of test methods are decorated with an amber background (as there are no results available for those tests)
    • The colours for passing, failing and untested tests are configurable if you don’t like them or they don’t fit with your VS Code theme. Alternatively you can turn test decoration off altogether if you don’t like it
  • Place the cursor in a test method and run the “AL Test Runner: Run Current Test” command (Ctrl+Alt+T)
  • You should be prompted to select a debug configuration (from launch.json), company name, test suite name and credentials as appropriate (depends if you’re running BC14 or BC15, if you have multiple companies, authentication type etc.)
    • I’ve noticed that sometimes the output isn’t displayed in the new terminal when it is first created – I don’t know why. Subsequent commands always seem to show up fine 🤷‍♂️
  • Use the “ttestprocedure” to create new test methods

.gitignore

If you’re using Git then I’d recommend adding the .altestrunner folder to your .gitignore file:

.altestrunner/

Committing the config file and the test results xml files doesn’t feel like a great idea.

Satisfying Your Case-Sensitive Obsession with Regex

Obsession is probably a little strong, but I do like tidy code. You know – proper indentation, a sensible number of comments (which can be more than none but there shouldn’t be as much comment as code) and good names. Hungarian notation brings me out in a rash.

This extends to having keywords, variables and methods in the right case. While in CAL there was a lot of UPPERCASE, in AL there is far more lowercase. It took me a while to get used to but I prefer it now.

If you convert some CAL into AL then likely all the keywords are going to be in uppercase. The code will run fine, it just doesn’t look nice. In the below example my eye is drawn to the fact that some filters are being set, rather than what those filters are – on which records and fields.

You’ll notice that all the UPPERCASE words are highlighted in that example. That’s because they are all search results for my regular expression.

\b(?!NAV)(?!CACTMN)[A-Z]+\b
  • \b will match a word boundary – spaces, punctuation, start and end of lines – anything that denotes the start or end of a word
  • (?!) is a negative lookahead and does not find matches for the expression inside the brackets. This is useful for uppercase words that should be left uppercase like NAV or the AppSource suffix that you’ve added all over the place
    • Disclaimer: don’t ask me to explain lookaheads in any more detail than that – I don’t know. I’m not convinced that anyone actually properly knows how regex works 😉
  • [A-Z] matches uppercase characters between A and Z
  • + indicates that we’re looking for one or more of the previous group i.e. one or more uppercase letters

Altogether it translate to something like: match any word of one or more uppercase letters except where it contains “NAV” or “CACTMN” (the suffix we’re using in this app).

Once you’ve found the matches find and replace is your friend. I love how VS Code gives you a preview of all the replaces that it is going to do. Very useful before you replace “IF” with “if” and realise you’ve also replaced “MODIFY” with “MODifY”.

Testing Microsoft Dynamics 365 Business Central from VS Code

Execute your Microsoft Dynamics 365 Business Central tests – with a keyboard shortcut – without leaving the comfort of your favourite IDE. What’s not to love?

Background

We’ve come a long way with testing our apps in Microsoft Dynamics 365 Business Central / NAV. By “we” I mean our internal development practices but also the capabilities of the platform.

  1. We didn’t have any automated testing – we had a joke written on the white board in the development office, “F11 is testing”
  2. Microsoft made it possible to create and run automated tests in Dynamics NAV – we didn’t use it
  3. Microsoft improved the tooling with the release of the “Test Tool” page
  4. We started to experiment with it and to write our own tests
  5. We picked up the idea of automating test runs with a build pipeline…

…and that was a bit of a problem. The best way to run the tests was through the Windows client. That meant using the dynamicsnav:// protocol to open the client, creating a ClientUserSettings.config file, ending the client process when it had finished…a big improvement on not testing at all – but hardly elegant.

With Microsoft’s adoption of Docker to distribute NAV / Business Central images came Freddy’s navcontainerhelper PowerShell module. A big, and relatively recent, step forward was the ability to run a test suite in a Docker container from PowerShell.

If you don’t know I’m talking about you’re best heading over to Freddy’s blog and doing some background reading.

Running the Tests

This is all great for our build process. We can create a Docker container, install the app(s), prepare the test suite (see here), run the tests from PowerShell, upload the results and bin the container.

As soon as the ability to run tests from PowerShell became available I was interested in how we could use them in our everyday development, not just in the pipeline. If you do any reading about Test Driven Development you’ll find that it is based on a very tight feedback loop. Write a test, write a small chunk of production code, run the tests. Repeat.

I want to be able to run the tests from the same environment that I write the code – Visual Studio Code. I don’t want to be switching back and forth from VS Code to the browser, refreshing test codeunits or methods and running them there. I just don’t find it a nice way to work.

With VS Code’s built in terminal and navcontainerhelper loaded you don’t have to.

  1. Set “launchBrowser” to false in launch.json
  2. Write a test
  3. Execute Run-TestsInNavContainer in the terminal and review the results
  4. Write some production code
  5. Publish without debugging
  6. Execute Run-TestsInNavContainer in the terminal and review the results
  7. Repeat steps 2-5

I actually use Run-BCTests, a function in our own PowerShell module. Run-BCTests is a wrapper for Run-TestsInNavContainer which:

  • Reads the name of the container from launch.json (see below) – unless a different container is specified
  • Creates a PowerShell credential object from the credentials we store in a json file in the repo
  • Optionally downloads our “Build Helper” app (from its last successful build – like this) to load our test codeunits and methods into the DEFAULT suite
  • Use Run-TestsInNavContainer to run the tests and output the results to the terminal

Get-ContainerFromLaunchJson

You already have to set the name of your container in the launch.json file to publish your app, so why not read it from there rather than typing your container name all the time?

function Get-ContainerFromLaunchJson {
  param (
    # Path to launch.json
    [Parameter(Mandatory=$false)]
    [string]
    $LaunchJsonPath = (Join-Path (Get-Location) '.vscode\launch.json')
  )

  if (!(Test-Path $LaunchJsonPath)) {
    return ''
  }

  $LaunchJson = ConvertFrom-Json (Get-Content $LaunchJsonPath -Raw)
  if ($LaunchJson.configurations.Count -ne 1) {
    return ''
  }
  else {
    $Container = $LaunchJson.configurations.Item(0).server
    $Container = $Container.Substring($Container.IndexOf('//') + 2)
    $Container
  }
}

Note that the container name in launch.json will need to match the case of the Docker container name. For reasons best known to Docker container names are case-sensitive.

VS Code Tasks: Running Tests With a Keyboard Shortcut

In pursuit of making it as quick and easy as possible to execute the tests from VS Code we can go a step further and create a VS Code task.

Create a tasks.json file in the .vscode folder. Mine looks like this:

{
    // See https://go.microsoft.com/fwlink/?LinkId=733558
    // for the documentation about the tasks.json format
    "version": "2.0.0",
    "tasks": [
        {
            "label": "Run BC Tests",
            "type": "shell",
            "command": "Run-BCTests -DoNotPrepTestSuite",            
            "group": {
                "kind": "test",
                "isDefault": true
            }
        }
    ]
}

This defines a task with the label “Run BC Tests” which runs the command “Run-BCTests -DoNotPrepTestSuite” (the PowerShell function described above). It is set as the default task in the test group.

This allows you to run “Run Test Task” from the command palette.

Run Test Task

Now you can assign a keyboard shortcut to that command. Open “Preferences: Open Keyboard Shortcuts” from the command palette and search for “Run Test Task”.

Run Test Task Keyboard Shortcut.JPG

Double click that entry to set whatever keyboard combination you like. I’ve opted for ctrl+shift+T.

Conclusion

This takes us a step closer to having a code-and-test-in-the-IDE development experience and allows this kind of tight test/production code iteration without having to open the browser:

  1. Write some test code
  2. Publish without debugging (Ctrl+F5)
  3. Run the tests (Ctrl+Shift+T) and check that they fail
  4. Write production code
  5. Publish without debugging (Ctrl+F5)
  6. Run the tests (Ctrl+Shift+T) and check that they pass
  7. Repeat

Of course, this does not remove the need to open the browser, check the look and feel and test your code manually at some point but it does go some way to alleviating the pain of publishing and executing the tests in the browser.

Future

I like it, but it’s still a little clunky. For one, it runs too slowly – the gif at the head of this post is real-time. Also, you still have to populate the test suite with the test codeunits and methods that you want to run at some point. Our PowerShell module can do that, but it’s another manual step to run.

There’s only so much we can do about these issues until Microsoft overhaul the whole testing framework – which I understand they are working on. In the meantime, here’s some other ideas that we haven’t implemented yet.

  • Use the previous commit, or uncommitted changes to determine the test codeunit(s) to run. Run-TestsInNavContainer has optional parameters to specify what to run
  • Filter the results so that you are only notified of failures instead of having to pick them out of the successes
  • Work some magic in VS Code to have the test task automatically triggered when the app is published to the server

VS Code, PowerShell & Git: 5 Things

Visual Studio Code has moved quickly from “what’s that? Part of Visual Studio? No? Then why did they call it that?” to become the hub of much of my daily work. This post contains a few of the things (5 to be precise) that I’ve done to make it work better for me. Maybe you can glean something useful. Maybe you can teach me something about how you use it – post a comment.

Extensions

You can use VS Code to write JavaScript, C#, CSS, HTML and a raft of other languages, use its native support for Git and install extensions for AL (obviously), developing Azure Functions, integrating with Azure DevOps, managing Docker, writing Power Shell, adding support for TFVC…

Beautiful.

Having said that, I’m not a big fan of having lots of extensions that I only occasionally use. I’m pretty ruthless in uninstalling stuff I’m not using in Chrome and Android. VS Code is the same. If I don’t use it all the time I generally go without it. (For those of us that make apps for a living it’s a sobering thought that our prospective users are likely to be the same).

Right now I’ve got these extensions installed:

  • AL Language – every so often I need an upcoming version or a NAV 2018 version but most of the time I’ve got the one from the marketplace installed
  • Azure Account – provides some sign in magic required by other extensions
  • Azure Functions – like it sounds
  • Azure Pipelines – intellisense for YAML build definitions
  • CRS AL Language Extensions – for renaming AL files to follow best practices and because I don’t like the convention. Including the object type when the files are already in subfolders by object type and including objects IDs when we all agree we want to get rid of them and don’t care what they are as long as they’re unique seems pretty redundant to me…but I digress
  • GitLens – add blame annotations i.e. “how did this line of code get here”, file history, compare revisions, open the file in Azure DevOps
  • PowerShell – like it sounds
  • Night Owl – a theme. Because we can! Having suffered for years with an IDE that didn’t even highlight keywords I took my time trying out different themes. I like a dark theme but didn’t quite get on with the one that comes with VS Code.

Terminal

VS Code has a built in terminal. I use PowerShell a lot during the day to manage containers (with the navcontainerhelper module), manage Git and various tasks with our own module to call the with Azure DevOps REST API. It’s nice to also be able to do all that from within VS Code.

These ideas aren’t strictly to do with VS Code, but tweaking PowerShell and Git to make them more efficient for you.

Run as Administrator

If you’re going to use the terminal to manage docker containers you’re going to want to run VS Code (and therefore the terminal) as administrator.

You can set this in the Advanced section of the properties of the shortcut. This will force VS Code to always open as admin.

VS Code Shortcut Properties.JPG

I believe Freddy K is working on some changes to the navcontainerhelper module that will remove the requirement to run the cmdlets as admin. That would be nice.

PowerShell Profile

Have PowerShell automatically execute some script on loading by editing your profile. PowerShell has a built-in $profile variable which points to the location of your .ps1 profile file.

I use that file to import the posh-git module (below) and our own TFS Tools module. You could create the file with something like this (sc is an alias for the Set-Content command):

sc $profile 'Import-Module posh-git
Write-Host "PowerShell Ready" -ForegroundColor Green'

Opening a new terminal will look like this:

VS Code Terminal PowerShell Ready.JPG

Note: PowerShell ISE has a different profile file to PowerShell.

Posh-Git

I mostly use Git from the command line. I started using the command line rather than a GUI as I found it helped me understand what commands are actually being used – how fetch is different to pull, how to set tracking information for a branch or edit a remote.

And yes, perhaps there is small part of it that boosts my shallow sense of “I’m a real developer, I type weird commands into a prompt rather than clicking a button on a GUI”. It’s OK to admit that. I draw the line at Vim though.

Anyway.

If you’re planning on using Git in PowerShell you’re going to want to install the posh-git module.

Install-Module posh-git

It adds some details into the prompt (see above): the branch that you are on, how it compares to the remote branch that it is tracking and the status of your index. It adds tab completion all over the place as well – indispensable.

Git Aliases

If you do start using Git from the terminal you’re probably going to find typing some of the longer commands quite tedious. For instance, git log –graph is great to get an overview of your project and has loads of switches to alter its output. I tend to use:

git log --graph --oneline --all

To show a graph of all the branches (remote as well as local) with commit details on a single line each.

Git Log Graph Oneline.JPG

It gets you something like the above. You can see the commits that each of the branches is pointing at, which branches commits are included in and how work has been merged over time.

I don’t want to type the full command out each time though. Fortunately, Git doesn’t force you to. You can create an alias. I have:

  • git lol – to show the above graph
  • git fap – to fetch all changes from the remote and prune any references to remote branches that no longer exist (I’ve never understood why Git doesn’t automatically remove references to remote branches that no longer exist)
  • git pff – pull and merge changes from the remote branch, as long as your branch can be fast-forwarded

Conclusion

There are lots of opportunities – more than 5 – to enhance and tune VS Code and PowerShell to make your daily work more efficient. Check it out.

Regular Expressions in Visual Studio Code Search

Having spent years developing in C/SIDE I still get a little giddy using Visual Studio Code’s modern IDE features. You know, finding references to a function, renaming symbols, peeking definitions. Also, being able to search across all of your source code. What a dream.

Even better, you can search for a regular expression – that very powerful, if not slightly bonkers, way of searching for text that your pattern(s).

I was searching through my code for “Post Code” – looking fields and variables that we’ve created. But my locals, parameters and function names won’t have a space between “Post” and “Code”. You can match “Post Code” and “PostCode” in one regular expression search.

These expressions can look a bit baffling to start with, but with a little knowledge and practice you can quickly search for specific things and cut down the results you need to look through:

Post ?Code

Space followed by ? indicates that the space is optional – find matches with and without the space.

Or find table fields that include “Post Code”:

field\(\d+;.*Post Code

Find “field” followed by an opening parenthesis (which must be escaped with a backslash), followed by any number of digits (\d = digit, + is one or more of them), followed by a semi-colon, then any number of character (. = any character except a new line), then “Post Code”

Or find function parameters that include Post Code or PostCode:

proc.*\(.*Post ?Code

Find “proc” (as in the keyword “procedure”) followed by any number of characters, then an opening parenthesis then “Post Code” where the space is optional.

Find a match from a group of terms:

(Sales ?Line|Purch.*Line)

Find “Sales Line” with an optional space and “Purch Line” with any number of characters in the middle (to match PurchLine, PurchaseLine, Purch. Line, Purchase Line etc.)

Regex cheat sheet: https://duckduckgo.com/?q=regex+cheat+sheet&ia=cheatsheet&iax=1