AL Test Runner for Visual Studio Code

Docs

https://jimmymcp.github.io/al-test-runner-docs/

Overview

The AL Test Runner extension for Visual Studio Code makes it easier to write and run automated tests in your AL apps for Microsoft Dynamics 365 Business Central. Read the intro blog post here. Or, you can watch an overview video just here…

Various development scenarios are supported by the extension including:

New blog posts about AL Test Runner will be posted here. The extension is published on the Visual Studio Code marketplace here.

Features

Open a test codeunit in VS Code and run one of the following commands:

  • Run current test (Ctrl+Alt+T) – the test which the cursor is currently in will be executed
  • Run tests in the current codeunit
  • Run all tests (Ctrl+Shift+Alt+T)
  • Debug current test (Ctrl+Alt+D) – the current test will be run by a custom test runner codeunit via a web service call and the debugger attached to that session (more about debugging here)
  • Calculate code coverage stats when tests are run (more about code coverage here)

The selected tests will be executed against your Docker container and the results shown in the output window. Test methods are decorated according to the latest result (passing, failing, untested). When tests fail the line which caused the failure is highlighted and the error message and callstack can be seen by hovering over the method name or failing line.

The output window will show the file path and line number of failing tests so that you can jump to the test definition (Alt + Click).

The style of the test method decorations and the failing line highlight can be configured.

Pre/Post Test Commands

Set PowerShell commands to be executed before/after tests are executed with Pre Test Command and Post Test Command.

These could be useful for:

  • Preparing the test suite in BC14 with the test codeunits and methods that need to be executed e.g. like this
  • Unpublishing apps after the tests have been run

The commands will be executed on the VS Code host (the local machine unless you are using Remote Development) but you can use the functions in the ALTestRunner module to invoke a command on the Docker host (see below). Multiple commands can be executed when separated by a semicolon.

Installation

Search for “AL Test Runner” in the Extensions sidebar in Visual Studio Code to download and install the extension.

Configuration

Use the Open Config File command to open the JSON config file.

Most of the required settings will be populated automatically when tests are run or you will be prompted to enter them.

Container Credential

If you are using NavUserPassword authentication in the container you must specify the user name and password to authenticate with. You will be prompted to enter these when running tests. Alternatively they can be entered with the Set Container Credential command.

Company

If there are multiple Business Central companies in the target container you will be prompted to select the correct company to execute the tests in.

Test Suite

If you are running tests in Business Central 14 you will be prompted to enter the test suite name that contains the test(s) to run. Note: in this case the test codeunits and methods must already be populated in the test suite before running the tests. (From Business Central 15 tests are automatically populated on the AL Test Tool page before they are run).

Remote Docker Host

If you are publishing your apps to a remote Docker host then the dockerHost key must be populated with the name of the host. Leaving this key blank implies that Docker is hosted on the local machine.

When using a remote host you may also need to specify the credentials that are used to authenticate with the host. These credentials can be set with the Set VM Credential command.

AL Test Runner will use PowerShell’s New-PSSession command to open a new remote PowerShell session to the host. You must be able to create a session with this command before you will be able to execute tests against a remote host. Credentials can be passed to New-PSSession by populating the VM Credential (as above). Any other options which must be passed to New-PSSession can be specified in the newPSSessionOptions key.

Extension Config

Separate to the config file, AL Test Runner has several configuration options which can be set through VS Code. They can either be applied globally or just to the current workspace.

  • Decorate Test Methods – determines whether test methods are decorated according to the last test result
  • Failing / Passing / Untested Test Color – defines the decoration colour for each result in RBGA format
  • Highlight Failing Line – determines whether the line which caused a test to fail is highlighted
  • Failing Line Decoration – the CSS style to decorate the failing line with
  • Publish Before Test – whether to publish the extension to the container before executing tests (select between None, Rapid application publish or Publish)
  • Test Codeunit Glob Pattern – the glob pattern which identifies test codeunits in the workspace. This is used to determine the filename of failing codeunits. By default all .al files in the workspace are searched. For large projects this might perform badly and a more precise glob pattern should be set e.g. **/Tests/*.Codeunit.al matches only files ending in .Codeunit.al in a Tests folder
  • Pre Test Command / Post Test Command – a PowerShell command to execute before and after tests have been executed

PowerShell Module

The ALTestRunner module will be imported into the al-test-runner terminal that is opened in VS Code when tests are executed. It will be imported when pre/post test commands are executed. Some functions that might be useful in that context are listed below.

  • Get-ALTestRunnerCredential returns a PSCredential object with the credentials to authenticate with the container. Add the -VM switch to return credentials to authenticate with the Docker host
  • Get-ContainerName returns the name of the container for both local and remote hosts
  • Get-DockerHostSession returns the PSSession object used to execute commands against the remote host
  • Get-ValueFromALTestRunnerConfig return the key value specified by KeyName from the AL Test Runner config
  • Get-ValueFromAppJson return the key value specified by KeyName from the app.json file
  • Get-ValueFromLaunchJson return the key value specified by KeyName from the app.json file
  • Invoke-CommandOnDockerHost takes a script block to execute on the Docker host

Telemetry

From v0.6.6 AL Test Runner sends some anonymous usage data to an Application Insights instance in my personal Azure subscription. I will not share any detailed information from the telemetry. I might occasionally post some summarised information that I think is of interest to the Business Central development community.

Why?

I’d like to know which features in the extension are being used. It might help me decide how to improve the extension in the future. I’d also like to if certain settings are enabled (do users have code coverage enabled, are they publishing apps from PowerShell, with the AL extension commands, or not at all?).

None of this data identifies users or the apps that they are working on. Even so, if you’re not OK with this anonymous data being sent you can set the Telemetry Level setting in VS Code to anything lower than all.

What?

  • Events when certain features are used:
    • Test run started
    • Test run finished
    • Test debug session started
    • Show Table Data used
    • (I will add to this list as I add more features)
  • The number of tests included in a test run (but not the name of the tests, or the results)
  • Specific extension settings
    • Whether code coverage is enabled
    • If apps are set to be published before the test run
    • If apps are being published from PowerShell (as opposed to using the commands from the AL Language extension)
  • Other data about the system which is automatically gathered by VS Code e.g. operating system, version of VS Code, version of AL Test Runner, approximate location from your IP address – see the below link for more information

Not included:

  • Anything that personally identifies you, your company or the machine you are working on
  • Anything that identifies the app that you are working on, i.e.
    • Any of your code
    • Anything from app.json
    • Anything from launch.json
    • The names of your tests

For more information see here: https://code.visualstudio.com/docs/getstarted/telemetry