v0.5.0 of AL Test Runner adds some capability to measure code coverage per object and highlight the lines of code that were hit in the previous test run.
This is an example of what we’re aiming for. Running one or more tests, seeing the list of objects and the percentage of their code lines that were hit, opening those objects and highlighting those lines. This should help identify objects and code paths aren’t covered by any tests. I don’t believe code coverage should be a target in itself (maybe more on that in a separate post) but it can be a useful tool to see where you might want to bolster your test suite.
Code coverage is started and stopped before each run of run or more tests. The Test Runner Service app is called to download the code coverage details and they are saved to a JSON file. This file is read and summarised into a per-object percentage which is output with the test results. Only objects which are defined in the workspace are included – so you won’t include standard objects, but you will see test objects.
The path to each object is included so you can Alt+Click to navigate to it. A new Toggle Code Coverage command (Ctrl+Alt+C) allows you to switch the highlighting for lines which have been hit on and off.
- Install the Test Runner Service app with the command in Visual Studio Code. If it is already installed you will need to uninstall and unpublish the existing version first
- In the AL Test Runner config.json file
- Set the path to save the code coverage JSON file to in the codeCoveragePath key. This path is relative to the folder that contains your test code e.g. .//.altestrunner//codecoverage.json to save it within the .altestrunner folder
- Set the testRunnerServiceUrl key to the OData URL of the Test Runner web service e.g. http://test:7048/BC/ODataV4/TestRunner?company=My%20Company&tenant=default for a container named test, default tenant and company called My Company
- In Visual Studio Code Settings
- Enable Code Coverage
- [Edit: this is now optional – see 0.5.3 update below] Select the path to the code coverage file relative to your app code i.e. if you have your test extension in a separate top level folder you might set it to ../tests/.altestrunner/codecoverage.json This allows AL Test Runner to find and display the code coverage details from an object in your app code
Use the Exclude Code Coverage Files to define file paths that should be excluded from the code coverage summary. This is a regex pattern which is matched against the file paths. For example, setting this to “Tests” will exclude any files with “Tests” in their path.
Test Folder Name – specify the name of the folder which contains the test app. Previously if you worked in a multi-root workspace and had an editor open at the production app it would attempt to run tests in the production app, create a config file, ask you which company to test in, prompt for credentials…meh. With this setting AL Test Runner will always run tests in the app which is contained in the folder with the name given in this setting.
Some of the early feedback from people who were trying to enable code coverage was that it was a bit of a game. And not in a good, fun way. More like Ludo. You’re trying to line up all your pieces but every time you think you’ve got them where you want them someone else lands on them and messes everything up.
From 0.5.3 it isn’t necessary to set the code coverage path in VS Code’s settings (see setup #3 above). If this path is not set then the extension will attempt to find the codecoverage.json file somewhere in the workspace.
The codeCoveragePath key in the AL Test Runner config file is still required, but has a default value which will be fine in most cases.
Ideas and feedback welcome as issues (or better yet, pull requests) in the repo on GitHub. These are some that I might work on.
- Maybe a setting to enter a glob pattern to include/exclude certain files and/or paths from the summary
- Smoother setup of the different settings that are required – I’ve tried to provide sensible default values but they are a few things to enter correctly to get it working
- Currently the code coverage file is overwritten with each test run – it would be more useful if this was cumulative so that running a single test didn’t overwrite all the results from a previous test run. If you want a complete overview you have to run the whole suite – but then maybe that isn’t a bad thing
- Perhaps an overall code coverage percentage for the whole project as well as per app (with the above caveat that I’m sceptical about fixed code coverage targets)
- A CodeLens at the top of an object with its percentage code coverage that toggles the highlighting on/off when clicked