Test Explorer in Visual Studio Code

The July 2021 release of Visual Studio Code (1.59) introduced a new testing API and Test Explorer UI. From v0.6.0 this API is used by AL Test Runner.

Test Explorer Demo

Improvements

UI

The biggest improvement is the Test Explorer view which shows your test codeunits, their test methods and the status of each.

Hovering over a test gives you three icons to run, debug or open an editor at the test.

You can run and debug all the tests in a given codeunit by hovering over the codeunit name or run and debug all tests at the top.

The filter box allows you to easily find specific tests, which I’ve found useful in projects which several test codeunits and hundreds of tests.

You can also filter to only show failed tests or only test which are present in the codeunit in the current editor. The explorer supports different ways of sorting and displaying the tests.

Icons are added into the gutter alongside test methods in the editor. Left click to run the test or right click to see this context menu with more options.

The old “Run Test” and “Debug Test” codelens actions are also still added above the test definition.

Commands & Shortcuts

A whole set of new commands are introduced with keyboard chords beginning with Ctrl + ; The existing AL Test Runner keyboard shortcuts still work but there are some nice options in the new set – like “Test: Rerun Last Run” to repeat the last run test without having to navigate to it again.

Using the Test Explorer

Using the Test Explorer is pretty self-explanatory if you’ve already been using AL Test Runner. When you open your workspace/folder the tests should be automatically discovered and loaded into the Test Explorer view. On first opening all of the tests will have no status i.e. neither pass or fail – but results from now on will be persisted.

Running one or more tests – regardless of where you run them from (Test Explorer, Command Palette, CodeLens, Keyboard Shortcut) – will start a test run. You’ll see “Running tests…” in the Status Bar.

Once the test(s) have finished running you’ll see the results at the top of the Test Explorer, “x / y tests passed (z %)”, and the status icons by each test will be updated.

If the tests do not actually run e.g. because your container isn’t started then the test run will not finish and “Running tests” will continue to spin at the bottom of the screen. You can stop the run manually from the top of the Test Explorer, fix the problem and go again.

Share Docker Containers With VS Code Live Share

If you weren’t working remotely before 2020 then we’ve all had to guess used to it this year. Collaboration tools like Teams are invaluable for keeping up with teammates during the day, discussing work and bouncing ideas and banter off each other. Video calls, screen sharing, chat, shared documents, gifs – that’s all great – but not quite a replacement for sitting at the same desk and looking at some code together.

There is a better way.

What is Live Share?

If you’re not already using VS Code Live Share then check it out. Get the extension pack rather than just the Live Share extension itself which adds audio calling through the sharing session. Sign in with a Microsoft or GitHub account and then start a live share session. A URL is generated that you can share for others to join, from VS Code, Visual Studio or even a browser.

Others will be able to see the source code, navigate and edit it. You can see each other’s cursors and choose to follow someone or work independently. Just like real-time collaborating on a shared Google doc or Word document but in the IDE. Your own extensions, your own theme, but working on shared code.

Whether you’re into pair-programming or just occasionally want a colleague to look at your code and help with something this is a great solution. Being able to independently navigate the source code and go-to definitions is far better than simply screen sharing.

However, seeing the source code is one thing, you really want to be able to publish the changes and see them in the web client.

Sharing a Server

We develop against local Docker containers. Which is great for giving each of us complete control over our own environment. It’s not so great for sharing that environment with others. But wait, you can share your local Docker container through the Live Share session.

First, make sure that you are publishing all ports when you create the Docker container. This will publish the container’s internal ports (80, 443, 7045-7049, 8080) to ports on the host. You’ll be able to access ports inside the container via the ports they are mapped to on the host.

Run docker ps to see the which host ports the container has been hooked up to. In my example, I’m accessing the web client over http so I’m interested in what port 80 is mapped to.

0.0.0.0:60425->80 indicates that port 60425 on the host (my local machine) has been mapped to port 0 in the container. In which case, http://localhost:60425/bc should load the web client. Check that you can open a browser to that address and log in.

Docker container details with docker ps

Now we can share that address as a shared server. Click on Share server… and enter the web client URL in the dialog box. You can give a friendly name to the shared server as well if you like.

Once you’ve shared the container anyone else who has joined the session will be able to click on the link under Shared Servers. That will load the web client to your local Docker container through the Live Share session.

I’ll say that again – it will load the web client, running in your local Docker container, through the Live Share session 👀 As far as your collaborator is concerned they just load the same URL as you (http://localhost:60425/bc) and have access.

Before you ask, I don’t know. Probably magic.

Now What?

Now you can genuinely collaborate on some code, in real-time and publish to the same Docker container to see the results and test. Maybe you want to work together on some code, maybe you need want some help with a tricky bug or requirement. Last week I used it to ask a colleague to quickly test a fix for a bug that he had reported.

Use your imagination. This is just one of the many benefits that moving our development to VS Code has brought. Good times.

Tip: Case Insensitive Glob Patterns

I must admit I hadn’t heard of glob patterns until seeing some settings in VS Code that use them. They are a way of defining a pattern that matches one or more files and folders. For example, the Text Editor section of the settings (UI) in VS Code allows you to exclude files and folders from the file explorer with glob patterns and gives you this useful link for more information.

My AL Test Runner extension has a glob pattern setting to identify your test codeunits (used when identifying the file path and line number of a failing test to give you a clickable link in the output window).

Turns out that glob patterns are case-sensitive. You can, however, match from a group of characters (like regular expressions) with square brackets. Useful if you’ve got inconsistently named files. My Test Codeunit Glob Pattern setting (in AL Test Runner) is set to **Tests/*.[cC]odeunit.al i.e. match files in a Tests folder which end with either codeunit.al or Codeunit.al

Tip: Format AL Files OnSave in Visual Studio Code

Maybe everyone else is already doing this and I’m just slow on the uptake but Visual Studio Code has options to automatically format files at various points.

The AL extension for VS Code provides a formatter for .al files. You can run it manually with the Format Document command (Shift+Alt+F). This inserts the correct number of spaces between parameters and brackets, indents lines correctly and generally tidies the current document up.

You can have VS Code automatically format the document when pasting, typing and saving. Search for format in the settings.

These settings will be applied globally. Alternatively you can enable the formatting just for specific file types. Click on the AL link in the right hand corner of the status bar and choose “Configure AL language based settings…”

This opens the VS Code settings JSON file in your AppData folder (on Windows) and adds an [al] object to the file. Create the “editor.formatOnSave” and set its value to true to enable AL formatting when the files are saved. You can use intellisense (Ctrl+Space) to list the valid options in this file.

VS Code for the win.

AL Test Runner for Visual Studio Code

TL;DR

I’ve written an extension for VS Code to help run your AL tests in local Docker containers. Search for “AL Test Runner” in the extension marketplace or click here. Feedback, bugs, feature suggestions all gratefully received on the GitHub repo or james@jpearson.blog

Intro

As soon as Freddy added capability to the navcontainerhelper module to execute automated tests I was excited about the potential for:

  1. Making test execution in our build pipeline simpler and more reliable
  2. Running tests from Visual Studio Code as part while developing

I’ve written about both aspects in the past, but especially #1 recently – about incorporating automated tests into your Azure DevOps pipeline.

This post is about #2 – incorporating running tests as early as possible into your development cycle.

Finding Bugs ASAP

You’ve probably heard the idea – and it’s common sense even if you haven’t – that the cost of finding a bug in your software increases the later in the development/deployment cycle you find it.

If you realise you made a silly mistake in code that you wrote 2 minutes ago – there’s likely no harm done. Realise there is a bug in software that is now live in customers’ databases and the implications could be much greater. Potentially annoyed customers, data that now needs fixing, support cases, having to rush out a hotfix etc.

We’ve all been there. It’s not a nice place to visit. I once deleted all the (hundreds of thousands of) records in the Purch. Rcpt. Line table with a Rec.DELETEALL on a temporary table…turns out it wasn’t temporary…and I was working in the live database.

Writing automated tests can help catch problems before you release them out into the wild. They force you to think about the expected behaviour of the code and then test whether it actually behaves like that. Hopefully if the code that we push to a branch in Azure DevOps has a bug it will cause a test to fail, the artifacts won’t be published, the developer will get an email and the customer won’t be the hapless recipient of our mistake. No harm done.

However, the rising cost of finding a bug over time still applies. Especially if the developer has started working on something else or gone home. Getting back your head back into the code, reproducing and finding the bug and fixing it are harder if you’ve had a break from the code than if you went looking for it straight away.

Running Tests from VS Code

That’s why I’m keen that we run tests from VS Code as we are writing them. Write a test, see it fail, write the code, see the test pass, repeat.

I’ve written about this before. You can use tasks in VS Code to execute the required PowerShell to run the tests. The task gives you access to the current file and line no. so that you can fancy stuff like running only the current test or test codeunit.

AL Test Runner

However, I was keen to improve on this and so have started work on a VS Code extension – AL Test Runner.

Running the current test with AL Test Runner and navcontainerhelper

The goals are to:

  • Make it as simple as possible to run the current test, tests in the current codeunit or all tests in the extension with commands and keyboard shortcuts
  • Cache the test results
  • Decorate test methods according to the latest test results – pass, fail or untested
  • Provide extra details e.g. error message and callstack when hovering over the test name
  • Add a snippet to make it easier to create new tests with placeholders for GIVEN, WHEN and THEN statements

Important: this is for running tests with the navcontainerhelper PowerShell module against a local Docker container. Please make sure that you are using the latest version of navcontainerhelper.

Getting Started

  • Download the extension from the extension marketplace in VS Code and reload the window.
  • Open a folder containing an AL project
  • Open a test codeunit, you should notice that the names of test methods are decorated with an amber background (as there are no results available for those tests)
    • The colours for passing, failing and untested tests are configurable if you don’t like them or they don’t fit with your VS Code theme. Alternatively you can turn test decoration off altogether if you don’t like it
  • Place the cursor in a test method and run the “AL Test Runner: Run Current Test” command (Ctrl+Alt+T)
  • You should be prompted to select a debug configuration (from launch.json), company name, test suite name and credentials as appropriate (depends if you’re running BC14 or BC15, if you have multiple companies, authentication type etc.)
    • I’ve noticed that sometimes the output isn’t displayed in the new terminal when it is first created – I don’t know why. Subsequent commands always seem to show up fine 🤷‍♂️
  • Use the “ttestprocedure” to create new test methods

.gitignore

If you’re using Git then I’d recommend adding the .altestrunner folder to your .gitignore file:

.altestrunner/

Committing the config file and the test results xml files doesn’t feel like a great idea.