Seeing as I’m on a bit of a run of posts about testing, let’s look at it from a slightly different angle.

Testing the Test

If we’re going to rely on automated tests to verify that our code (still) works then we need to have confidence that the tests themselves actually work.

Writing the Test First

This is why it is helpful to write and run the tests first. When you start developing a new feature or working on a bug fix you have identified some desired behaviour that the system doesn’t yet exhibit. Given this and this, when something or other then this is the behaviour I’m expecting.

Writing a test for that behaviour and seeing it fail confirms that the desired behaviour is missing. That gives you some confidence that you’re on the right lines – the system should do this, but doesn’t – yet.

When you write the bug fix or new feature and see the test pass it gives you much more confidence that your code actually works. You demonstrated beforehand that the desired behaviour was missing and that now it is there. Have a gold star.

Writing the Test Afterwards

You could write the test afterwards and we’ve done a lot of that as we’ve built up tests for our older code that didn’t have any. Whenever I write tests after the fact I do miss the initial stage of having an expected failing test though.

Not completing the given or the when can be a useful way to test the test. Asserting the expected results when you haven’t done all the required steps should normally cause the test to fail.

For example:

//[GIVEN] an item with my bespoke field populated
LibraryInventory.CreateItem(Item);
SomeBespokeValue := ...;

//leave these lines commented out initially to see the test fail
//Item.Validate("Bespoke Field",SomeBespokeValue);
//Item.Modify(true);

//[WHEN] the item is validated on a sales line
LibrarySales.CreateSalesDocumentWithItem(...)

//[THEN] some bespoke field on the sales line should be set
Assert.AreEqual(SalesLine."Bespoke Field",SomeBespokeValue,...)

Seeing the test fail with those lines commented out and then seeing it pass when you uncomment them will give you more confidence that the test and the behaviour that it is testing work as required. If you are testing code that you think already works and the test always passes it is hard to be sure why the test is passing. Hopefully because the code works – but possibly because the test itself is broken and will always pass, even if the code doesn’t work.

Confidence

The point is to try and get some confidence in your test results. Are you happy to ship the software when all your tests pass? If not, why not? Because you don’t have enough tests? Because you don’t trust that a passing test means working software?

Having a bunch of tests whose results you don’t trust is probably worse than having no tests at all.

Business Central

This is all pretty generic and if you’re interested in the principles you can search for Test-Driven Development (TDD) or Behaviour-Driven Development (BDD) and read what people far more qualified than me have to say about it.

Let’s talk about Business Central specifically for a minute. One of the best things about automated testing compared to manual testing is that everything is rolled back at the end to return the database to the same state it was in at the start. However, that can make life a little difficult when you are trying to inspect the data mid-test and see what is happened.

There are various ways you might want to extract the data at a given moment: write to a file, throw an error with a bunch of values you are interested in, read uncommitted data in SQL. We’ll just talk about two approaches:

Debugger

You can debug test code just like any other code. Set a breakpoint in your test, attach the debugger and run the test from the Test Tool page. Step through, add watches and evaluate debug expressions. The debugger in VS Code is getting better all the time, exposing more details about the variables you are interested in and SQL statements that have been executed.

Perfect for diving into the details and stepping through line by line, but not always the easiest to get an overview of what is happening.

Another Client Session

Another option is to open another client session while you are debugging. Set a breakpoint, attach the debugger and start running a test from the Test Tool page.

Debugging the test will block the session that you started it from – you’ll get the “working on it” dialog – but you can open a different session in another tab or in another browser.

The only snag with this is that some of the records that you want to read might be locked and you’ll get an error trying to open the corresponding page.

“The operation could not complete because a record was locked by another user.” Bummer.

Turns out there is another way to read the data in that session.

Avoid Locks With Page=<pageid>

You can add parameters to the web client URL to navigate to specific tables, reports or pages. In my example I can’t open the Items list from the menu because the record is locked by another user.

If I go to the Item List page with http://<base web client URL>?page=32 then the page loads with my test data. I can open the item card, navigate to other pages and run the Page Inspector (Ctrl+Alt+F1) to view all the fields in the table, filters, extension details etc.

As I step through the code in the VS Code debugger I can refresh the pages in this session and see the updates to the record. Beautiful.

Further Reading

If you’re interested in getting stuck into testing in Business Central grab yourself a copy of Automated Testing in Microsoft Dynamics 365 Business Central.

It was my pleasure to make a small contribution to this book as a technical reviewer and writing the foreword.

https://www.packtpub.com/business/automated-testing-microsoft-dynamics-365-business-central

Another instalment of my musings on running automated tests for Microsoft Dynamics 365 Business Central from Visual Studio Code.

Objective

What are we up to this time? As a brief reminder, I’m trying to make it as easy as possible to run automated tests from Visual Studio Code. I figure the faster and simpler it is to publish your code changes and run the tests the more inclined you are going to be to do it. The more you test the better your code will be.

If you’re trying to follow a discipline like Test-Drive Development you need tight feedback loops writing code and running tests. Being able to do that from the IDE and without having to switch back and forth to the browser is so much nicer.

To that end, if you are working on a particular test codeunit maybe it makes sense to only run the tests in that codeunit – in the interests of getting the feedback from those tests as quickly as possible. Or maybe just running the single test method that you’re working on. Once, you’re happy with those changes you can run the whole test suite again to make sure you haven’t broken anything else in the meantime.

Fortunately, it turns out we’ve already got the pieces we need to assemble this jigsaw.

NavContainerHelper

Being the considerate sort of chap that he is, Freddy has already included testCodeunit and testFunction parameters for the Run-TestsInNavContainer function. We just need to figure out how to plug the right values into those parameters.

Some More About VS Code Tasks

In this post I showed how you can define a task in VS Code to run some PowerShell. It turns out you can do more in tasks.json than I realised.

This is what my file looks like now:

{
    // See https://go.microsoft.com/fwlink/?LinkId=733558
    // for the documentation about the tasks.json format
    "version": "2.0.0",
    "tasks": [
        {
            "label": "Run BC Tests",
            "type": "shell",
            "command": "Run-TestsFromVSCode -CurrentFile ${file} -CurrentLine ${lineNumber}",
            "group": {
                "kind": "test",
                "isDefault": true
            },
            "presentation": {
                "echo": true,
                "reveal": "always",
                "focus": false,
                "panel": "shared",
                "showReuseMessage": true,
                "clear": true
            }
        }
    ]
}

The presentation object controls the behaviour of the terminal that the task is run in. I’m just using it to clear the terminal each time.

The really interesting part is ${file} and ${lineNumber}. These placeholders are replaced with the current file and current line number in the editor when the task is executed. Ooooo. You can probably see where I’m going with this. That’s all the information we need to run the current test codeunit and method.

I’ve created a new Run-TestsFromVSCode function in our PowerShell module to handle this. Notice I’m calling that function from the task now instead of our Run-BCTests function directly.

Run-TestsFromVSCode

Assuming a file path has been passed to the function it determines whether the current file is a test codeunit i.e. does it contain the text “Subtype = Test”? If so, use Regex to find the id of the codeunit from the first line.

Now attempt to find the previous declaration of a test function from the current line no (search backwards for a line that contains “[Test]” then forwards from that line to a procedure declaration).

Then call Run-BCTests with all the information that you’ve been able to find. Passing an asterisk rather than blank for the codeunit and/or method acts as a wildcard and will run everything, instead of nothing.

function Run-TestsFromVSCode {
    param(
        # The current file the tests are invoked from
        [Parameter(Mandatory=$false)]
        [string]
        $CurrentFile,
        # The current line no. in the current file
        [Parameter(Mandatory=$false)]
        [string]
        $CurrentLine
    )

    if ($null -eq $CurrentFile) {
        Run-BCTests
    }
    else {
        # determine if the current file is a test codeunit
        if ((Get-Content $CurrentFile -Raw).Contains('Subtype = Test')) {
            $TestCodeunit = [Regex]::Match((Get-Content $CurrentFile).Item(0),' \d+ ').Value.Trim()
            if ($null -ne $CurrentLine) {
                $Method = Get-TestFromLine -Path $CurrentFile -LineNumber $CurrentLine
                if ($null -ne $Method) {
                    Run-BCTests -TestCodeunit $TestCodeunit -TestMethod $Method
                }
                else {
                    Run-BCTests -TestCodeunit $TestCodeunit -TestMethod '*'
                }
            }
        }
        else {
            Run-BCTests
        }
    }
}

function Get-TestFromLine {
    param (
        # file path to search
        [Parameter(Mandatory=$true)]
        [string]
        $Path,
        # line number to start at
        [Parameter(Mandatory=$true)]
        [int]
        $LineNumber
    )
    
    $Lines = Get-Content $Path
    for ($i = ($LineNumber - 1); $i -ge 0; $i--) {
        if ($Lines.Item($i).Contains('[Test]')) {
            # search forwards for the procedure declaration (it might not be the following line)
            for ($j = $i; $j -le $Lines.Count; $j++)
            {
                if ($Lines.Item($j).Contains('procedure')) {
                    $ProcDeclaration = $Lines.Item($j)
                    $ProcDeclaration = $ProcDeclaration.Substring($ProcDeclaration.IndexOf('procedure') + 10)
                    $ProcDeclaration = $ProcDeclaration.Substring(0,$ProcDeclaration.IndexOf('('))
                    return $ProcDeclaration
                }
            }
        }
    }
}

Export-ModuleMember -Function Run-TestsFromVSCode

The terminal output from the task now looks like this:

Notice the current file and line number passed to the task and the current codeunit id and method passed to navcontainerhelper to run.

To run all the methods in the current codeunit move the cursor above the line of the first test method declaration.

To run all methods in all codeunits move the cursor outside a test codeunit – although you must have some file open otherwise VS Code will fail to resolve the ${file} and ${lineNumber} placeholders.

You could of course define more tasks to run these options if you prefer.

Conclusion

All of this enables this kind of workflow, if that’s what you’re into:

1. Create a new (failing) test and publish app
2. Run (just) that test and check that it fails
3. Write production code and publish app
4. Run (just) that test and check that it now passes
5. Run all the tests in that codeunit / your whole suite
6. Commit your changes

Wish List

I’m pleased with how far I’ve been able to get, but there’s still some significant items not crossed off the wish list.

Debugging Tests

For now you still have to launch the browser (at least we can debug without publishing these days) and start the test from that session. I believe the old “debug next” functionality we used to have in the Windows client debugger is on the roadmap somewhere. That would do the trick.

Performance

Leaves a lot to be desired. Running a single method in a single test codeunit takes anywhere between 6 and 10 seconds. Obviously, some of that time is the test itself and is in our control, but most of that is preparing the suite and creating and connecting the client.

I know Microsoft are overhauling how tests are executed by the platform, so maybe some performance gains are also in the pipeline.