Performance of Test Code

Let’s talk about the performance of the test code that we write for Business Central. What do I mean by “performance” and how can we improve it?

Defining “Performance”

Obviously, before we set out to improve something we need to have an idea of what it is we’re trying to optimise for. I’m coming to think of the performance of test code in a couple of key ways:

1. How easy/quick is it to write test code?
2. How quickly do the tests run?

Performance of Writing Tests

I suppose none of the below points are specific to test code. They are relevant to any sort of code that we are writing but we can be more inclined to neglect them for test code than production code. If you embrace any sort of automated tested discipline you’re going to spend a significant proportion of your time reading and writing test code – perhaps even as much as you do on production code. It is well worth investing a little time to clean up the code and making it easier to read and maintain.

Comments

Say what you like about comments in production code – variable names should declare the intent, comments are evil blah blah – I do find a few comments valuable in a test.

In fact, I write them first. Given some set of circumstances, when this happens then this is the expected result. Writing those sentences first helps to be clear about what I’m trying to test and what the desired behaviour actually is.

Of course the code in between those comments should be readable and easy to follow – but if you are diligent with a few comments per test you can describe the expected behaviour of that part of the system without having to read any code.

Grouping Tests Logically

I appreciate the situation is different for VARs but as an ISV we have large object ranges to do our development in. There is no reason for us to bundle unrelated code into the same codeunit. If we are starting work on separate from existing business logic then it belongs in its own codeunit. In which case, why wouldn’t the corresponding tests also go in their own codeunit?

Ideally I want to be able to glance down the list of test codeunits that we have and see logical grouping of tests that correspond to recognisable entities or concepts in the production code. If I want to see how our app changes the behaviour of warehouse receipts I can look in WhseReceiptTests.Codeunit.al.

Refactoring Into Library Codeunits

As soon as you start writing tests you’ll start working with the suite of library codeunits provided by Microsoft. You’ll notice that they are separated into different areas of the system e.g. Library – Sales, Library – Warehouse, Library – Manufacturing and so on.

Very likely you’ll want to create your own library codeunit to:

• Initialise some setup, perhaps create your app’s setup record, create some No. Series etc.
• Create new records required by your tests – it is useful to follow the convention LibraryCodeunit.CreateXYZ(var XYZ: Record XYZ);
• Consider having those Create functions returning the primary key of the new record as well so the result can be used inline in the test e.g.
  • LibraryCodeunit.CreateXYZ(var XYZ: Record XYZ): Code[20]
  • LibraryCodeunit.CreateXYZNo(): Code[20]
• Use method overloads for the create functions – have an overload with more parameters when you need to specify extra field values but keep a simple overload for when you don’t
• Identify blocks of code that are often required in tests and consider moving them to a library method e.g. creating an item with some bespoke fields populated in a certain way

Having a comprehensive library codeunit brings two benefits:

1. Tests are easier and faster to write if you already have library methods to implement the Given part of the test
2. Less code in the tests, making them easier to read

Performance of Running Tests

First, why do we care about how long tests take to run? Does it really matter if your test suite takes an extra minute or two to run?

Obviously, we want our builds to complete as quickly as possible, while still performing all the checks and steps that we want to include. The longer a build takes the more likely another one is going to be queued at the same time and eventually someone is going to end up having to wait. We’ve got a finite number of agents to run simultaneous builds (we host our own – more on that here if you’re curious).

But that isn’t the biggest incentive.

I’m a big fan of running tests while I’m developing – both new tests that I’m writing to cover my new code and existing tests (more on that here). I usually run:

• the test that I’m working on very frequently (at least every few minutes – see it fail, see it pass)
• all the tests in that codeunit frequently (maybe each time I’ve finished with a new test)
• the whole test suite every so often (at least 2-3 times before pushing my work and creating a pull request)

After all, if you’ve got the tests, why not run them? I should know sooner rather than later if some code that I’ve changed has broken something. If the whole test suite takes 60 seconds to run that’s fine. If it takes 10 minutes that’s more of a problem.

In that case I’ll be more inclined to push my changes to the server without waiting, keep a build agent busy for half an hour, start working on something else and then get an email saying the build has failed. Something I could have realised and fixed if I’d run the test suite before I pushed my changes.

So, how to make them faster?

Minimal Setup

First, only create the scenario that is sufficient for your test. For example, we work with warehousing functionality a lot. If we’re testing something to do with warehouse shipments do we need a location with advanced warehousing, zones, bin types, bins, warehouse employees…?

Probably not. Likely I can create a location without Bin Mandatory or Requires Pick and still create a sufficient test.

If you need ledger entries to test with you may be able to create and post the relevant journals rather than creating documents and posting them. Creating an item ledger entry by posting an item journal line is faster than posting a sales order.

Or, you probably want to prevent negative inventory in real life – but does that matter for your test? Save yourself the trouble of having to post some inventory before shipping an item and just allow it to go negative.

Try to restrict the setup of your tests to what is actually essential for the scenario that you are testing. Answering that question is, in itself, a useful thought process.

Shared Fixture

Better yet, set something up once per test codeunit and reuse it in each of the tests. This what Luc van Vugt refers to as a “shared fixture”. You should check out his blog for more about that.

I feel a little mixed about this. I like the idea that each test is entirely responsible for creating its own given scenario and isn’t dependent on anything else but this is denying that this is faster. Finding a posted sales invoice that already exists is much faster than creating a customer, item, sales order and shipping and invoicing it.

No Setup

What is even faster than setting up some data one time? Doing it no times. Depending on what you are testing you may just be able to insert the records you need or call field validation on a record without inserting it.

If I’m testing that validating a field behaves in a certain way I may not need a record to actually be inserted in the table.

Alternatively if you need a sales invoice header to test with you might be able just to initialise it and call SalesInvHeader.Insert; It feels so wrong – but if your test just needs a record and doesn’t care where it came from, who cares? It will all be rolled back after the tests have run anyway.