Performance Profiling Tests with AL Test Runner

This is a feature that has been in AL Test Runner for a months now but I haven’t got round to blogging about it. I haven’t done much blogging or work on AL Test Runner for a while now. I started a new job at the start of the year and had more important stuff keeping me occupied, but Johannes (https://github.com/jwikman) has prompted me into some action with some contributions recently – thanks very much for that 🥳

Scenario

My original scenario was that we had some poorly performing code. Complex code. Complex code I hadn’t been involved with much before. Complex code that would be easy for me to break functionally, while trying to improve the performance.

My first task was to surround the code with some integration tests. I find this an effective way to learn some existing code. You have to learn how to construct the GIVENs – what is the data structure and what setup is required for each test? It also gives you an easy way to step through the code and see what’s going on when certain processes are run. Crucially it also gave me some confidence that I wasn’t completely screwing up the app that I was working on while I was changing it.

Comparing Performance

OK, so I’ve got some tests to validate the functional behaviour before and after my changes, but what about performance? We have the Performance Toolkit, but I think that is less about the performance of a single process and more about concurrency. The obvious choice is to use the Performance Profiler.

I wanted to leave the existing code paths intact and just use a setup field as a rudimentary feature flag to switch between the old code and the new code. But, I didn’t want to be opening the client to run the performance profiler page or initialize and download snapshot profiles in between each test. I wondered if it was possible to could automate capturing a profile and downloading it to the workspace somehow.

It was 🙂

Setup

There is a new setting “Enable Performance Profiler” which defaults to true. This uses some new functions in the Test Runner Service app. With each test run the performance profile is captured and downloaded to the .altestrunner folder in your test project.

This should all be handled automatically. The Test Runner Service app should be downloaded when required and the serviceUrl in the config file set automatically. Check the docs for the required setup if not though.

Use

There is a new icon in the status bar which will open the performance profile viewer with the latest trace.

With this I could run two tests with the old and the new and compare the results side by side. If you want to do that just take a copy of the first trace file as it will be overwritten by the second test.

The trace, of course, has other benefits too. It is much easier to see the whole callstack and use the links on the right hand side to jump straight into the code. This also gives the potential for other features. I could maybe do something that allows you to choose two tests to run and download separate traces for them all in one action? Or read through the trace file to update the test coverage map? Let me know whether either of those sounds interesting or if you have other ideas or issues.