Parallel test execution in Go

efrag

Since version 1.7 the Go testing package provides the ability to run some of the tests in parallel (well not entirely in parallel but that is another blog post 😉).
This is optional functionality you can enable by adding a single line of code t.Parallel() in your subtests. But how does that work exactly and what does Go actually do with these tests?

Setting up

You can find the source code for this blog post under my Github profile. The repository includes the sort library with an implementation of the Counting Sort algorithm (a stable sorting algorithm) along with the tests that make sure that our code works.
The structure of the repository is shown below – we can see the count_sort.go package and the corresponding test package (which is named the same just suffixed with _test).

Sort library directory structure

Tests, table tests and sub-tests in Go

Each test in Go is a function inside an _test.go file that roughly looks like this:

Basic Go test of our counting sort algorithm

This test defines the unsorted array of integers, the expected outcome (the sorted array), runs the CountSort algorithm and compares the result to see if our algorithm managed to correctly sort the input.
Inside count_sort_test.go we can define multiple of these test functions for all the different use cases we can think of. But this is not considered to be best practice as it will result to our file becoming really long and messy really quickly. So here come Go’s table tests.

Go table tests

Table tests are similar to the Data Providers that you can find in languages like PHP and Java. They basically take all the different test cases for a particular test and create a structure that we can then loop through from within our test function.

An example of a Go table test

Each test case is given a Name (used for the debug output and for more verbose comments if our tests fail), a list of Numbers to sort, the Expected outcome of our sorting algorithm and the Range (needed for the algorithm itself). Then in our for loop we go through each of the test cases and we trigger t.Run that defines a sub-test in Go.
This significantly improves the readability of our code but still these tests will run sequentially – which means the more test cases we add the slower our test run is going to become. So here come Go’s parallel tests !

Go parallel tests

All we need to add in our sub-tests to enable the above test cases to run in parallel is this line t.Parallel(). So let’s add one more test function namedTestCountSortParallel and amend the for loop from the above example to add the line(s) required for the parallel run.

for _, tc := range testCases {
  tc := tc
  t.Run(tc.Name, func(t *testing.T) {
    t.Parallel()
    res := CountSort(tc.Numbers, tc.Range)
    assert.True(t, assert.ObjectsAreEqualValues(tc.Expected, res))
  })
}

Now this is great ! We have 3 tests in our test package:

  • the basic one that runs 1 test case
  • the table one that includes 2 test cases and the for loop
  • the parallel one that also includes 2 test cases

And if we introduce a bit of a delay in the CountSort function (say 2 seconds) we can make our tests last a bit longer so that we can see in the output: 

PASS
ok      github.com/efrag/common-algorithms/sort 8.005s

Go runs tests sequentially in the same package but can run some of the test cases in parallel if we have defined them as such, ie. the execution for our test package now looks like this

Execution of go tests in the count_sort_test package

And if we combine all 3 tests in a single test with 5 test cases our entire execution time would be a little more than 2 seconds.

Tip

If you noticed we actually added 2 lines (the t.Parallel() on line 4 and the tc := tc on line 2. From the documentation of the testing package we have: 

 tc := tc // capture range variable

which was a bit of a mystery for me when I first saw it. Also running the CountSortParallel with and without that line produces a PASS so what’s the meaning of this line 🤔 ?

In order to test what this line actually does we add one more line in our sub-test that prints the name of the test case and one more line in the CountSort library that prints the input array. Then we run our tests twice one with the tc := tc line and one without and we check out the results.
[ℹ️ if you run the go test command with the -v flag you will be able to see additional debug output for your tests].

=== RUN   TestCountSortParallel
=== RUN   TestCountSortParallel/All_the_numbers_in_the_range_[1-9]
=== PAUSE TestCountSortParallel/All_the_numbers_in_the_range_[1-9]
=== RUN   TestCountSortParallel/3_numbers_in_the_range_[1-9]
=== PAUSE TestCountSortParallel/3_numbers_in_the_range_[1-9]
=== CONT  TestCountSortParallel/All_the_numbers_in_the_range_[1-9]
All the numbers in the range [1-9]
=== CONT  TestCountSortParallel/3_numbers_in_the_range_[1-9]
3 numbers in the range [1-9]
[4 1 9 6 3 8 7 2 5]
[4 1 9]
--- PASS: TestCountSortParallel (0.00s)
    --- PASS: TestCountSortParallel/All_the_numbers_in_the_range_[1-9] (2.00s)
    --- PASS: TestCountSortParallel/3_numbers_in_the_range_[1-9] (2.00s)

And then after commenting out line 2 and running the tests again:

=== RUN   TestCountSortParallel
=== RUN   TestCountSortParallel/All_the_numbers_in_the_range_[1-9]
=== PAUSE TestCountSortParallel/All_the_numbers_in_the_range_[1-9]
=== RUN   TestCountSortParallel/3_numbers_in_the_range_[1-9]
=== PAUSE TestCountSortParallel/3_numbers_in_the_range_[1-9]
=== CONT  TestCountSortParallel/All_the_numbers_in_the_range_[1-9]
3 numbers in the range [1-9]
=== CONT  TestCountSortParallel/3_numbers_in_the_range_[1-9]
3 numbers in the range [1-9]
[4 1 9]
[4 1 9]
--- PASS: TestCountSortParallel (0.00s)
    --- PASS: TestCountSortParallel/3_numbers_in_the_range_[1-9] (2.00s)
    --- PASS: TestCountSortParallel/All_the_numbers_in_the_range_[1-9] (2.00s)

Which basically means that if we do not properly capture the range variable we will be in a situation where we think we run both test cases and passed but in reality we would have only ever executed one of the two test cases!

Leave a Reply