Create your own self-marking python tests

Step 3: Define the tests

Any python code can be turned into a self-marking challenge by creating a file called tests.json

You will only be able to view or edit this file if you have enabled experimental features on your profile page so make sure that you have checked that box when you click on your Google account profile picture:

The tests.json file defines how code will be tested.

You can define two properties within the root json object: expect and provide

The expect property is an array of conditions that will be tested based on the user’s code or output when their code is run

The provide property is an array that specifies what data the program will be provided with when tests are run

Both the expect and provide properties are optional in the tests.json file. Normally, you will add a new item to the expect array for each test or challenge but you only use provide if you need the code to react in a certain way when the user’s python code contains input()

Example tests.json file

The following example is the simplest possible definition for a self-marking test.

The above example defines one test which checks if the code is equal to print('Hello World')

Note:

This test will only pass if the code exactly and completely matches what you put in the for property (on line 5 in the example above). If the user doesn’t use the same combination of capital letters , uses double quotes instead of single quotes or has additional lines of code then the test will fail.

The description of the test that appears in the testing table will be automatically generated, but it will rarely be what you want the user to see. In order to address these issues, you can specify additional information to customise each test:

The second example (above) sets the match property to contains so that what the test searches for can be found anywhere within the code. It also adds a description property which overrides the text displayed in the testing table.

If you want more control over how the test is decided to be a pass or fail, you can add a process property which defines an array of processes that will be applied to data before it is tested.

Line 7 of example 3 (above) means that the lines of the code will be counted, ignoring any blank lines. If there are 10 or more lines of code which aren’t blank, the code will pass the test.

The next page explains all of the different ways that you can customise the expect array to define what your self-marking python activities will expect for each test.