Step 3: Define the tests
Any python code can be turned into a self-marking challenge by creating a file called
You will only be able to view or edit this file if you have enabled experimental features on your profile page so make sure that you have checked that box when you click on your Google account profile picture:
tests.json file defines how code will be tested.
You can define two properties within the root json object:
expectproperty is an array of conditions that will be tested based on the user’s code or output when their code is run
provideproperty is an array that specifies what data the program will be provided with when tests are run
provide properties are optional in the
tests.json file. Normally, you will add a new item to the
expect array for each test or challenge but you only use
provide if you need the code to react in a certain way when the user’s python code contains
Example tests.json file
The following example is the simplest possible definition for a self-marking test.
The above example defines one test which checks if the code is equal to
This test will only pass if the code exactly and completely matches what you put in the
forproperty (on line 5 in the example above). If the user doesn’t use the same combination of capital letters , uses double quotes instead of single quotes or has additional lines of code then the test will fail.
The description of the test that appears in the testing table will be automatically generated, but it will rarely be what you want the user to see. In order to address these issues, you can specify additional information to customise each test:
The second example (above) sets the
match property to
contains so that what the test searches for can be found anywhere within the code. It also adds a
description property which overrides the text displayed in the testing table.
If you want more control over how the test is decided to be a pass or fail, you can add a
process property which defines an array of processes that will be applied to data before it is tested.
Line 7 of example 3 (above) means that the lines of the code will be counted, ignoring any blank lines. If there are 10 or more lines of code which aren’t blank, the code will pass the test.
The next page explains all of the different ways that you can customise the
expect array to define what your self-marking python activities will expect for each test.