expect reference guide
Each test that you write in your
tests.json file is defined by a new object in the expect array:
These are the properties that are currently supported for each test item in the
search property defines where the test will look to find what is looking for. It can have the following string values:
file: searches to see if a file exists, or if you specify a
nameproperty it will search the contents of that file
code: searches the content of the default file (
errors: searches through any error messages
output: searches through any program output
input: searches through any prompts displayed when the user is asked to input a value
Defining where to search
globals: searches to see if a global variable exists, or if you specify a
nameproperty it will search the contents of that variable
Line 4 and 5 of example 4 (above) means search through the list of files to see if there’s one called
name property is only useful if the search property is either
globals. It is used to specify the name of the file or global variable to be tested.
Lines 4-6 of example 5 (above) means search through the file called
text.txt and check if it contains just the text
for property defines what the test will search for (within the data specified by the search property)
TheDefining what to look for
forproperty can be an integer, float or boolean value but is usually a string which specifies what the test is looking for
match property defines the way the test will compare what it finds with what is specified by the
for property. It can be any of the following values:
=: data specified by
searchproperty (when the test runs) must exactly match the data specified by the
forproperty (when the test is defined).
>: data specified by
searchproperty must be greater than the data specified by the
<: data specified by
searchproperty must be less than the data specified by the
>=: data specified by
searchproperty must be greater than or equal to the data specified by the
<=: data specified by
searchproperty must be less than or equal to the data specified by the
Defining how to look
contains: data specified by
searchproperty include the data specified by the
process property allows the data specified by the
search property to be processed before it is compared with what you specify in the
for property. If defined, the process property must be an array that can include any of the following strings:
ignoreblanks: any empty lines are ignored
count: the number of characters / lines / files is tested rather than the data itself. This is the same as length (see below)
length: as above, the number of characters / lines / files is tested rather than the data itself
lower: the data is converted to lowercase
upper: the data is converted to uppercase
Defining how to process data before testing
lines: the data is split into separate lines (useful for combining with
countto count the number of lines)
Example 6 (above) shows how to use the process property to count the number of error messages so that the test will pass if there are no syntax or runtime errors.
description property lets you customise how each test is displayed in the test table.
The test in Example 7 (above) behaves in exactly the same as example 6 but line 7 defines the text which describes that test in the testing table.
You can define as many tests as you like by creating each one as a new item in the
expect array in the
Example 8 (above) defines three separate tests.
Test 1 (lines 3-8) counts all of the error messages and checks if there are none of them.
Test 2 (lines 10-13) searches the list of files to see if a file called
Test 3 (lines 15-19) searches the file called
output.txt to see if it contains just the text
The next page is a reference guide for how to use the
provide array in
tests.json in order to provide input data to each test as it runs.