Testing Toolchain Integration Overview

The primary use case of the SPSD is for enabling users to identify scenarios to test with and making those scenarios available for them to test with, either in simulation of the real-world. To achieve this, the SPSD provides an extensive, searchable database of scenarios, and facilities for downloading the scenarios, either manually, or via the API.

Interactions between an Organisation and the SPSD

 External testing systems will be able to download scenarios from the SPSD to feed into a testing toolchain, which could be a simulated environment with logical scenarios from the SPSD being used to generate test cases to be executed in a simulation or a live environment with scenarios being passed to test engineers to setup real-world testing.

Method of Operation

The User will login into the SPSD Web Application and will be able to search for scenarios which meet their testing requirements. This will be primarily done be using the Scenario Search functionality to search for scenarios with the user selecting a set of ODD tags which correspond with the elements which must be present in their test cases. The set of tags will then be used to search for scenarios in the database that match these tags. The search facility allows for complex search criteria to be constructed which includes the ability to: use logical operators, exclude tags, use hierarchical tag searches, specify values and ranges of values for tags where applicable, and specify which scenario libraries to search in.

It is envisaged that rather than performing ad-hoc searches for scenarios, that users will want to identify sets of scenarios to test with, and for this purpose, the database has functionality in the form of Test Suites which allow users to select and group scenarios together into sets geared around a particular testing objective. The benefit of using Test Suites is they provide a means for easily downloading a set of scenarios together, and that they can be re-used at a later date.

To further assist users in identifying scenarios, the SPSD has the facility to allow users to search Test Bed route data, for locations where scenarios can be tested in the real-world. This helps users optimise their testing resources for their testing objectives. 

Having created a Test Suite and added scenarios to it, the scenarios are then available to be extracted from the database in order to supply to a testing system which can be either done manually or via the API. 

The SPSD User Interface has facilities for downloading scenarios individually or as a set from a Test Suite, and allows for ad-hoc scenario retrieval, and for situations where integration via the API is not possible. The files are downloaded to the user’s browser as JSON files, and can then be copied to a location for the user’s testing system to pick up. However, the recommended method of retrieving scenarios is via the API

In order to retrieve scenarios via the API, the external system needs to be made aware of which Test Suite needs to be retrieved - this could be done in one of two ways: the User could make a note of the unique identifier of the Test Suite as displayed in the SPSD User Interface and then enter it into the external system, or the external system could present the User with a list of Test Suites retrieved using the API so that the user can select which one to test with. The external system would then be able to use the API to retrieve the scenarios for the chosen Test Suite and feed them into the external testing toolchain.

Once the external testing system has downloaded the scenarios, it will be its responsibility to generate the actual test cases and execute them.