- 20 Aug 2021
- 4 Minutes to read
- Updated on 20 Aug 2021
- 4 Minutes to read
Frequently Asked Questions
We are working hard to bring all the information to you at the earliest. Kindly be patient as we update the FAQ with more details.
How many scenarios are there in the Safety Pool Scenario Database?
Currently there are in excess of 250000 scenarios populating the Safety Pool Scenario Database (SPSD). This is a steadily growing number thanks to the rolling contributions both from the Safety Pool founders and from the members of the Safety Pool community.
Are all the scenarios unique?
Scenarios, while being described at a logical level, are different to each other at a rather functional level. Meaning that there are no simple re-parametrizations of scenarios, but instead scenarios in SPSD have some substantial differences among each other either in the scenery, the environment conditions or in the unfolding of the actors' behavior.
Are the scenarios Concrete/Logical/functional?
SPSD gathers scenarios at the logical level (according to the PEGASUS project definition), detailing intervals/ ranges for the different scenario parameters.
The “concretization” of a scenario, so going from logical to a concrete scenario with a specific instantiation of parameters values can be achieved in different ways following different approaches and techniques. Concretization is currently out of scope of the SPSD and it is left to the individual organizations.
Where do the scenarios come from? How are they generated?
Scenarios are generated from 3 main sources:
|Scenarios generated as a result of an analysis of accident databases (such as STATS-19) or insurance claims records.|
|Scenarios mined and extracted from logged data ingested in real world drives of instrumented vehicles, sensor equipped infrastructure, or aerial data.|
|Handcrafted scenarios by human experts with a specific intent in mind or a specific function under test. Specifically, SP research team has evolved STPA analysis to generate and design scenarios for failure of a specific system. Learn more about Safety Pool STPA approach here.|
How are the scenarios organized or clustered?
There are multiple dimensions along which scenarios are organized or can be organized within SPSD:
|Libraries||Scenarios are grouped in libraries (folders) usually according to their generation or intent, e.g. the STATS-19 library gathers all the scenarios generated from a thorough analysis of the STATS-19 scenario database, the ALKS library gathers all the scenarios designed to test ALKS systems, etc.|
|Library type||Sourced vs Feature. Sourced libraries are libraries generated from data (database or real world logs), while feature libraries are generated by expert knowledge / engineered libraries.|
|Library restrictions||Private vs Public: Public libraries are accessible by all other SPSD authorised users within SPSD members organizations. Private libraries are privately accessible exclusively by a specific organization within its organization account.|
|Tags||Each and every scenario is tagged along three dimensions: ODD, Behavior, and Admin Data. The full breakdown hierarchy of tags can be found at: (link coming soon) Tags are also utilized for searching and filtering scenarios in the database.|
Scenario Tags are being currently standardized in a dedicated work package within the ASAM OpenLABEL standard.
How many ODDs are covered already, and how many scenarios are there for a specific OD class?
Currently scenarios already cover a diverse set of ODD attributes and maneuver types: SPSD gathers scenarios in urban environments, highways, under varied environmental conditions where actors perform different maneuvers (cut-ins, overtaking etc).
At the library level it is possible to have an aggregated view of the distribution of the tags for the scenarios so that the user can get an idea of the number of scenarios that fulfil certain characteristics within a library (How many scenarios in library x have rain conditions, how many take place in a roundabout etc.).
A more powerful and detailed dashboard visualization feature that conveys more detailed information about the ODD and characteristic distribution of scenarios to users is now under development.
How is the quality of the scenarios assured?
All the scenarios in SPSD are highly curated and validated for integrity. Automated Validation checks are in place to ensure consistency of SPSDL Grammar. Moreover, scenarios are tested on CARLA to ensure their executability in simulation
Do the scenarios in the Database cover different functions or use cases?
Currently there is a growing variety of use cases supported by SPSD scenarios, namely all ALKS, low speed shuttle, urban level 4, highway ADAS etc. We are working to extend this even more and we highly encourage scenarios contributions from the community to help grow and expanding further the variety of supported use cases
What Scenario Description Language are you using?
SPSD uses a custom developed Scenario Description Language: the Safety Pool Scenario Description Language (SPSDL). SPSDL supports 2 levels of abstractions and it is structured according to Scenery, Dynamic Elements and Environmental conditions.
The research paper for SPSDL can be found here
Detailed documentation for SPSDL can be found here
Why has Safety Pool developed a custom SDL?
Safety Pool has carried out extensive research with all the relevant stakeholders (including OEMs, Tier 1s, AV stack developers, Policy makers, governments and academia) to collect requirements for the development of SPSD.
Are standards such as OpenSCENARIO and OpenDRIVE supported?
Safety Pool highly supports standards and members of its steering committee cover key roles in many standard organizations and specific standardization projects (ISO, BSI, ASAM OpenX, etc.).
The Safety Pool team has been working on developing converters for OpenSCENARIO 1.x and OpenDRIVE. SPSD users will find the corresponding OSC1.x and ODRIVE1.x files in the “Files'' tab attached to every scenario (currently work in progress). OSC2.0 support will be available after the release of the standard.
My organization has developed a proprietary SDL and toolchain, can I still benefit from Safety Pool Scenario Database?
In order to maximize inclusion, Safety Pool is open to support the development of converters to organization specific and proprietary SDLs.
Please reach out to firstname.lastname@example.org to request assistance.