scenic.syntax.translator

Translator turning Scenic programs into Scenario objects.

The top-level interface to Scenic is provided by two functions:

• scenarioFromString – compile a string of Scenic code;

• scenarioFromFile – compile a Scenic file.

These output a Scenario object, from which scenes can be generated. See the documentation for Scenario for details.

When imported, this module hooks the Python import system so that Scenic modules can be imported using the import statement. This is primarily for the translator’s own use, but you could import Scenic modules from Python to inspect them. 1 Because Scenic uses Python’s import system, the latter’s rules for finding modules apply, including the handling of packages.

Scenic is compiled in two main steps: translating the code into Python, and executing the resulting Python module to generate a Scenario object encoding the objects, distributions, etc. in the scenario. For details, see the function compileStream below.

Footnotes

1

Note however that care must be taken when importing Scenic modules which will later be used when compiling multiple Scenic scenarios. Because Python caches modules, there is the possibility of one version of a Scenic module persisting even when it should be recompiled during the compilation of another module that imports it. Scenic handles the most common case, that of Scenic modules which refer to other Scenic modules at the top level; but it is not practical to catch all possible cases. In particular, importing a Python package which contains Scenic modules as submodules and then later compiling those modules more than once within the same Python process may lead to errors or unexpected behavior. See the cacheImports argument of scenarioFromFile.

Summary of Module Members

Functions

compileStream	Compile a stream of Scenic code and execute it in a namespace.
compileTopLevelStream
compileTranslatedTree
constructScenarioFrom	Build a Scenario object from an executed Scenic module.
executeCodeIn	Execute the final translated Python code in the given namespace.
findConstructorsIn	Find all constructors (Scenic classes) defined in a namespace.
functionForStatement
gatherBehaviorNamespacesFrom	Gather any global namespaces which could be referred to by behaviors.
nameForStatement
parseTranslatedSource
partitionByImports	Partition the tokens into blocks ending with import statements.
peek
purgeModulesUnsafeToCache	Uncache loaded modules which should not be kept after compilation.
scenarioFromFile	Compile a Scenic file into a Scenario.
scenarioFromStream	Compile a stream of Scenic code into a Scenario.
scenarioFromString	Compile a string of Scenic code into a Scenario.
scenic_path_hook
storeScenarioStateIn	Post-process an executed Scenic module, extracting state from the veneer.
topLevelNamespace	Creates an environment like that of a Python script being run directly.
translateParseTree	Modify the Python AST to produce the desired Scenic semantics.

Classes

ASTSurgeon
AttributeFinder	Utility class for finding all referenced attributes of a given name.
Constructor
InfixOp
LocalFinder	Utility class for finding all local variables of a code block.
ModifierInfo
Peekable	Utility class to allow iterator lookahead.
ScenicFileFinder
ScenicLoader
ScenicModule
TokenTranslator	Translates a Scenic token stream into valid Python syntax.

Member Details

scenarioFromString(string, params={}, model=None, scenario=None, filename='<string>', cacheImports=False)

Compile a string of Scenic code into a Scenario.

The optional filename is used for error messages. Other arguments are as in scenarioFromFile.

scenarioFromFile(path, params={}, model=None, scenario=None, cacheImports=False)

Compile a Scenic file into a Scenario.

Parameters

• path (str) – Path to a Scenic file.

• params (dict) – Global parameters to override, as a dictionary mapping parameter names to their desired values.

• model (str) – Scenic module to use as world model.

• scenario (str) – If there are multiple modular scenarios in the file, which one to compile; if not specified, a scenario called ‘Main’ is used if it exists.

• cacheImports (bool) – Whether to cache any imported Scenic modules. The default behavior is to not do this, so that subsequent attempts to import such modules will cause them to be recompiled. If it is safe to cache Scenic modules across multiple compilations, set this argument to True. Then importing a Scenic module will have the same behavior as importing a Python module. See purgeModulesUnsafeToCache for a more detailed discussion of the internals behind this.

Returns

A Scenario object representing the Scenic scenario.

scenarioFromStream(stream, params={}, model=None, scenario=None, filename='<stream>', path=None, cacheImports=False)

Compile a stream of Scenic code into a Scenario.

topLevelNamespace(path=None)

Creates an environment like that of a Python script being run directly.

Specifically, __name__ is ‘__main__’, __file__ is the path used to invoke the script (not necessarily its absolute path), and the parent directory is added to the path so that ‘import blobbo’ will import blobbo from that directory if it exists there.

purgeModulesUnsafeToCache(oldModules)

Uncache loaded modules which should not be kept after compilation.

Keeping Scenic modules in sys.modules after compilation will cause subsequent attempts at compiling the same module to reuse the compiled scenario: this is usually not what is desired, since compilation can depend on external state (in particular overridden global parameters, used e.g. to specify the map for driving domain scenarios).

Parameters

oldModules – List of names of modules loaded before compilation. These will be skipped.

compileStream(stream, namespace, params={}, model=None, filename='<stream>', dumpScenic3=False)

Compile a stream of Scenic code and execute it in a namespace.

The compilation procedure consists of the following main steps:

1. Tokenize the input using the Python tokenizer.

2. Partition the tokens into blocks separated by import statements. This is done by the partitionByImports function.

3. Translate Scenic constructions into valid Python syntax. This is done by the TokenTranslator.

4. Parse the resulting Python code into an AST using the Python parser.

5. Modify the AST to achieve the desired semantics for Scenic. This is done by the translateParseTree function.

6. Compile and execute the modified AST.

7. After executing all blocks, extract the global state (e.g. objects). This is done by the storeScenarioStateIn function.

class Constructor(name, bases)

Bases: tuple

_asdict()

Return a new dict which maps field names to their values.

classmethod _make(iterable)

Make a new Constructor object from a sequence or iterable

_replace(**kwds)

Return a new Constructor object replacing specified fields with new values

bases

Alias for field number 1

name

Alias for field number 0

class ModifierInfo(name, terminators, contexts)

Bases: NamedTuple

Parameters

• name (str) –

• terminators (Tuple[str]) –

• contexts (Optional[Tuple[str]]) –

name: str

Alias for field number 0

terminators: Tuple[str]

Alias for field number 1

contexts: Optional[Tuple[str]]

Alias for field number 2

_asdict()

Return a new dict which maps field names to their values.

classmethod _make(iterable)

Make a new ModifierInfo object from a sequence or iterable

_replace(**kwds)

Return a new ModifierInfo object replacing specified fields with new values

class InfixOp(syntax, implementation, arity, token, node, contexts)

Bases: NamedTuple

Parameters

• syntax (str) –

• implementation (Optional[str]) –

• arity (int) –

• token (Tuple[int, str]) –

• node (AST) –

• contexts (Optional[Tuple[str]]) –

syntax: str

Alias for field number 0

implementation: Optional[str]

Alias for field number 1

arity: int

Alias for field number 2

token: Tuple[int, str]

Alias for field number 3

node: AST

Alias for field number 4

contexts: Optional[Tuple[str]]

Alias for field number 5

_asdict()

Return a new dict which maps field names to their values.

classmethod _make(iterable)

Make a new InfixOp object from a sequence or iterable

_replace(**kwds)

Return a new InfixOp object replacing specified fields with new values

partitionByImports(tokens)

Partition the tokens into blocks ending with import statements.

We avoid splitting top-level try-except statements, to allow the pattern of trying to import an optional module and catching an ImportError. If someone tries to define objects inside such a statement, woe unto them.

findConstructorsIn(namespace)

Find all constructors (Scenic classes) defined in a namespace.

class Peekable(gen)

Utility class to allow iterator lookahead.

class TokenTranslator(constructors=(), filename='<unknown>')

Translates a Scenic token stream into valid Python syntax.

This is a stateful process because constructor (Scenic class) definitions change the way subsequent code is parsed.

translate(tokens, dumpScenic3=False)

Do the actual translation of the token stream.

class AttributeFinder(target)

Bases: NodeVisitor

Utility class for finding all referenced attributes of a given name.

class LocalFinder

Bases: NodeVisitor

Utility class for finding all local variables of a code block.

translateParseTree(tree, constructors, filename)

Modify the Python AST to produce the desired Scenic semantics.

executeCodeIn(code, namespace)

Execute the final translated Python code in the given namespace.

storeScenarioStateIn(namespace, requirementSyntax)

Post-process an executed Scenic module, extracting state from the veneer.

gatherBehaviorNamespacesFrom(behaviors)

Gather any global namespaces which could be referred to by behaviors.

We’ll need to rebind any sampled values in them at runtime.

constructScenarioFrom(namespace, scenarioName=None)

Build a Scenario object from an executed Scenic module.