mirror of https://github.com/briot/gnatbdd
88 lines
3.3 KiB
ReStructuredText
88 lines
3.3 KiB
ReStructuredText
***************
|
|
The test driver
|
|
***************
|
|
|
|
The test driver is the executable generated by GNATbdd. In fact, there could
|
|
be several drivers built, depending on how you chose to run GNATbdd. For
|
|
instance, one of the driver might stub part of your application, whereas
|
|
another driver would test that same part and stub another part.
|
|
|
|
Finding features
|
|
================
|
|
|
|
Let's first examine how the driver finds all the features for your application.
|
|
|
|
.. index:: switches; --ext
|
|
|
|
Typically, the features are written in text files with a :file:`.feature`
|
|
extension. Several editors will automatically provide syntax highlighting with
|
|
such an extension. However, you can choose your own extension by using the
|
|
:option:`--ext .EXT` switch when you run the driver.
|
|
|
|
.. index:: switches; --features
|
|
|
|
By default, the driver will find all the files with a matching extension in
|
|
a subdirectory names :file:`features`, within the directory where the driver
|
|
is run. You can however specify one or more other directories to check,
|
|
by using the :option:`--features DIR` switch one or more times.
|
|
|
|
The driver will always search recursively in those directories. This lets you
|
|
organize the files into subdirectories when you have several of them, and
|
|
thus perhaps make it easier to refer to them.
|
|
|
|
The driver will run the features file in alphabetical order of the full path of
|
|
the feature file, by default. This provides a more consistent output when
|
|
the driver is run multiple times.
|
|
|
|
.. index:: switches; --random
|
|
|
|
However, by using the :option:`--random` switch, the order of features is
|
|
randomized. This is a convenient way to check that the various tests are indeed
|
|
independent of each other, as they should be to make it possible to run tests
|
|
individually.
|
|
|
|
Running feature
|
|
===============
|
|
|
|
.. index:: switches; --tags
|
|
|
|
When you run the driver, you can use the switch :option:`--tags` to control
|
|
which scenarios should be run. For instance, all scenarios related to '@gui',
|
|
or all scenarios not related to '@startup'. You can of course select subsets of
|
|
scenarios based on the file names, but tags provide a file-system-agnostic
|
|
selection mechanism.
|
|
|
|
Step timeout
|
|
============
|
|
|
|
.. note::
|
|
This will likely require each scenario to be run in its own task. There
|
|
will be only one such task, so it doesn't really add constraints on the
|
|
user code or step definitions, but it makes debugging slightly more
|
|
difficult.
|
|
|
|
.. note::
|
|
Perhaps we can also change the way the driver is generated, rather than do
|
|
everything dynamically in the driver itself.
|
|
|
|
.. index:: switches; --timeout
|
|
|
|
You can use the :option:`--timeout` switch to specify a maximum time that
|
|
steps can take to execute. A test that times out will automatically fail
|
|
with an appropriate error message.
|
|
|
|
|
|
Running tests in parallel
|
|
=========================
|
|
|
|
There are multiple modes to run features in parallel. The parallelism is always
|
|
between :file:`.feature` file, never between the Scenario of a given file.
|
|
|
|
When your application is task safe, you can run multiple features in parallel
|
|
by running each in its own task (up to a maximum number of tasks of course).
|
|
|
|
In other cases, the driver can automatically spawn several instances of itself,
|
|
each running a single feature in parallel of the others. This is slightly less
|
|
efficient, but does not impose task-safety constraints on your application.
|
|
|