tbot can keep record of test-runs in a log-file. For this, entries detailing a number of events will be stored:
Testcase begin and end, their duration and success state.
All commands which are run, their output, return-code, and on which machine they were run.
Custom messages and events emitted by testcases (for example by using
To enable logging, set either the
--log=<file.json> or the
--log-auto parameter. The latter
will automatically create a log-file in
NNNN incrementing so
old files are not overwritten.
The log format is a stream of json objects. To read it, you can use the
A json stream is not just a json file. This means trying to parse the log-file as plain json will fail! Use the provided log-parser instead.
One use of the log is generating visualizations and statistics of a test run. There are a few so called generators already included with tbot:
generators/dot.py: Generate a graphviz dot graph of the testcase hirarchy that was run.
generators/htmllog.py: Generate a html representation of the log that gives an overview of the occurences that happened during the run.
generators/junit.py: Convert the log to a JUnit xml file which can be displayed by CI tools like Jenkins.
generators/messages.py: A simple demo generator that just prints all message events.
Log events are json objects. Each one has at least the following fields:
"type": An array that identifies the event type. Events for similar occurences have a common prefix in the
"time": Time in seconds since the test run began.
"data": An object with event specific elements.
The following events are emitted by tbot (possibly incomplete):
["tc", "begin"]- Testcase being called,
["tc", "end"]- Testcase ending,
["cmd", "<machine-name>"]- A command was issued on the specified machine.
datais filled with the
"cmd"that was sent and its
["msg", "<LEVEL>"]- A log message at the specified level,
["tbot", "end"]- Emitted once after all testcases completed (or failed).
dataknows about the total
More might be added in the future.