tbot

Testcase Decorators

In tbot, testcases are marked using one of the following decorators. This will make tbot aware of the testcase and allows you to call it from the commandline. The testcase decorators will also track time and success of each run.

tbot.testcase(arg: str) → ContextManager[None][source]
tbot.testcase(arg: F_tc) → F_tc

Mark a testcase.

This function can be used in two ways; either as a decorator for a function or as a context-manager. The first form is probably what is most commonly used because testcases defined using this decorator will be callable from the commandline.

Decorator:

@tbot.testcase
def foobar_testcase(x: str) -> int:
    return int(x, 16)

Context-Manager:

with tbot.testcase("test_foo_bar"):
    ...
tbot.named_testcase(name: str) → Callable[F_tc, F_tc][source]

Decorate a function to make it a testcase, but with a different name.

The testcase’s name is relevant for log-events and when calling it from the commandline.

Example:

@tbot.named_testcase("my_different_testcase")
def foobar_testcase(x: str) -> int:
    return int(x, 16)

(On the commandline you’ll have to run tbot my_different_testcase now.)

Context

The new mechanism for machine management is the context (superseding tbot.selectable). The global context is stored in tbot.ctx which is an instance of tbot.Context. Read the Context guide for a detailed introduction.

tbot.ctx: tbot.Context

The global context. This context should be used in testcases for accessing machines via the following pattern:

Single Machine:

@tbot.testcase
def test_with_labhost():
    with tbot.ctx.request(tbot.role.LabHost) as lh:
        lh.exec0("uname", "-a")

Multiple Machines:

@tbot.testcase
def test_with_board_and_lab():
   with tbot.ctx() as cx:
      lh = cx.request(tbot.role.LabHost)
      lnx = cx.request(tbot.role.BoardLinux)

      lh.exec0("hostname")
      lnx.exec0("hostname")

See the tbot.Context class below for the API details.

class tbot.Context(*, add_defaults: bool = False)[source]

Bases: typing.ContextManager

A context which machines can be registered in and where instances can be retrieved from.

You will usually access the global context tbot.ctx which is an instance of this class instead of instanciating tbot.Context yourself. See the Context guide for a detailed introduction.

register(machine: Type[M], roles: Union[Any, List[Any]], *, weak: bool = False)None[source]

Register a machine in this context for certain roles.

Registers the machine-class machine for the role roles or, if roles is a list, for all roles it contains. If for any role a machine is already registered, an exception is thrown.

This function is usually called in the register_machines() function of a lab- or board-config:

class SomeLabHostClass(...):
    ...

def register_machines(ctx: tbot.Context) -> None:
    ctx.register(SomeLabHostClass, tbot.role.LabHost)

    # or, to register for multiple roles
    ctx.register(SomeLabHostClass, [tbot.role.LabHost, tbot.role.BuildHost])
Parameters
  • machine – A concrete machine-class to be registered. This machine-class will later be instanciated on request via its Connector.from_context() classmethod.

  • roles – Either a single role or a list of roles for which machine should be registered.

  • weak (bool) – Changes the way registration works: The machine is only registered for those roles which do not already have a machine registered. It will be registered as a weak default which means a later register will overwrite it without erroring. This is usually not necessary and should be used with care.

request(type: Callable[, M], *, reset: bool = False, exclusive: bool = False) → Iterator[M][source]

Request a machine instance from this context.

Requests an instance of the role type from the context. If no instance exists, one will be created. If a previous testcase has already requested such an instance, the same instance is returned (this behavior can be controlled with the reset and exclusive keyword arguments).

This function must be used as a context manager:

@tbot.testcase
def foo_bar():
    with tbot.ctx.request(tbot.role.LabHost) as lh:
        lh.exec0("uname", "-a")

Alternatively, if you need multiple machines, a pattern similar to contextlib.ExitStack can be used:

@tbot.testcase
def foo_bar():
    with tbot.ctx() as cx:
        lh = cx.request(tbot.role.LabHost)
        bh = cx.request(tbot.role.BuildHost)

        lh.exec0("cat", "/etc/os-release")
        bh.exec0("free", "-h")

The semantics of a request() can be controlled further by the reset and exclusive keyword arguments. See their documentation for the details.

Parameters
  • type (tbot.role.Role) – The role for which a machine instance is requested.

  • reset (bool) –

    Controls what happens if an instance already exists:

    • False (default): If an instance already exists due to a previous request(), it will be returned (both requests share it).

    • True: If an instance already exists due to a previous request(), it will be torn down and re-initialized (the previous request thus looses access to it).

    reset=True can, for example, be used to write a testcase where the DUT is powercycled:

    @tbot.testcase
    def test_with_reboot():
        with tbot.ctx.request(tbot.role.BoardUBoot) as ub:
            ub.exec0("version")
    
        # Device will be powercycled here, even though if some "outer"
        # context for U-Boot is still active.  Note that such an outer
        # context will loose access to the instance after this point.
    
        with tbot.ctx.request(tbot.role.BoardUBoot, reset=True) as ub:
            ub.exec0("version")
    

  • exclusive (bool) –

    Controls whether other requests can get access to the same instance while this request is active:

    • False (default): A request() after this one will get shared access to the same instance.

    • True: Any future request() while this one is active is forbidden and will fail. Once this request() ends, the instance will be torn down so future requests will need to re-initialize it.

    This mode should be used when you are going to do changes to the instance which could potentially bring it into a state that other testcases won’t expect.

get_machine_class(type: Callable[, M]) → Type[M][source]

Return the registered machine class for a Role.

Convenience Decorators

To make writing testcase interacting with machines easier, tbot provides three more decorators to allow easily writing extensible testcases. They look like this:

import tbot
from tbot.machine import board

@tbot.testcase
@tbot.with_uboot
def uboot_testcase(ub: board.UBootShell, foo: bool = False):
   ub.exec0("version")

uboot_testcase() can now be called in three different ways:

  1. Without any arguments (eg. from the commandline): The decorator will take care of first connecting to the lab-host and then powering up the board and initializing the U-Boot machine.

  2. Passing a lab-host as an argument: The decorator will take care of powering up the board and initializing the U-Boot machine.

  3. Passing a U-Boot machine: No additional work is needed and the test can run immediately.

Note

As seen above, you can still have additional arguments as well. Those will work as expected, but you need to pass them as kwargs now:

uboot_testcase(foo=True)
uboot_testcase(lh, foo=True)
uboot_testcase(ub, foo=True)

Or on the commandline:

$ tbot @myargs uboot_testcase -pfoo=True

Three decorators of this style are currently available:

tbot.with_lab(tc: F_lh) → Callable[[None, None, None], Any][source]

Decorate a function to automatically supply the lab-host as an argument.

The idea is that when using this decorator and calling the testcase without a lab-host, tbot will automatically acquire the default lab.

Example:

from tbot.machine import linux

@tbot.testcase
@tbot.with_lab
def testcase_with_lab(lh: linux.Lab) -> None:
    lh.exec0("uname", "-a")

This is essentially syntactic sugar for:

import typing
import tbot
from tbot.machine import linux

@tbot.testcase
def testcase_with_lab(
    lab: typing.Optional[linux.Lab] = None,
) -> None:
    with lab or tbot.acquire_lab() as lh:
        lh.exec0("uname", "-a")

Warning

While making your life a lot easier, this decorator unfortunately has a drawback: It will erase the type signature of your testcase, so you can no longer rely on type-checking when using the testcase downstream.

tbot.with_uboot(tc: F_ub) → Callable[[None, None, None], Any][source]

Decorate a function to automatically supply a U-Boot machine as an argument.

The idea is that when using this decorator and calling the testcase without an already initialized U-Boot machine, tbot will automatically acquire the selected one.

Example:

from tbot.machine import board

@tbot.testcase
@tbot.with_uboot
def testcase_with_uboot(ub: board.UBootShell) -> None:
    ub.exec0("version")

This is essentially syntactic sugar for:

import contextlib
import typing
import tbot
from tbot.machine import board, linux

@tbot.testcase
def testcase_with_uboot(
    lab_or_ub: typing.Union[linux.Lab, board.UBootShell, None] = None,
) -> None:
    with contextlib.ExitStack() as cx:
        lh: linux.Lab
        ub: board.UBootShell

        if isinstance(lab_or_ub, linux.Lab):
            lh = cx.enter_context(lab_or_ub)
        elif isinstance(lab_or_ub, board.UBootShell):
            lh = cx.enter_context(lab_or_ub.host)
        else:
            lh = cx.enter_context(tbot.acquire_lab())

        if isinstance(lab_or_ub, board.UBootShell):
            ub = cx.enter_context(lab_or_ub)
        else:
            b = cx.enter_context(tbot.acquire_board(lh))
            ub = cx.enter_context(tbot.acquire_uboot(b))

        ub.exec0("version")

Warning

While making your life a lot easier, this decorator unfortunately has a drawback: It will erase the type signature of your testcase, so you can no longer rely on type-checking when using the testcase downstream.

tbot.with_linux(tc: F_lnx) → Callable[[None, None, None], Any][source]

Decorate a function to automatically supply a board Linux machine as an argument.

The idea is that when using this decorator and calling the testcase without an already initialized Linux machine, tbot will automatically acquire the selected one.

Example:

from tbot.machine import linux

@tbot.testcase
@tbot.with_linux
def testcase_with_linux(lnx: linux.LinuxShell) -> None:
    lnx.exec0("uname", "-a")

Warning

While making your life a lot easier, this decorator unfortunately has a drawback: It will erase the type signature of your testcase, so you can no longer rely on type-checking when using the testcase downstream.

Testcase Skipping

Sometimes a test can only run with certain prerequisites met. You can write a testcase to automatically skip when they aren’t, using tbot.skip():

tbot.skip(reason: str) → NoReturn[source]

Skip this testcase.

A skipped testcase will return None instead of whatever type it would return normally. This might not make sense for certain testcases and might violate the type-annotation. Only use it, if it really makes sense!

Example:

@tbot.testcase
@tbot.with_lab
def test_something(lh) -> None:
    p = lh.fsroot / "dev" / "somedevice"

    if not p.is_char_device():
        tbot.skip("somedevice not present on this host")

    ...
class tbot.SkipException[source]

Bases: Exception

Exception to be used when a testcase is skipped.

Raising a SkipException will be caught in the tbot.testcase() decorator and the testcase will return None. This might violate the type-annotations so it should only be used if calling code can deal with a testcase returning None.

Default Machine Access

There are a few machines which can be configured in the Configuration and then accessed through the following functions. This allows you to write generic testcases, based on using one or more of them:

tbot.acquire_lab()tbot.selectable.LocalLabHost[source]

Acquire a new connection to the LabHost.

If your lab-host is using a ParamikoConnector this will create a new ssh connection.

You should call this function as little as possible, because it can be very slow. If possible, try to reuse the labhost. A recipe for doing so is

import typing
import tbot
from tbot.machine import linux

@tbot.testcase
def my_testcase(
    lab: typing.Optional[linux.LinuxShell] = None,
) -> None:
    with lab or tbot.acquire_lab() as lh:
        # Your code goes here
        ...
Return type

tbot.selectable.LabHost

tbot.acquire_local()tbot.selectable.LocalLabHost[source]

Acquire a machine for the local host.

Localhost machines are very cheap so they do not need to be shared like the others and you can create as many as you want. One usecase might be copying test-results to you local machine after the run.

Example:

import tbot

@tbot.testcase
def my_testcase() -> None:
    with tbot.acquire_local() as lo:
        lo.exec0("id", "-un")
        # On local machines you can access tbot's working directory:
        tbot.log.message(f"CWD: {lo.workdir}")
tbot.acquire_board(lh: tbot.selectable.LocalLabHost)tbot.selectable.Board[source]

Acquire the selected board.

If configured properly, tbot.acquire_board() will power on the hardware and open a serial-console for the selected board. Just by itself, this is not too useful, so you will usually follow it up immediately with a call to either tbot.acquire_uboot() or tbot.acquire_linux().

Example:

with tbot.acquire_lab() as lh:
    lh.exec0("echo", "Foo")
    with tbot.acquire_board(lh) as b, tbot.acquire_uboot(b) as ub:
        ub.exec0("version")
tbot.acquire_uboot(board: tbot.selectable.Board, *args: Any)tbot.selectable.UBootMachine[source]

Acquire the selected board’s U-Boot shell.

As there can only be one instance of the selected board’s UBootShell at a time, your testcases should optionally take the UBootShell as a parameter. The recipe looks like this:

import contextlib
import typing
import tbot
from tbot.machine import board


@tbot.testcase
def my_testcase(
    lab: typing.Optional[tbot.selectable.LabHost] = None,
    uboot: typing.Optional[board.UBootShell] = None,
) -> None:
    with contextlib.ExitStack() as cx:
        lh = cx.enter_context(lab or tbot.acquire_lab())
        if uboot is not None:
            ub = uboot
        else:
            b = cx.enter_context(tbot.acquire_board(lh))
            ub = cx.enter_context(tbot.acquire_uboot(b))

        ...
Return type

tbot.selectable.UBootMachine

tbot.acquire_linux(b: Union[tbot.selectable.Board, tbot.selectable.UBootMachine], *args: Any)tbot.selectable.LinuxMachine[source]

Acquire the board’s Linux shell.

Can either boot from a previously created U-Boot (if the implementation supports this) or directly.

To write testcases that work both from the commandline and when called from other testcases, use the following recipe:

import contextlib
import typing
import tbot
from tbot.machine import board


@tbot.testcase
def test_testcase(
    lab: typing.Optional[tbot.selectable.LabHost] = None,
    board_linux: typing.Optional[board.LinuxMachine] = None,
) -> None:
    with contextlib.ExitStack() as cx:
        lh = cx.enter_context(lab or tbot.acquire_lab())
        if board_linux is not None:
            lnx = board_linux
        else:
            b = cx.enter_context(tbot.acquire_board(lh))
            lnx = cx.enter_context(tbot.acquire_linux(b))

        ...
Return type

tbot.machine.linux.LinuxShell