tbot.machine.linux

This module contains utilities for interacting with linux hosts. This includes:

Linux Shells

The base-class tbot.machine.linux.LinuxShell defines the common interface all linux-shells should provide. This interface consists of the following methods:

And the following properties:

  • lnx.username - Current user.

  • lnx.fsroot - Path to the file-system root (just for convenience).

  • lnx.workdir - Path to a directory which testcases can store files in.

tbot contains implementations of this interface for the following shells:

class tbot.machine.linux.Bash[source]

Bases: tbot.machine.linux.linux_shell.LinuxShell

Bourne-again shell.

class tbot.machine.linux.LinuxShell[source]

Bases: tbot.machine.shell.Shell

Base-class for linux shells.

This class defines the common interface for linux shells.

abstract escape(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → str[source]

Escape a string according to this shell’s escaping rules.

If multiple arguments are given, .escape() returns a string containing each argument as a separate shell token. This means:

bash.escape("foo", "bar")
# foo bar

bash.escape("foo bar", "baz")
# "foo bar" baz
Parameters

*args – Arguments to be escaped. See Argument Types for details.

Returns

A string with quoted/escaped versions of the input arguments.

abstract exec(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → Tuple[int, str][source]

Run a command on this machine/shell.

Example:

retcode, output = mach.exec("uname", "-a")
assert retcode == 0
Parameters

*args – The command as separate arguments per command-line token. See Argument Types for more info.

Return type

tuple(int, str)

Returns

A tuple with the return code of the command and its console output. Note that the output is stdout and stderr merged. It will also contain a trailing newline in most cases.

abstract exec0(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → str[source]

Run a command and assert its return code to be 0.

Example:

output = mach.exec0("uname", "-a")

# This will raise an exception!
mach.exec0("false")
Parameters

*args – The command as separate arguments per command-line token. See Argument Types for more info.

Return type

str

Returns

The command’s console output. Note that the output is stdout and stderr merged. It will also contain a trailing newline in most cases.

abstract test(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → bool[source]

Run a command and return a boolean value whether it succeeded.

Example:

if lnx.test("which", "dropbear"):
    tbot.log.message("Dropbear is installed!")
Parameters

*args – The command as separate arguments per command-line token. See Argument Types for more info.

Return type

bool

Returns

Boolean representation of commands success. True if return code was 0, False otherwise.

abstract env(var: str, value: Union[str, tbot.machine.linux.path.Path[Self], None] = None) → str[source]

Get or set an environment variable.

Example:

# Get the value of a var
value = lnx.env("PATH")

# Set the value of a var
lnx.env("DEBIAN_FRONTEND", "noninteractive")
Parameters
Return type

str

Returns

Current (new) value of the environment variable.

abstract open_channel(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → tbot.machine.channel.channel.Channel[source]

Transform this machine into a channel for something else by running a command.

This is meant to be used for tools like picocom which connect the terminal to a serial console.

Example:

ch = lnx.open_channel("picocom", "-b", "115200", "/dev/ttyUSB0")

# You can now interact with the channel for the serial console directly
Return type

tbot.machine.channel.Channel

abstract interactive() → None[source]

Start an interactive session on this machine.

This method will connect tbot’s stdio to the machine’s channel so you can interactively run commands. This method is used by the interactive_lab and interactive_linux testcases.

abstract subshell(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → ContextManager[Self][source]

Start a subshell environment.

Sometimes you need to isolate certain tests into their own shell environment. This method returns a context manager which does this:

lnx.env("FOO", "bar")

with lnx.subshell():
    lnx.env("FOO", "baz")

assert lnx.env("FOO") == "bar"

You can also spawn a subshell with a custom command. This can be used, for example, to elevate privileges or switch user:

# Not root right now
assert int(lnx.env("EUID")) != 0

with lnx.subshell("sudo", "-ni", "bash", "--norc", "--noprofile"):
    # Root now!
    assert int(lnx.env("EUID")) == 0

Warning

tbot expects the shell inside the subshell environment to be the same shell as outside. This means, spawning a sudo environment which uses zsh instead of bash might lead to failures.

For bash, please spawn a bash --norc --noprofile for best compatibility.

For ash, an ash is good enough.

property username

Current username.

property fsroot

Path to the filesystem root of this machine, for convenience.

p = lnx.fsroot / "usr" / "lib"
assert p.is_dir()
property workdir

Path to a directory which testcases can use to store files in.

If configured properly, tbot will make sure this directory exists. Testcases should be able to deal with corrupt or missing files in this directory. Implementations should use tbot.machine.linux.Workdir.

Example:

# This is the defaut implementation
def workdir(self):
    return linux.Workdir(self, "/tmp/tbot-wd")

Argument Types

For a lot of methods defined in LinuxShell, a special set of types can be given as arguments. This protects against a lot of common errors. The allowed types are:

  • str - Every string argument will be properly quoted so the shell picks it up as only one parameter. Example:

    lnx.exec0("echo", "Hello World!", "Is this tbot?")
    
    # Will run:
    #    echo "Hello World!" "Is this tbot?"
    
  • tbot.machine.linux.Path - tbot’s path class keeps track of the machine the path is associated with. This prevents you from accidentally using a path from one host on another. The path class behaves like pathlib paths, so you can do things like:

    tftpdir = lnx.fsroot / "var" / "lib" / "tftpboot"
    lnx.exec0("ls", "-1", tftpdir)
    
    if (tftpdir / "u-boot.bin").is_file():
        tbot.log.message("Binary exists!")
    
  • “Specials” - Sometimes you will need special shell-syntax for certain operations, for example to redirect output of a command. For these things, tbot provides special “tokens”. The full list can be found in Specials. As an example, redirecting to a file works like this:

    lnx.exec0("dmesg", linux.RedirStdout(lnx.workdir / "kernel.log"))
    # Will run:
    #    dmesg >/tmp/tbot-wd/kernel.log
    

Specials

Specials can be used as part of commands to use certain shell-syntaxes. This can be used to chain multiple commands or to redirect output.

tbot.machine.linux.AndThen

Chain commands using &&.

Example:

lnx.exec0("sleep", str(10), linux.AndThen, "echo", "Hello!")
tbot.machine.linux.OrElse

Chain commands using ||.

tbot.machine.linux.Then

Chain commands using ;.

tbot.machine.linux.Pipe

Pipe the output of one command into another:

lnx.exec0("dmesg", linux.Pipe, "grep", "usb")
class tbot.machine.linux.RedirStdout(file)[source]

Redirect stdout (>...) to a file.

class tbot.machine.linux.RedirStderr(file)[source]

Redirect stderr (2>...) to a file.

class tbot.machine.linux.RedirBoth(file)[source]

Redirect both stdout and stderr (2>&1 >...) to a file.

class tbot.machine.linux.Raw(str)[source]

Emits a raw string, bypassing shell quoting/escaping.

Warning

Only use this if nothing else works! linux.Raw can quickly lead to hard-to-find bugs.

Paths

class tbot.machine.linux.Path(host: H, *args: Any)[source]

Bases: pathlib.PurePosixPath, typing.Generic

A path that is associated with a tbot machine.

A path can only be used with its associated host. Using it with any other host will raise an exception and will be detected by a static typechecker.

Apart from that, Path behaves like a pathlib.Path:

from tbot.machine import linux

p = linux.Path(mach, "/foo/bar")
p2 = p / "bar" / "baz"
if not p2.exists():
    mach.exec0("mkdir", "-p", p2.parent)
    mach.exec0("touch", p2)
elif not p2.is_file():
    raise Exception(f"{p2} must be a normal file!")

Create a new path.

Parameters
property host

Host associated with this path.

stat() → os.stat_result[source]

Return the result of stat on this path.

Tries to imitate the results of pathlib.Path.stat(), returns a os.stat_result.

exists() → bool[source]

Whether this path exists.

is_dir() → bool[source]

Whether this path points to a directory.

is_file() → bool[source]

Whether this path points to a normal file.

Whether this path points to a symlink.

is_block_device() → bool[source]

Whether this path points to a block device.

is_char_device() → bool[source]

Whether this path points to a character device.

is_fifo() → bool[source]

Whether this path points to a pipe(fifo).

is_socket() → bool[source]

Whether this path points to a unix domain-socket.

property parent

Parent of this path.

glob(pattern: str) → Iterator[tbot.machine.linux.path.Path[H]][source]

Iterate over this subtree and yield all existing files (of any kind, including directories) matching the given relative pattern.

Example:

ubootdir = lh.workdir / "u-boot"

# .glob() returns a list which can be iterated.
for f in ubootdir.glob("common/*.c"):
    tbot.log.message(f"Found {f}.")

# To use the globs in another commandline (note the `*`!):
lh.exec0("ls", "-l", *ubootdir.glob("common/*.c"))

Warning

The glob pattern must not contain spaces or other special characters!

Workdir

class tbot.machine.linux.Workdir[source]
classmethod static(host: H, pathstr: str) → tbot.machine.linux.workdir.Workdir[H][source]

Create a workdir in a static location, described by pathstr.

Example:

with tbot.acquire_lab() as lh:
    workdir = linux.Workdir.static(lh, "/tmp/tbot-my-workdir")
Parameters
  • host (LinuxShell) – Machine where the workdir should be created

  • pathstr (str) – Path for the workdir

Return type

tbot.machine.linux.Path

Returns

A tbot path to the workdir which is now guaranteed to exist

classmethod athome(host: H, subdir: str) → tbot.machine.linux.workdir.Workdir[H][source]

Create a workdir below the current users home directory.

Example:

with tbot.acquire_lab() as lh:
    # Use ~/.local/share/tbot-foo-dir
    workdir = linux.Workdir.athome(lh, ".local/share/tbot-foo-dir")

tbot will query the $HOME environment variable for the location of the current users home directory.

Parameters
  • host (LinuxShell) – Machine where the workdir should be created

  • subdir (str) – Subdirectory of the user’s home where the workdir should be created

Return type

tbot.machine.linux.Path

Returns

A tbot path to the workdir which is now guaranteed to exist

Lab-Host

class tbot.machine.linux.Lab[source]

Bases: tbot.machine.linux.linux_shell.LinuxShell

Mixin for marking a machine as a lab-host.

build() → tbot.machine.linux.build.Builder[source]

Return the default build-host for this lab.

If your lab does not contain a build-capable machine, just leave this method as is. tbot will raise an exception if a testcase attempts accessing the build-host anyway.

Builder

The Builder mixin allows marking a machine as a build-host. This means generic testcases like uboot.build can use it to automatically build projects. For this to work, a build-host needs to specify which toolchains it has installed and where tbot can find them.

class tbot.machine.linux.Builder[source]

Bases: tbot.machine.linux.linux_shell.LinuxShell

Mixin to mark a machine as a build-host.

You need to define the toolchain() method when using this mixin. You can then use the enable() method to enable a toolchain and compile projects with it:

with MyBuildHost(lh) as bh:
    bh.exec0("uptime")

    with bh.enable("generic-armv7a-hf"):
        cc = bh.env("CC")
        bh.exec0(linux.Raw(cc), "main.c")

Note

If you look closely, I have used linux.Raw(cc) in the exec0() call. This is necessary because a lot of toolchains define $CC as something like

CC=arm-poky-linux-gnueabi-gcc -march=armv7-a -mfpu=neon -mfloat-abi=hard -mcpu=cortex-a8

where some parameters are already included. Without the linux.Raw, tbot would run

$ "${CC}" main.c

where the arguments are interpreted as part of the path to the compiler. This will obviously fail so instead, with the linux.Raw, tbot will run

$ ${CC} main.c

where the shell expansion will do the right thing.

abstract property toolchains

Return a dictionary of all toolchains that exist on this buildhost.

Example:

@property
def toolchains(self) -> typing.Dict[str, linux.build.Toolchain]:
    return {
        "generic-armv7a": linux.build.EnvScriptToolchain(
            linux.Path(
                self,
                "/path/to/environment-setup-armv7a-neon-poky-linux-gnueabi",
            )
        ),
        "generic-armv7a-hf": linux.build.EnvScriptToolchain(
            linux.Path(
                self,
                "/path/to/environment-setup-armv7ahf-neon-poky-linux-gnueabi",
            )
        ),
    }
enable(arch: str) → Iterator[None][source]

Enable the toolchain for arch on this BuildHost instance.

Example:

with lh.build() as bh:
    # Now we are on the buildhost

    with bh.enable("generic-armv7a-hf"):
        # Toolchain is enabled here
        cc = bh.env("CC")
        bh.exec0(linux.Raw(cc), "--version")

Toolchains

class tbot.machine.linux.build.Toolchain[source]

Bases: abc.ABC

Generic toolchain type.

abstract enable(host: tbot.machine.linux.build.Builder) → None[source]

Enable this toolchain on the given host.

class tbot.machine.linux.build.EnvScriptToolchain(path: tbot.machine.linux.path.Path[H])[source]

Bases: tbot.machine.linux.build.Toolchain

Toolchain that is initialized using an env script.

Create a new EnvScriptToolchain.

Parameters

path (linux.Path) – Path to the env script

enable(host: H) → None[source]

Enable this toolchain on the given host.

Authenticators

For logging in via SSH using ParamikoConnector or SSHConnector, tbot provides the following ‘authenticators’:

class tbot.machine.linux.auth.NoneAuthenticator[source]

Bases: tbot.machine.linux.auth.AuthenticatorBase

Most primitive authenticator.

Tries not passing any specific credentials and hopes ssh-config already contains all necessary infos. This is the default.

class tbot.machine.linux.auth.PrivateKeyAuthenticator(key_file: Union[str, pathlib.PurePath])[source]

Bases: tbot.machine.linux.auth.AuthenticatorBase

Authenticate using a private-key file.

Example:

class MySSHHost(connector.SSHConnector, linux.Bash):
    username = "foouser"
    authenticator = linux.auth.PrivateKeyAuthenticator("/home/foo/.ssh/id_rsa_foo")
class tbot.machine.linux.auth.PasswordAuthenticator(password: str)[source]

Bases: tbot.machine.linux.auth.AuthenticatorBase

Authenticate using a password.

Danger

This method is very insecure and might lead to PASSWORDS BEING STOLEN.

Example:

class MySSHHost(connector.SSHConnector, linux.Bash):
    username = "root"
    authenticator = linux.auth.PasswordAuthenticator("hunter2")