tbot.machine.linux
¶
This module contains utilities for interacting with linux hosts. This includes:
tbot’s own path type:
tbot.machine.linux.Path
Linux Shells¶
The base-class tbot.machine.linux.LinuxShell
defines the common
interface all linux-shells should provide. This interface consists of the
following methods:
lnx.escape()
- Escape args for the underlying shell.lnx.exec0()
- Run command and ensure it succeeded.lnx.exec()
- Run command and return output and return code.lnx.run()
- Start a command and allow a testcase to interact with its stdio.lnx.test()
- Run command and return boolean whether it succeeded.lnx.env()
- Get/Set environment variables.lnx.subshell()
- Start a subshell environment.lnx.interactive()
- Start an interactive session for this machine.
And the following properties:
lnx.username
- Current user.lnx.fsroot
- Path to the file-system root (just for convenience).lnx.workdir
- Path to a directory which testcases can store files in.
tbot contains implementations of this interface for the following shells:
-
class
tbot.machine.linux.
Bash
[source]¶ Bases:
tbot.machine.linux.linux_shell.LinuxShell
Bourne-again shell.
-
class
tbot.machine.linux.
LinuxShell
[source]¶ Bases:
tbot.machine.shell.Shell
Base-class for linux shells.
This class defines the common interface for linux shells.
-
abstract
escape
(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → str[source]¶ Escape a string according to this shell’s escaping rules.
If multiple arguments are given,
.escape()
returns a string containing each argument as a separate shell token. This means:bash.escape("foo", "bar") # foo bar bash.escape("foo bar", "baz") # "foo bar" baz
- Parameters
*args – Arguments to be escaped. See Argument Types for details.
- Returns
A string with quoted/escaped versions of the input arguments.
-
abstract
exec
(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → Tuple[int, str][source]¶ Run a command on this machine/shell.
Example:
retcode, output = mach.exec("uname", "-a") assert retcode == 0
- Parameters
*args – The command as separate arguments per command-line token. See Argument Types for more info.
- Return type
- Returns
A tuple with the return code of the command and its console output. Note that the output is
stdout
andstderr
merged. It will also contain a trailing newline in most cases.
-
abstract
exec0
(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → str[source]¶ Run a command and assert its return code to be 0.
Example:
output = mach.exec0("uname", "-a") # This will raise an exception! mach.exec0("false")
- Parameters
*args – The command as separate arguments per command-line token. See Argument Types for more info.
- Return type
- Returns
The command’s console output. Note that the output is
stdout
andstderr
merged. It will also contain a trailing newline in most cases.
-
abstract
test
(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → bool[source]¶ Run a command and return a boolean value whether it succeeded.
Example:
if lnx.test("which", "dropbear"): tbot.log.message("Dropbear is installed!")
- Parameters
*args – The command as separate arguments per command-line token. See Argument Types for more info.
- Return type
- Returns
Boolean representation of commands success.
True
if return code was0
,False
otherwise.
-
abstract
env
(var: str, value: Optional[Union[str, tbot.machine.linux.path.Path[Self]]] = None) → str[source]¶ Get or set an environment variable.
Example:
# Get the value of a var value = lnx.env("PATH") # Set the value of a var lnx.env("DEBIAN_FRONTEND", "noninteractive")
- Parameters
var (str) – Environment variable name.
value (tbot.machine.linux.Path,str) – Optional value to set the variable to.
- Return type
- Returns
Current (new) value of the environment variable.
-
run
(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → ContextManager[tbot.machine.linux.util.RunCommandProxy][source]¶ Start an interactive command.
Interactive commands are started in a context-manager. Inside, direct interaction with the commands stdio is possible using a
RunCommandProxy
. You must call one of theterminate*()
methods before leaving the context! The proxy object provides an interface similar to pexpect for inteaction (see the methods of theChannel
class).Example:
with lh.run("gdb", "-n", exe) as gdb: # Interact with gdb in this context # Wait for gdb to start up gdb.read_until_prompt("(gdb) ") # Better for automated interaction gdb.sendline("set confirm off") gdb.read_until_prompt("(gdb) ") # Necessary so output is not clobbered with escape sequences gdb.sendline("set style enabled off") gdb.read_until_prompt("(gdb) ") gdb.sendline("break main") gdb.read_until_prompt("(gdb) ") gdb.sendline("run") # We have hit the breakpoint gdb.read_until_prompt("(gdb) ") gdb.sendline("info locals", read_back=True) local_info = gdb.read_until_prompt("(gdb) ").strip() for line in local_info.split("\n"): var, val = line.split(" = ", 1) tbot.log.message(f"Local variable `{var}` has value `{val}`!") gdb.sendline("quit") gdb.terminate0()
-
abstract
open_channel
(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → tbot.machine.channel.channel.Channel[source]¶ Transform this machine into a channel for something else by running a command.
This is meant to be used for tools like
picocom
which connect the terminal to a serial console.Example:
ch = lnx.open_channel("picocom", "-b", "115200", "/dev/ttyUSB0") # You can now interact with the channel for the serial console directly
- Return type
-
abstract
interactive
() → None[source]¶ Start an interactive session on this machine.
This method will connect tbot’s stdio to the machine’s channel so you can interactively run commands. This method is used by the
interactive_lab
andinteractive_linux
testcases.
-
abstract
subshell
(*args: Union[str, tbot.machine.linux.special.Special[Self], tbot.machine.linux.path.Path[Self]]) → ContextManager[Self][source]¶ Start a subshell environment.
Sometimes you need to isolate certain tests into their own shell environment. This method returns a context manager which does this:
lnx.env("FOO", "bar") with lnx.subshell(): lnx.env("FOO", "baz") assert lnx.env("FOO") == "bar"
You can also spawn a subshell with a custom command. This can be used, for example, to elevate privileges or switch user:
# Not root right now assert int(lnx.env("EUID")) != 0 with lnx.subshell("sudo", "-ni", "bash", "--norc", "--noprofile"): # Root now! assert int(lnx.env("EUID")) == 0
Warning
tbot expects the shell inside the subshell environment to be the same shell as outside. This means, spawning a sudo environment which uses
zsh
instead ofbash
might lead to failures.For bash, please spawn a
bash --norc --noprofile
for best compatibility.For ash, an
ash
is good enough.
-
property
username
¶ Current username.
-
property
fsroot
¶ Path to the filesystem root of this machine, for convenience.
p = lnx.fsroot / "usr" / "lib" assert p.is_dir()
-
property
workdir
¶ Path to a directory which testcases can use to store files in.
If configured properly, tbot will make sure this directory exists. Testcases should be able to deal with corrupt or missing files in this directory. Implementations should use
tbot.machine.linux.Workdir
.Example:
# This is the defaut implementation @property def workdir(self): return linux.Workdir.xdg_data(self, "")
-
abstract
Argument Types¶
For a lot of methods defined in LinuxShell
, a
special set of types can be given as arguments. This protects against a lot of
common errors. The allowed types are:
str
- Every string argument will be properly quoted so the shell picks it up as only one parameter. Example:lnx.exec0("echo", "Hello World!", "Is this tbot?") # Will run: # echo "Hello World!" "Is this tbot?"
tbot.machine.linux.Path
- tbot’s path class keeps track of the machine the path is associated with. This prevents you from accidentally using a path from one host on another. The path class behaves likepathlib
paths, so you can do things like:tftpdir = lnx.fsroot / "var" / "lib" / "tftpboot" lnx.exec0("ls", "-1", tftpdir) if (tftpdir / "u-boot.bin").is_file(): tbot.log.message("Binary exists!")
“Specials” - Sometimes you will need special shell-syntax for certain operations, for example to redirect output of a command. For these things, tbot provides special “tokens”. The full list can be found in Specials. As an example, redirecting to a file works like this:
lnx.exec0("dmesg", linux.RedirStdout(lnx.workdir / "kernel.log")) # Will run: # dmesg >/tmp/tbot-wd/kernel.log
Specials¶
Specials can be used as part of commands to use certain shell-syntaxes. This can be used to chain multiple commands or to redirect output.
-
tbot.machine.linux.
AndThen
¶ Chain commands using
&&
.Example:
lnx.exec0("sleep", str(10), linux.AndThen, "echo", "Hello!")
-
tbot.machine.linux.
OrElse
¶ Chain commands using
||
.
-
tbot.machine.linux.
Then
¶ Chain commands using
;
.
-
tbot.machine.linux.
Pipe
¶ Pipe the output of one command into another:
lnx.exec0("dmesg", linux.Pipe, "grep", "usb")
-
class
tbot.machine.linux.
RedirBoth
(file)[source]¶ Redirect both
stdout
andstderr
(2>&1 >...
) to a file.
-
tbot.machine.linux.
Background
¶ Tells the shell to run this command in the background (
&
).By default all output from the command is suppressed (by redirection to
/dev/null
). If you do want to save command output, use the alternate form ofBackground
:f1 = lh.workdir / "stdout.txt" f2 = lh.workdir / "stderr.txt" # Just redirect stdout to a file, discard stderr linux.Background(stdout=f1) # Just redirect stderr to a file, discard stdout linux.Background(stderr=f2) # Redirect both to files linux.Background(stdout=f1, stderr=f2)
Warning
Beware of the side-effects of running commands in the background! Never ever add
linux.Background
after a command that you expect will terminate at some point on its own! This will potentially clobber an unrelated command’s output which can have unexpected effects. Only use this special token with commands that will run forever until terminated manually. A good pattern to follow is this:lh.exec0("some", "cmd", "that", "won't", "terminate", linux.Background) pid = lh.env("!") ... lh.exec0("kill", pid, linux.Then, "wait", pid)
RunCommandProxy¶
-
class
tbot.machine.linux.
RunCommandProxy
(chan: tbot.machine.channel.channel.Channel, cmd_context: Callable[tbot.machine.linux.util.RunCommandProxy, Generator[str, None, Tuple[int, str]]])[source]¶ Bases:
tbot.machine.channel.channel.Channel
Proxy for interacting with a running command.
A
RunCommandProxy
is created with a context-manager andLinuxShell.run()
.Example:
with lh.run("gdb", lh.workdir / "a.out") as gdb: gdb.sendline("target remote 127.0.0.1:3333") gdb.sendline("load") gdb.sendline("mon reset halt") gdb.sendline("quit") gbd.terminate0()
A
RunCommandProxy
has all methods of aChannel
for interacting with the remote. Additionally, a few more methods exist which are necessary to end a command’s invokation properly. You must always call one of them before leaving the context-manager! These methods are:
-
class
tbot.machine.linux.
CommandEndedException
(match: Union[bytes, Match[bytes]] = b'')[source]¶ Bases:
tbot.machine.channel.channel.DeathStringException
,tbot.machine.channel.channel.ChannelTakenException
The command which was run (interactively) ended prematurely.
This exception might be raised when reading from (or writing to) a
RunCommandProxy
and the remote command exited during the call. You can catch the exception but after receiving it, no more interaction with the command is allowed except the finalterminate0()
orterminate()
.Example:
with lh.run("foo", "command") as foo: try: while True: foo.read_until_prompt("$ ") foo.sendline("echo some command") except linux.CommandEndedException: pass foo.terminate0()
Paths¶
-
class
tbot.machine.linux.
Path
(host: H, *args: Any)[source]¶ Bases:
pathlib.PurePosixPath
,Generic
[typing.H
]A path that is associated with a tbot machine.
A path can only be used with its associated host. Using it with any other host will raise an exception and will be detected by a static typechecker.
Apart from that,
Path
behaves like apathlib.Path
:from tbot.machine import linux p = linux.Path(mach, "/foo/bar") p2 = p / "bar" / "baz" if not p2.exists(): mach.exec0("mkdir", "-p", p2.parent) mach.exec0("touch", p2) elif not p2.is_file(): raise Exception(f"{p2} must be a normal file!")
Create a new path.
- Parameters
host (linux.LinuxShell) – Host this path should be associated with
args –
pathlib.PurePosixPath
constructor arguments
-
property
host
¶ Host associated with this path.
-
stat
() → os.stat_result[source]¶ Return the result of
stat
on this path.Tries to imitate the results of
pathlib.Path.stat()
, returns aos.stat_result
.
-
property
parent
¶ Parent of this path.
-
glob
(pattern: str) → Iterator[tbot.machine.linux.path.Path[H]][source]¶ Iterate over this subtree and yield all existing files (of any kind, including directories) matching the given relative pattern.
Example:
ubootdir = lh.workdir / "u-boot" # .glob() returns a list which can be iterated. for f in ubootdir.glob("common/*.c"): tbot.log.message(f"Found {f}.") # To use the globs in another commandline (note the `*`!): lh.exec0("ls", "-l", *ubootdir.glob("common/*.c"))
Warning
The glob pattern must not contain spaces or other special characters!
-
write_text
(data: str, encoding: Optional[str] = None, errors: Optional[str] = None) → int[source]¶ Write
data
into the file this path points to.Example:
f = lnx.workdir / "foo.sh" f.write_text('''\ #!/bin/sh set -e echo "Hello tbot!" ps ax ''') f.exec0("chmod", "+x", f)
Warning
This string must contain ‘text’ in the sense that some control characters are not allowed. Consult the documentation of
LinuxShell.run()
for details.Additionally, line-endings might be transformed according to the tty’s settings. This function is not meant for byte-by-byte transfers, but for configuration files or small scripts. If you want to transfer a blob or a larger file, consider using
tbot.tc.shell.copy()
or (for small files)Path.write_bytes()
.
-
read_text
(encoding: Optional[str] = None, errors: Optional[str] = None) → str[source]¶ Read the contents of a text file, pointed to by this path.
Warning
This method is for ‘text’ content only as line-endings might not be transferred as contained in the file (Will always use a single
\n
). If you want to transfer a file byte-by-byte, consider usingtbot.tc.shell.copy()
instead. For small files,Path.write_bytes()
might also be an option.
-
write_bytes
(data: bytes) → int[source]¶ Write binary
data
into the file this path points to.Note
This method ensures exact byte-by-byte transfer. To do so, it encodes the data using base64 which makes console output less readable. If you intend to transfer text data, please use
Path.write_text()
.
-
read_bytes
() → bytes[source]¶ Read the contents of a file, pointed to by this path.
Note
This method ensures exact byte-by-byte transfer. To do so, it encodes the data using base64 which makes console output less readable. If you intend to transfer text data, please use
Path.read_text()
.
Workdir¶
-
class
tbot.machine.linux.
Workdir
[source]¶ -
classmethod
static
(host: H, pathstr: str) → tbot.machine.linux.workdir.Workdir[H][source]¶ Create a workdir in a static location, described by
pathstr
.Example:
with tbot.acquire_lab() as lh: workdir = linux.Workdir.static(lh, "/tmp/tbot-my-workdir")
-
classmethod
athome
(host: H, subdir: str) → tbot.machine.linux.workdir.Workdir[H][source]¶ Create a workdir below the current users home directory.
Example:
with tbot.acquire_lab() as lh: # Use ~/tbot-foo-dir workdir = linux.Workdir.athome(lh, "tbot-foo-dir")
tbot will query the
$HOME
environment variable for the location of the current users home directory.
-
classmethod
xdg_data
(host: H, subdir: str) → tbot.machine.linux.workdir.Workdir[H][source]¶ Create a workdir in
$XDG_DATA_HOME/tbot
(usually~/.local/share/tbot
).Example:
with tbot.acquire_lab() as lh: # Use ~/.local/share/tbot/foo-dir workdir = linux.Workdir.xdg_data(lh, "foo-dir")
-
classmethod
xdg_runtime
(host: H, subdir: str) → tbot.machine.linux.workdir.Workdir[H][source]¶ Create a workdir in
$XDG_RUNTIME_DIR
.$XDG_RUNTIME_DIR
is meant for non-essential runtime files and other file objects (such as sockets, named pipes, …) as specified by the XDG Base Directory Specification.Example:
with tbot.acquire_lab() as lh: # Results in `$XDG_RUNTIME_DIR/tbot/tbot-pipes` workdir = linux.Workdir.xdg_runtime(lh, "tbot-pipes")
-
classmethod
Lab-Host¶
Builder¶
The Builder
mixin allows marking a machine as a build-host. This
means generic testcases like uboot.build
can use it to automatically build projects. For this
to work, a build-host needs to specify which toolchains it has installed and where tbot can find
them.
-
class
tbot.machine.linux.
Builder
[source]¶ Bases:
tbot.machine.linux.linux_shell.LinuxShell
Mixin to mark a machine as a build-host.
You need to define the
toolchain()
method when using this mixin. You can then use theenable()
method to enable a toolchain and compile projects with it:with MyBuildHost(lh) as bh: bh.exec0("uptime") with bh.enable("generic-armv7a-hf"): cc = bh.env("CC") bh.exec0(linux.Raw(cc), "main.c")
Note
If you look closely, I have used
linux.Raw(cc)
in theexec0()
call. This is necessary because a lot of toolchains define$CC
as something likeCC=arm-poky-linux-gnueabi-gcc -march=armv7-a -mfpu=neon -mfloat-abi=hard -mcpu=cortex-a8
where some parameters are already included. Without the
linux.Raw
, tbot would run$ "${CC}" main.c
where the arguments are interpreted as part of the path to the compiler. This will obviously fail so instead, with the
linux.Raw
, tbot will run$ ${CC} main.c
where the shell expansion will do the right thing.
-
abstract property
toolchains
¶ Return a dictionary of all toolchains that exist on this buildhost.
Example:
@property def toolchains(self) -> typing.Dict[str, linux.build.Toolchain]: return { "armv7a": linux.build.DistroToolchain("arm", "arm-linux-gnueabi-"), "armv8": linux.build.DistroToolchain("aarch64", "aarch64-linux-gnu-"), "mipsel": linux.build.DistroToolchain("mips", "mipsel-linux-gnu-"), }
-
abstract property
Toolchains¶
-
class
tbot.machine.linux.build.
DistroToolchain
(arch: str, prefix: str)[source]¶ Bases:
tbot.machine.linux.build.Toolchain
Toolchain that was installed from distribution repositories.
Example:
@property def toolchains(self): return { "armv7a": linux.build.DistroToolchain("arm", "arm-linux-gnueabi-"), "armv8": linux.build.DistroToolchain("aarch64", "aarch64-linux-gnu-"), }
-
class
tbot.machine.linux.build.
EnvScriptToolchain
(path: tbot.machine.linux.path.Path[H])[source]¶ Bases:
tbot.machine.linux.build.Toolchain
Toolchain that is initialized using an env script (e.g. yocto toolchain).
Example:
@property def toolchains(self): yocto_root = self.fsroot / "opt" / "poky" / "2.6" return { "cortexa8hf": linux.build.EnvScriptToolchain( yocto_root / "environment-setup-cortexa8hf-neon-poky-linux-gnueabi", ), }
Create a new EnvScriptToolchain.
- Parameters
path (linux.Path) – Path to the env script
Authenticators¶
For logging in via SSH using ParamikoConnector
or
SSHConnector
, tbot provides the following ‘authenticators’:
-
class
tbot.machine.linux.auth.
NoneAuthenticator
[source]¶ Bases:
tbot.machine.linux.auth.AuthenticatorBase
Most primitive authenticator.
Tries not passing any specific credentials and hopes ssh-config already contains all necessary infos. This is the default.
-
class
tbot.machine.linux.auth.
PrivateKeyAuthenticator
(key_file: Union[str, pathlib.PurePath])[source]¶ Bases:
tbot.machine.linux.auth.AuthenticatorBase
Authenticate using a private-key file.
Example:
class MySSHHost(connector.SSHConnector, linux.Bash): username = "foouser" authenticator = linux.auth.PrivateKeyAuthenticator("/home/foo/.ssh/id_rsa_foo")
-
class
tbot.machine.linux.auth.
PasswordAuthenticator
(password: str)[source]¶ Bases:
tbot.machine.linux.auth.AuthenticatorBase
Authenticate using a password.
Danger
This method is very insecure and might lead to PASSWORDS BEING STOLEN.
Example:
class MySSHHost(connector.SSHConnector, linux.Bash): username = "root" authenticator = linux.auth.PasswordAuthenticator("hunter2")