aboutgitcodebugslistschat
path: root/test/lib
Commit message (Collapse)AuthorAgeFilesLines
* test: Rename slightly misleading "valgrind" testsDavid Gibson2022-08-201-2/+0
| | | | | | | | | | | | | | | | | | | The "valgrind" test cases are designed to pick up errors reported when passt is running under valgrind. But what it actually does is just kill the passt process, then see if it had a non-zero exit code. That means it will equally well pick up any other problems which caused passt to exit with an error status: either something detected within passt or as a result of passt being killed by an unexpected signal. The fact that the "valgrind" test is actually responsible for shutting down the passt process is non-obvious and can lead to problems when selectively running tests during debugging. Rename the "valgrind" tests to "shutdown" tests and run it regardless of whether we're using valgrind or not. This allows us to remove an ugly speacial case in the passt_in_ns teardown code. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Split setup/teardown functions for build and distro testsDavid Gibson2022-08-201-0/+15
| | | | | | | | | | | | Currently the build tests and distro tests share a common setup function. That works for now, but changes we want to make will mean they need slightly different setup, so split the setup functions in preparation. Currently, neither build nor distro tests have any teardown function. Again, future changes are going to mean we need to do some teardown, so create some empty for now teardown functions in preparation. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Remove unused *_XTERM variablesDavid Gibson2022-08-201-4/+0
| | | | | | | The DEMO_XTERM and CI_XTERM variables defined in test/lib/term aren't used anywhere. Remove them. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Clean up better after iperf testsDavid Gibson2022-07-221-1/+1
| | | | | | | | | | | | | The iperf based test commands create a bunch of .bw and .pid files for each iperf client and server. The server side .bw files are cleaned up afterwards, but the pid files are not, and none of the client side files are cleaned up. The latter doesn't really matter when the client is run on ephemeral guests, but sometimes we run it in a namespace that shares the filesystem with the host. Clean up all of these files after the tests. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Use dhclient --no-pid for namespaces in two_guests testsDavid Gibson2022-07-221-4/+4
| | | | | | | | | | Before starting the guests, these tests configure addresses in a pasta namespace using dhclient. However, because it's a user namespace, it's not running as "real" root and can't write to the dhclient pid file. This doesn't stop it working, but causes an ugly error message which we can avoid by using the --no-pid option. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Remove unnecessary ^D in passt_in_ns teardownDavid Gibson2022-07-221-2/+0
| | | | | | | | | | | | | | | | | teardown_passt_in_ns() sends a ^D to the NS pane, which appears to be intended to terminate the nsenter running there, leaving the namespace. However, we've also sent a ^D to the PASST pane which will exit the pasta instance which created the namespace. With the namespace destroyed the nsenter in the NS pane will be killed, so it does not need to be exited explicitly. In fact sending the extra ^D can be harmful, since it will exit the shell in which the nsenter was run, causing the whole pane to be closed. That can then mean that the "pane_wait NS" hangs indefinitely. I believe this will sometimes work, because there's a race between the various options here, but it should be more reliable without the extra ^D. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Fix detection of empty 'hout' responses in passt{,_in_ns} testsDavid Gibson2022-07-221-0/+14
| | | | | | | | | | | | | | The dhcp/passt and dhcp/passt_in_ns tests at least, and maybe others use 'hout' commands that need to be able to detect empty output. However, we don't set PS1, which means the screen-scraping logic which detects this may not be reliable. In addition, if the host is using a recent bash, it will have bracketed paste mode enabled which will also add escape codes which will mess up the empty output detection. Set the prompt and disable bracketed paste mode from the passt and passt_in_ns setups to avoid these problems. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Drop further ^D in passt demo teardownStefano Brivio2022-07-221-5/+1
| | | | | | | | | Similar case as the one fixed by David's patch "tests: Remove unnecessary ^D in passt_in_ns teardown": we happen to pseudo-randomly close panes by unnecessarily exiting the parent shells there, and subsequent pane_wait directives hang. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Explicitly list test files in test/run, remove "onlyfor" supportDavid Gibson2022-07-142-19/+0
| | | | | | | | | | | | | | | | | Currently test/run uses wildcards to run all of the tests in a directory. However, that wildcard list is filtered down by the "onlyfor" directives in the test files... usually to a single file. Therefore, just explicitly list the files we *really* want to run for this test mode. This makes it easier to see at the top level what tests will be executed, and to change that list temporarily while debugging specific failures. This means the "onlyfor" directive no longer has any purpose, and we can remove it. "onlyfor" was also the only used of the $MODE variable, so we can remove that too. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Don't automatically traverse directories of test filesDavid Gibson2022-07-141-2/+2
| | | | | | | | | | | | | | | The top level listing control of which tests to run is in test/run, however it uses the test() function which runs an entire directory of test files, filtered by some criteria. This makes it awkward to narrow down to a subset of tests when debugging a specific failure. To make this easier, have test() take an explicit list of test files to run, and have the caller in test/run handle the directory traversal. The construct we use for this is pretty awkward to handle the fact that we're in the source tree root directory rather than test/ at this point in test/run. Later cleanups will improve that. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Remove not-very-useful "req" directiveDavid Gibson2022-07-141-18/+10
| | | | | | | | | | | | | | | | | The test scripts support a "req" directive which requires one test script to be run before another. It's implemented by doing a topological sort based on these directives in the runner scripts, which is about as awkward as you'd expect in Bourne shell. It turns out we only use this functionality in one place - to make the "make install" test run after the plain "make" test. We also already have a simpler way of making sure tests run in a specific order: just put them into the same test script file. So, remove support for the "req" directive and just fold the build/all and build/install test scripts together. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Remove unused set_mode() functionDavid Gibson2022-07-141-13/+0
| | | | | | This utility function is never called. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Move mbuto download and execution to asset buildDavid Gibson2022-07-141-17/+6
| | | | | | | | | | | | Move the download of mbuto and using it to create a sample initramfs to the asset build makefile, rather than embedding it in the test scripts themselves. The two_guests tests used to use two separate copies of the mbuto image. As an initramfs the mbuto image is strictly readonly though, so that's not necessary. So, also use the same image for both guests. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Introduce makefile for building test assetsDavid Gibson2022-07-141-1/+1
| | | | | | | | | | | | | | A number of passt/pasta testcases have initial steps which are just about building images or other assets we need for the test proper. Repeating these for each test run can be quite costly. This patch makes a start on moving this sort of test asset building to a separate phase before running the tests proper. For now just add a Makefile to handle the asset building (although it doesn't build anything yet), and make the path where we'll be building the assets available to the tests. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* Invoke specific qemu-system-* binariesDavid Gibson2022-07-141-4/+13
| | | | | | | | | | | | | | | | | | | | | | A lot of tests and examples invoke qemu with the command "kvm". However, as far as I can tell, "kvm" being aliased to the appropriate qemu system binary is Debian specific. The binary names from qemu upstream - qemu-system-$ARCH - also aren't universal, but they are more common (they should be good for both Debian and Fedora at least). In order to still get KVM acceleration when available, we use the option "-M accel=kvm:tcg" to tell qemu to try using either KVM or TCG in that order A number of the places we invoked "kvm" are expecting specifically an x86 guest, and so it's also safer to explicitly invoke qemu-system-x86_64. Some others appear to be independent of the target arch (just wanting the same arch as the host to allow KVM acceleration). Although I suspect there may be more subtle x86 specific options in the qemu command lines, attempt to preserve arch independence by using $(uname -m). Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Add external mbuto profile, drop udhcpc, and switch to itStefano Brivio2022-07-061-1/+1
| | | | | | | | | This depends on a future change in mbuto to accept external profile files. Add a file defining what we need for tests and demos, dropping udhcpc and script as they're not needed anymore, and switch to it. Suggested-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* Use dhclient instead of udhcpcDavid Gibson2022-06-151-3/+3
| | | | | | | | | | | | For some reason, the passt/pasta tests and examples use dhclient for DHCPv6, but in most cases use udhcpc for DHCPv4. Change it to use dhclient for both DHCPv4 and DHCPv6. This means one less tool we need for testing, plus dhclient is easily available on Fedora whereas udhcpc is not. Note that the passt tests still rely on udhcpc indirectly because mbuto wants to put it into the guest images it generates. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* Tweak dhclient arguments for readabilityDavid Gibson2022-06-151-1/+1
| | | | | | | | | | | | | | A number of tests and examples use dhclient in both IPv4 and IPv6 modes. We use "dhclient -6" for IPv6, but usually just "dhclient" for IPv4. Add an explicit "-4" argument to make it more clear and explicit. In addition, when dhclient is run from within pasta it usually won't be "real" root, and so will not have access to write the default global pid file. This results in a mostly harmless but irritating error: Can't create /var/run/dhclient.pid: Permission denied We can avoid that by using the --no-pid flag to dhclient. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* Don't abbreviate ip(8) arguments in examples and testsDavid Gibson2022-06-151-2/+2
| | | | | | | | ip(8)'s ability to take abbreviated arguments (e.g. "li sh" instead of "link show") is very handy when using it interactively, but it doesn't make for very readable scripts and examples when shown that way. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Don't check exit code for every command in demo modeStefano Brivio2022-05-191-0/+3
| | | | | | | Having all those 'echo $?' is rather distracting in demos. Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Sefano Brivio <sbrivio@redhat.com>
* tests: Don't count number of test units for demosStefano Brivio2022-05-191-2/+4
| | | | | | | | ...there are no 'test' directives in demo, and this causes a script failure. Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Simplify *tools commands using pane_statusDavid Gibson2022-05-191-15/+10
| | | | | | | | | Now that we have pane_status to check the success of commands issued to panes, we can more easily check for the success of the 'which' commands used to check tool availability, rather than constructing, then parsing special "skip" output. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Add pane_status command to check for success of issued commandsDavid Gibson2022-05-193-50/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | When we use pane_wait to wait for a command issued to a tmux pane to finish we have no idea whether the command succeeded or not. This means that the test scripts can keep running long after the point something vital has failed, making it difficult to work out what went wrong. Add a new pane_status command that checks for success of the issued command and use it in most places instead of pane_wait. We still need explicit pane_wait where we're gathering explicit output with pane_parse, because the way we check the status with 'echo $?' means we lose track of that output. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: - instead of quitting the script, make a test fail if a command issued in a pane fails during a test, and loop until the status code is numeric in pane_status() as a hack to make it a bit more robust - retain usage of pane_wait() in iperf3 and teardown functions as we interrupt iperf3, passt, and pasta, so a non-zero exit code is expected - drop bogus ns_{1,2}_wait() calls in teardown_two_guests(), those functions were never implemented - use pane_status() for "guest" test directives too ] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Don't ignore errors during scriptDavid Gibson2022-05-191-5/+5
| | | | | | | | | | | | | | | | Most commands issued during the testing scripts aren't explicitly checked for errors. Therefore, if they fail, the shell will just keep on executing. This makes it difficult to figure out where things started going wrong if things fall over. Run the whole script with the set -e mode so that it will exit in the case of any (unchecked) failing command. To make this work we do need to add explicit checks / fallbacks for some commands which we expect to fail. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: use sh -e instead of setting -e later, so that we don't miss anything before set -e is issued] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Improve control character filtering in pane_parseDavid Gibson2022-05-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | pane_parse() attempts to grab the output from the last command issued into a tmux pane. It strips out control characters using tr, which in particular includes the final \r\n. However, this won't fully strip out terminal escape sequences. In particular this breaks if the shell in the pane is bash, with enable-bracketed-paste enabled in readline. That issues terminal sequences to enable and disable bracketed paste mode around every shell prompt. We can work around this because these escapes are followed by a \r (CR). More generally, it seems reasonable to assume that any terminal shenanigans followed by a CR, but not an LF is supposed to be hidden. So, use sed to strip everything before the second last CR. We still need the tr to remove the final \r\n from the string (sed processes a line at a time, and doesn't consider the CRLF part of the buffer it's processing). Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: modify regexp to keep foo\r\r\n unchanged, by matching on at least one CR and a non-CR afterwards: that's the usual output pattern for bash on Debian 8 and Debian 9] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Don't globally set tmux default-shellDavid Gibson2022-05-191-6/+7
| | | | | | | | | | | | | | | | | run_term() uses tmux set-option -g to globally set the default shell. Unfortunately this hits a chicken-and-egg problem that's common with many of tmux's session options. If there isn't already a tmux server running, we can't connect to set the option. If we attempt this after starting our session (and therefore the server), then the session will already be started with the previous default shell. In any case it's not a good idea to set tmux global options, since that might interfere with whatever else the user is doing in tmux. So, instead set the default-shell option locally to the session after starting it. To make sure we get the right shell for our initial script, explicitly invoke /bin/sh to interpret it. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Don't use tmux update-environmentDavid Gibson2022-05-191-4/+5
| | | | | | | | | | | | | | | | | The semantics of tmux's update-environment option are a bit confusing. It says it means the given variables are copied into the session environment from the source environment, but it's not entirely clear what the "source" environment means. From my experimentation it appeast to be the environment from which the tmux *server* is launched, not the one issuing the 'new-session' command. That makes it pretty much useles, certainly in our case where we have no way of knowing if the user has pre-existing tmux sessions. Instead use the new-session -e option to explicitly pass in the variables we want to propagate. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Add some debugging output for the test scripts themselvesDavid Gibson2022-05-191-0/+2
| | | | | | | | | | | | The DEBUG option for tests/run enables debugging options to passt/pasta, however that doesn't help with debugging the test scripts themselves, which are fairly fragile. Extend the DEBUG option so it also prints information on each command in the test scripts to make it easier to work out where things are falling over. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Update mbuto git URLsStefano Brivio2022-05-191-1/+1
| | | | | | The project is now at mbuto.sh, and git transport is enabled. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib: Add small delay before trying to parse outputStefano Brivio2022-04-071-0/+1
| | | | | | | Don't fetch the log too early, we might get output from previous commands. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib: Consistent cols, rows, poster attributes for asciinema playerStefano Brivio2022-04-072-2/+2
| | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test, seccomp, Makefile: Switch to valgrind runs for passt functional testsStefano Brivio2022-03-291-12/+28
| | | | | | | | | | | | | | Pass to seccomp.sh a list of additional syscalls valgrind needs as EXTRA_SYSCALLS in a new 'valgrind' make target, and add corresponding support in seccomp.sh itself. In test setup functions, start passt with valgrind, but not for performance tests. Add tests checking that valgrind exits without errors after all the other tests in the group are done. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/video: Fill in href attributes of video shortcutsStefano Brivio2022-03-281-1/+1
| | | | | | ...so that they can be indexed. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* conf, util, tap: Implement --trace option for extra verbose loggingStefano Brivio2022-03-251-0/+10
| | | | | | | | --debug can be a bit too noisy, especially as single packets or socket messages are logged: implement a new option, --trace, implying --debug, that enables all debug messages. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/setup: Unshare PID namespace in pasta_setup()Stefano Brivio2022-03-021-4/+8
| | | | | | ...otherwise, we'll leave processes (dhclient) around. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/term: Don't run demo when started as ./runStefano Brivio2022-02-281-1/+1
| | | | | | | I changed this in a previous commit by mistake, restore the original command. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test: Fix name of CI asciinema player in perf links handlerStefano Brivio2022-02-231-5/+4
| | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* demo, ci: Switch to asciinema(1) for terminal recordingsStefano Brivio2022-02-223-67/+84
| | | | | | | | | | For demos, cool-retro-term(1) looked fancier, but several threads of that and ffmpeg(1) are just messing up with performance testing. The CI videos started getting really big as well, and they were difficult to read. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test: Add demo for Podman with pastaStefano Brivio2022-02-224-3/+101
| | | | | | | | ...showing setup steps, some peculiarities as --net option, and a general side-to-side comparison with slirp4netns(1), including "quick" TCP and UDP throughput and latency benchmarks. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* passt, pasta: Namespace-based sandboxing, defer seccomp policy applicationStefano Brivio2022-02-211-12/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To reach (at least) a conceptually equivalent security level as implemented by --enable-sandbox in slirp4netns, we need to create a new mount namespace and pivot_root() into a new (empty) mountpoint, so that passt and pasta can't access any filesystem resource after initialisation. While at it, also detach IPC, PID (only for passt, to prevent vulnerabilities based on the knowledge of a target PID), and UTS namespaces. With this approach, if we apply the seccomp filters right after the configuration step, the number of allowed syscalls grows further. To prevent this, defer the application of seccomp policies after the initialisation phase, before the main loop, that's where we expect bad things to happen, potentially. This way, we get back to 22 allowed syscalls for passt and 34 for pasta, on x86_64. While at it, move #syscalls notes to specific code paths wherever it conceptually makes sense. We have to open all the file handles we'll ever need before sandboxing: - the packet capture file can only be opened once, drop instance numbers from the default path and use the (pre-sandbox) PID instead - /proc/net/tcp{,v6} and /proc/net/udp{,v6}, for automatic detection of bound ports in pasta mode, are now opened only once, before sandboxing, and their handles are stored in the execution context - the UNIX domain socket for passt is also bound only once, before sandboxing: to reject clients after the first one, instead of closing the listening socket, keep it open, accept and immediately discard new connection if we already have a valid one Clarify the (unchanged) behaviour for --netns-only in the man page. To actually make passt and pasta processes run in a separate PID namespace, we need to unshare(CLONE_NEWPID) before forking to background (if configured to do so). Introduce a small daemon() implementation, __daemon(), that additionally saves the PID file before forking. While running in foreground, the process itself can't move to a new PID namespace (a process can't change the notion of its own PID): mention that in the man page. For some reason, fork() in a detached PID namespace causes SIGTERM and SIGQUIT to be ignored, even if the handler is still reported as SIG_DFL: add a signal handler that just exits. We can now drop most of the pasta_child_handler() implementation, that took care of terminating all processes running in the same namespace, if pasta started a shell: the shell itself is now the init process in that namespace, and all children will terminate once the init process exits. Issuing 'echo $$' in a detached PID namespace won't return the actual namespace PID as seen from the init namespace: adapt demo and test setup scripts to reflect that. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/test: Introduce 'def' directive for frequently used patternsStefano Brivio2022-01-281-229/+267
| | | | | | | | | For distribution tests, we'll repeat some tests frequently. Add a 'def' directive that starts a block, ended by 'endef', whose execution can then be triggered by simply giving its name as a directive itself. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/term: Allow for a wider variety of prompt characters in pane_wait()Stefano Brivio2022-01-281-7/+10
| | | | | | | | We might have highlighting and slightly different prompts across different distributions, allow a more reasonable set of prompt strings to be accepted as prompts. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/video: Drop -preset ultrafast from ffmpeg argumentsStefano Brivio2022-01-261-1/+1
| | | | | | | It's not really needed on a reasonably powered CPU, and makes the video contents way less readable. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/setup: Don't rely on IFS to properly separate qemu argumentsStefano Brivio2022-01-261-32/+32
| | | | | | ...this gets needlessly annoying while playing with test cases. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* README, perf_report: Markdown and CSS fixesStefano Brivio2021-10-221-7/+8
| | | | | | Updating md2html on the server needs a few adjustments. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/perf: Actually load passt enough to test UDP performanceStefano Brivio2021-10-211-5/+8
| | | | | | | | | With recent improvements, we're not CPU-bound at all while testing UDP performance. Give the VM more memory and CPUs, forward two additional ports, start up to four threads in parallel, and give single iperf3 threads higher bandwidth targets. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/test: Wait a bit longer before terminating iperf3 processesStefano Brivio2021-10-211-3/+3
| | | | | | | Sometimes tests run a few seconds longer than expected, wait a few more seconds. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test: Drop debugging left-overs in lib/utilStefano Brivio2021-10-141-4/+0
| | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/term: Export PCAP and DEBUG variables for tmux sessions globallyStefano Brivio2021-10-051-0/+1
| | | | | | Otherwise, this would depend on the local tmux configuration. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/setup: Increase --max-stackframe in commented-out valgrind commandStefano Brivio2021-10-051-1/+1
| | | | | | ...so that I don't keep fighting with this for pasta clone() calls. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-05 08:06:24 +0000