aboutgitcodebugslistschat
path: root/test
Commit message (Collapse)AuthorAgeFilesLines
* test: Move pidfiles and nsholder sockets into state directoryDavid Gibson2022-09-133-35/+35
| | | | | | | Currently they go in the passt source tree with a fixed names, which means their presence can mess with subsequent test runs. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Store pcap files in $LOGDIR instead of /tmpDavid Gibson2022-09-131-8/+8
| | | | | | | The capture files are more or less a different form of log output from the tests, so place them in $LOGDIR. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move pause temporary file to state directoryDavid Gibson2022-09-131-2/+2
| | | | Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use paths in __STATEDIR__ instead of 'temp' and 'tempdir' directivesDavid Gibson2022-09-1315-82/+69
| | | | | | | | | | | | Instead of using the 'temp' and 'tempdir' DSL directives to create temporary files, use fixed paths relative to __STATEDIR__. This has two advantages: 1) The files are automatically cleaned up if the tests fail (and even if that doesn't work they're easier to clean up manuall) 2) When debugging tests it's easier to figure out which of the temporary files are relevant to whatever's going wrong Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Don't redundantly regenerate small test file in pasta/tcpDavid Gibson2022-09-131-3/+0
| | | | | | | In what looks like a copy/paste error, pasta/tcp generates its small test file twice. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move context temporary files to state dirDavid Gibson2022-09-131-29/+32
| | | | | | | | | Currently the context command dispatch subsystem creates a bunch of temporary files in $LOGDIR, which is messy. Store them in $STATEDIR which is for precisely this purpose. The logs from each context still go into $LOGDIR. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move passt_test_log_pipe to state directoryDavid Gibson2022-09-133-19/+16
| | | | | | | We use this fifo to send messages to the information pane. Put it in the state directory so it doesn't need its own cleanup. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Create common state directories for temporary filesDavid Gibson2022-09-134-1/+9
| | | | | | | | | | | | | | | | | | The test scripts create a bunch of temporary files to keep track of internal state. Some are made in /tmp with individual mktemp calls, some go in the passt source directory, and some go in $LOGDIR. This can sometimes make it messy to clean up after failed test runs. Start cleaning this up by creating a single "state" directory ($STATEBASE) in /tmp for all the state or temporary files used by a single test run. Clean it up automatically in cleanup() - except when DEBUG==1, because those files can be useful for debugging test script failures. We create subdirectories under $STATEBASE for each setup function, exposed as $STATESETUP. We also create subdirectories for each test script and expose those to the scripts as __STATEDIR__. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Actually run cleanup functionDavid Gibson2022-09-131-1/+0
| | | | | | | | | | | | | | We install a cleanup() function with 'trap' in order to clean up temporary files we generate during the tests. However, we deinstall it after run_term, which means it won't run in most of the cases where it would be useful. Even if "run from_term" exits with an error, that error will be hidden from the run_term wrapper because it's within a tmux session, so we will return from run_term normally, uninstall the trap and never clean up. In fact there's no reason to uninstall the trap at all, it works just as well on the success exit path as an error exit path. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Remove unused variable FFMPEG_PID_FILEDavid Gibson2022-09-131-1/+0
| | | | | | FFPMPEG_PID_FILE is set (creating a temporary file), then never used. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Group tests by mode then protocol, rather than the reverseDavid Gibson2022-09-1314-32/+32
| | | | | | | | For example, passt/dhcp rather than dhcp/passt. This is more consistent with the two_guests and other test groups, and makes some other cleanups simpler. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use new-style command issue for passt_in_ns testsDavid Gibson2022-09-135-52/+37
| | | | | | | Put the pieces together to use the new style context based dispatch for the passt_in_pasta tests. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use context system for two_guests testsDavid Gibson2022-09-132-73/+57
| | | | | | | | Now that we have all the pieces we need for issuing commands both into namespaces and into guests, we can use those to convert the two_guests to using only the new style context command issue. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use context system for guest commandsDavid Gibson2022-09-1310-16/+90
| | | | | | | | | | | | | | | Extends the context system in the test scripts to allow executing commands within a guest. Do this without requiring an existing network in the guest by using socat to run ssh via a vsock connection. We do need some additional "sleep"s in the tests, because the new faster dispatch means that sometimes we attempt to connect before socat has managed to listen. For now, only use this for the plain "passt" tests. The "passt_in_ns" and other tests have additional complications we still need to deal with. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Extend context system to run commands in namespace for pasta testsDavid Gibson2022-09-133-13/+29
| | | | | | | Extend the context system to allow commands to be run in a namespace created with unshare, and use it for the namespace used in the pasta tests. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Add nsholder utilityDavid Gibson2022-09-133-1/+147
| | | | | | | | | | | | | | | | | | | | | | | | | | | In our test scripts we need to do some ugly parsing of /proc and/or pstree output in order to get the PIDs of processes running in namespaces so that we can connect to those namespaces with nsenter or pasta. This is actually a pretty tricky problem with standard tools. To determine the PID from the outside of the namespace we need to know how the process of interest is related to the unshare or pasta process (child? one of several children? grandchild?) as well as then parsing /proc or ps output. This is slightly awkward now, and will get worse with future changes I'd like to make to have processes are dispatched. The obvious solution would be to have the process of interest (which we control) report its own PID, but that doesn't work easily, because it is in a PID namepace and sees only its local PID not the global PID we need to address it from outside. To handle this, add a small custom tool, "nsholder". This takes a path and a mode parameter. In "hold" mode it will create a unix domain socket bound to the path and listening. In "pid" mode it will get the "hold"ing process's pid via the unix socket using SO_PEERCRED, which translates between PID namespaces. In "stop" mode it will send a message to the socket causing the "hold"ing process to clean up and exit. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use new-style contexts for passt pane in the pasta and passt testsDavid Gibson2022-09-133-19/+12
| | | | | | | | | | Convert the pasta and passt tests to use new-style context execution for the things that run in the "passt" frame. Don't touch the passt_in_ns or two_guests tests yet, because they run passt inside a namespace which introduces some additional complications we have yet to handle. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Issue host commands via context for most testsDavid Gibson2022-09-132-25/+42
| | | | | | | | | Convert most of the tests to use the new-style system for issuing commands for all host commands. We leave the distro tests for now: they use the same pane for both host and guest commands which we'll need some more things to deal with. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Integration of old-style pane execution and new context executionDavid Gibson2022-09-133-81/+121
| | | | | | | | | | | | | | | | We're creating a system for tests to more reliably execute commands in various contexts (e.g. host, guest, namespace). That transition is going to happen over a number of steps though, so in the meantime we need to deal with both the old-style issuing of commands via typing into and screen scraping tmux panels, and the new-style system for executing commands in context. Introduce some transitional helpers which will issue a command via context if the requested context is initialized, but will otherwise fall back to the old style tmux panel based method. Re-implement the various test DSL commands in terms of these new helpers. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Allow a tmux pane to watch commands executed in contextsDavid Gibson2022-09-131-0/+18
| | | | | | | | | | | | | | | | We're moving to a new way of the tests dispatching commands to running in contexts (host, guest, namespace, etc.). As we make this transition, though, we still want the user to be able to watch the commands running in a context, as they previously could from the commands issued in the pane. Add a helper to set up a pane to watch a context's log to allow this. In some cases we currently issue commands from several different logical contexts in the same pane, so allow a pane to watch several contexts at once. Also use tail's --retry option to allow starting the watch before we've initialized the context which will be useful in some cases. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Context execution helpersDavid Gibson2022-09-131-0/+80
| | | | | | | | | | | | | | | | | | For the tests, we need to run commands in various contexts: in the host, in a guest or in a namespace. Currently we do this by running each context in a tmux pane, and using tmux commands to type the commands into the relevant pane, then screen-scrape the output for the results if we need them. This is very fragile, because we have to make various assumptions to parse the output. Those can break if a shell doesn't have the prompt we expect, if the tmux pane is too small or in various other conditions. This starts some library functions for a new "context" system, that provides a common way to invoke commands in a given context, in a way that properly preserves stdout, stderr and the process return code. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Correctly match "background" with "wait" commandsDavid Gibson2022-09-134-8/+6
| | | | | | | | | | | | Our test DSL has a number of paired commands to run something in the background in a pane, then later to wait for it to complete. However, in some of the tests we have these mismatched - starting a command in one pane, then waiting for it in another. We appear to get away with this for some reason, but it's not correct and future changes make it cause more problems. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Rewrite test_iperf3David Gibson2022-09-073-44/+30
| | | | | | | | | | | | | | | test_iperf3() is a pretty inscrutable mess of nested background processes. It has a number of ugly sleeps needed to wait for things to complete. Rewrite it to be cleaner: * Use the construct (a & b & wait) to run 'a' and 'b' in parallel, but then wait for them both to complete before continuing * This allows us to wait for both the server and client to finish, rather than sleeping * Use jq to do all the math we need to get the final result, rather than jq followed by some complicated 'bc' mangling Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Parameterize run time for throughput performance testsDavid Gibson2022-09-075-87/+95
| | | | | | | | | | | | Currently all the throughput tests are run for 30s. This is reflected in both the actual parameters given to the iperf commands, but also in the matching sleeps in test_iperf3. Allow this to be adjusted more easily with a new parameter to test_iperf3. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: Reflect new parameter in comment to test_iperf3()] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test: Combine iperf3c and iperf3s into a single DSL commandDavid Gibson2022-09-075-197/+122
| | | | | | | | | | | | | | | | | These two commands in the DSL to run an iperf client and server are always used together, and some of the parameters must match between them. The iperf3s must also be run more or less immediately after iperf3c, since iperf3c will run a client in the background after a sleep and requires a server to be running before it will work. A bunch of things can be made cleaner if we make a single DSL command that runs both sides of the test. For now make the combined command work exactly like the two commands together did, warts and all. This does lose the ability for the DSL scripts to give additional options to the iperf3 server, but we weren't using that anyway. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Wait for systemd-resolved to be ready on Ubuntu 22.04 for s390xStefano Brivio2022-09-051-2/+6
| | | | | | | | | | | | | | On new Ubuntu 22.04 images, stopping systemd-resolved to get the dhclient script override resolv.conf doesn't work anymore. I originally used that hack to avoid introducing a delay which is needed when running it on TCG. Keep systemd-resolved running instead, and wait for it to be ready by retrying to resolve a domain a few times before installing packages, so that we don't add another ugly delay that might unnecessarily slow down things even further. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/README: Requirements for socket buffer sizes and hardware performance ↵Stefano Brivio2022-09-021-0/+16
| | | | | | | | | | | | | | | events Performance tests use iperf3(1) with large windows, and these sysctl entries are needed to run them unmodified. The passt demo uses perf(1) to report syscall overhead, and that needs access to hardware performance counters for unprivileged users. Reported-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
* test: debian: Export DEBIAN_FRONTEND=noninteractive for sid2022_08_20.f233d6cStefano Brivio2022-08-201-0/+3
| | | | | | | We start getting prompts about restarting outdated services: we're using daily images but they might have been cached for a while now. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test: Kill qemu by pidfile rather than ^CDavid Gibson2022-08-201-8/+12
| | | | | | | | | | | | | Currently in at least some of the testcases we kill qemu processes we're done with by issuing a Control-C to the tmux panel it's running in. That makes things harder as we try to move towards allowing "headless" testing without tmux. So, instead always use an explicit kill on a pid derived from a pidfile for killing qemu. Note that we don't need to remove the pidfiles afterwards, because qemu does that itself when terminated. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Log debugging output from test scriptDavid Gibson2022-08-201-0/+2
| | | | | | | | | | | | | | | | | | | The test scripts run with sh -e, which means they will stop if any commands return an error. That's generally desirable, because we won't continue after things are hopeless due to an earlier step failing. Unfortunately, the tmux setup we run the script in means it's not obvious where any error messages related to such a failure will go. Depending on exactly where the error occurs they might go to the original terminal hidden behind tmux, or they might go to a tmux panel that's not visible in the normal layouts. To make it easier to find such error message, redirect direct output and errors from the test script itself to a 'script.log' file in the logs directory. When in DEBUG=1 mode, additionaly 'set -x' so we log all the commands we execute to that file. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use shutdown test for pastaDavid Gibson2022-08-203-5/+4
| | | | | | | | | For the passt and passt_in_ns tests we have a "shutdown" testcase that checks for any errors from the passt process we were using (including valgrind warnings). Do the same for pasta tests, so that we catch any error codes from the pasta process. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Rename slightly misleading "valgrind" testsDavid Gibson2022-08-204-12/+13
| | | | | | | | | | | | | | | | | | | The "valgrind" test cases are designed to pick up errors reported when passt is running under valgrind. But what it actually does is just kill the passt process, then see if it had a non-zero exit code. That means it will equally well pick up any other problems which caused passt to exit with an error status: either something detected within passt or as a result of passt being killed by an unexpected signal. The fact that the "valgrind" test is actually responsible for shutting down the passt process is non-obvious and can lead to problems when selectively running tests during debugging. Rename the "valgrind" tests to "shutdown" tests and run it regardless of whether we're using valgrind or not. This allows us to remove an ugly speacial case in the passt_in_ns teardown code. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Only select a single interface or gateway in testsDavid Gibson2022-08-204-7/+7
| | | | | | | | | | The queries we use in the test scripts to locate the external interface or gateway can return multiple results. We get away with this because the way we parse command output only looks at the last line. It's not really correct, though, and improvements to our handling of command output will mean it breaks. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Split setup/teardown functions for build and distro testsDavid Gibson2022-08-202-0/+19
| | | | | | | | | | | | Currently the build tests and distro tests share a common setup function. That works for now, but changes we want to make will mean they need slightly different setup, so split the setup functions in preparation. Currently, neither build nor distro tests have any teardown function. Again, future changes are going to mean we need to do some teardown, so create some empty for now teardown functions in preparation. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Ignore video processing temporary filesDavid Gibson2022-08-201-0/+4
| | | | | | | | When recording tests or demos with asciinema we generate several temporary files during post-processing. Add these to the .gitignore file so they're not accidentally comitted. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Remove unused *_XTERM variablesDavid Gibson2022-08-201-4/+0
| | | | | | | The DEMO_XTERM and CI_XTERM variables defined in test/lib/term aren't used anywhere. Remove them. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Split cppcheck and clang-tidy tests into different filesDavid Gibson2022-08-203-5/+20
| | | | | | | | | | Both clang-tidy and cppcheck linting are handled by the same test file, test/build/static_checkers. The two linters are independent of each other though, and each one takes quite a long time. Split them into separate files to make it easier to control which are executed from the top level test script. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Convert distro tests to use socat instead of nc/ncatDavid Gibson2022-08-204-52/+74
| | | | | | | | | | | | | | | | | We've recently converted most of our tests to use socat instead of nc/netcat/ncat, because socat is more powerful and we don't need to deal with the several possible variants of netcat. We still use nc or ncat for the distro tests. Because there we control the guest environment and can pick our tools, there isn't the same reason to switch to socat. However, using socat here as well makes the tests a bit easier to read, and doesn't require people reading or modifying them to become familiar with an additional tool. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: keep using netcat-openbsd in Ubuntu 16.04 ppc64 test, as socat is unavailable there] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* Makefile: Ugly hack to get a "plain" Markdown version of READMEStefano Brivio2022-08-203-3/+3
| | | | | | | | | | | | | | | | | | Distribution packages reasonably expect to have a human-readable Markdown version of the README under /usr/share/doc/, but all we have right now is a heavily web-oriented version. Introduce a ugly hack to strip web-oriented parts from the current README and install it. It should probably work the other way around: a human-readable README could be used as a source for the web page. But cgit needs a file that's in the tree, not something that can be built, and https://passt.top/ is based on cgit. It should eventually be doable to work around this in cgit, instead. Reported-by: Benson Muite <benson_muite@emailplus.org> Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* contrib, test: Rebase Podman patch, enable three-way merge on git am in demo2022_08_04.b516d15Stefano Brivio2022-08-011-1/+1
| | | | | | | | Given that a three-way git merge was enough to cope with context changes in man pages, it's probably a good idea to enable that for 'git am' in the demo too. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* Separately locate external interfaces for IPv4 and IPv6David Gibson2022-07-304-5/+8
| | | | | | | | | | | | | | | | | Now that the back end allows passt/pasta to use different external interfaces for IPv4 and IPv6, use that to do the right thing in the case that the host has IPv4 and IPv6 connectivity via different interfaces. If the user hasn't explicitly chosen an interface, separately search for a suitable external interface for each protocol. As a bonus, this substantially simplifies the external interface probe. It also eliminates a subtle confusing case where in some circumstances we would pick the first interface in interface index order, and sometimes in order of routes returned from netlink. On some network configurations that could cause tests to fail, because the logic in the tests was subtly different (it always used route order). Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Correct determination of host interface name in testsDavid Gibson2022-07-302-2/+2
| | | | | | | | | | | | | By default, passt itself attaches to the first host interface with a default route. However, when determining the host interface name the tests implicitly select the *last* host interface: they use a jq expression which will list all interfaces with default routes, but the way output detection works in the scripts, it will only pick up the last line. If there are multiple interfaces with default routes on the host, and they each have a different address, this can cause spurious test failures. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Expand root partition of Debian sid amd64 and aarch64 imagesStefano Brivio2022-07-291-0/+12
| | | | | | | | | | | | | | | A couple of days ago, we started running out of space there as we're about to install gcc -- about 50 MiB are missing. Given that virt-resize (which could be conveniently invoked by the Makefile for tests) reorders partitions if we expand the first one, resize the image using qemu-img from the test script itself, and then take care of expanding root partition and filesystem online later. This is probably a temporary hack, so I'm not looking for a more generic or elegant solution at the moment. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* demo: Use git protocol downloadsDavid Gibson2022-07-222-2/+2
| | | | | | | | For some reason using https to clone from the passt git repo is very slow, at least from network-distant places. Use git protocol in the demo instead to avoid a tedious wait to get the source. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: No need to retrieve host ifname in ndp/pastaDavid Gibson2022-07-221-3/+2
| | | | | | | | | With pasta, the namespace interface name is generally the same as the host interface name. We already rely on this in the dhcp/pasta tests, but for no clear reason ndp/pasta separately determines the host interface name. Remove this unnecessary step. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Clean up better after iperf testsDavid Gibson2022-07-221-1/+1
| | | | | | | | | | | | | The iperf based test commands create a bunch of .bw and .pid files for each iperf client and server. The server side .bw files are cleaned up afterwards, but the pid files are not, and none of the client side files are cleaned up. The latter doesn't really matter when the client is run on ephemeral guests, but sometimes we run it in a namespace that shares the filesystem with the host. Clean up all of these files after the tests. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Use dhclient --no-pid for namespaces in two_guests testsDavid Gibson2022-07-221-4/+4
| | | | | | | | | | Before starting the guests, these tests configure addresses in a pasta namespace using dhclient. However, because it's a user namespace, it's not running as "real" root and can't write to the dhclient pid file. This doesn't stop it working, but causes an ugly error message which we can avoid by using the --no-pid option. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Remove unnecessary truncation of temporary files in udp testsDavid Gibson2022-07-223-18/+0
| | | | | | | | | All the UDP tests use :> to truncate some temporary data files. This appears to be so that they're empty before writing data to them with tee. However tee, by default, truncates its output file anyway (you need tee -a to append). So drop the unnecessary truncations. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Remove unnecessary ^D in passt_in_ns teardownDavid Gibson2022-07-221-2/+0
| | | | | | | | | | | | | | | | | teardown_passt_in_ns() sends a ^D to the NS pane, which appears to be intended to terminate the nsenter running there, leaving the namespace. However, we've also sent a ^D to the PASST pane which will exit the pasta instance which created the namespace. With the namespace destroyed the nsenter in the NS pane will be killed, so it does not need to be exited explicitly. In fact sending the extra ^D can be harmful, since it will exit the shell in which the nsenter was run, causing the whole pane to be closed. That can then mean that the "pane_wait NS" hangs indefinitely. I believe this will sometimes work, because there's a race between the various options here, but it should be more reliable without the extra ^D. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Use socat instead of netcatDavid Gibson2022-07-2211-200/+200
| | | | | | | | | | | | | | | Commit 41c02e10 ("tests: Use nmap-ncat instead of openbsd netcat for pasta tests") updated the pasta tests to use the nmap version of ncat instead of the openbsd version, for greater portability. For some upcoming changes, however, we'll be wanting to use socat. "socat" can do everything "ncat" can and more, so let's move all the tests using host tools (either directly on the host or via mbuto generated images) to using socat instead. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: Fix a typo in port specification, 31337 instead of x31337] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>