aboutgitcodebugslistschat
path: root/test
Commit message (Collapse)AuthorAgeFilesLines
...
* tests: Remove not-very-useful "req" directiveDavid Gibson2022-07-143-53/+31
| | | | | | | | | | | | | | | | | The test scripts support a "req" directive which requires one test script to be run before another. It's implemented by doing a topological sort based on these directives in the runner scripts, which is about as awkward as you'd expect in Bourne shell. It turns out we only use this functionality in one place - to make the "make install" test run after the plain "make" test. We also already have a simpler way of making sure tests run in a specific order: just put them into the same test script file. So, remove support for the "req" directive and just fold the build/all and build/install test scripts together. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Remove unused set_mode() functionDavid Gibson2022-07-141-13/+0
| | | | | | This utility function is never called. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Search multiple places for aarch64 EDK2 bios imageDavid Gibson2022-07-146-13/+30
| | | | | | | | | | | | | | | | | Apparently qemu's ARM virt machine needs to be explicitly given a firmware image, rather than just supplying a sane default. Unfortunately the EDK2 firmware image we need isn't in the same place on all host distros. Currently the test scripts hardcode the Debian location, meaning it will break on hosts that have it somewhere else. This patch searches multiple locations for the firmware, and creates a local link during the asset build phase, which the tests can then use. For now it only searches the locations used by Debian and Fedora, but that's a small improvement in robustness already, and can be later improved further if we need to. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Move mbuto download and execution to asset buildDavid Gibson2022-07-143-19/+16
| | | | | | | | | | | | Move the download of mbuto and using it to create a sample initramfs to the asset build makefile, rather than embedding it in the test scripts themselves. The two_guests tests used to use two separate copies of the mbuto image. As an initramfs the mbuto image is strictly readonly though, so that's not necessary. So, also use the same image for both guests. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Introduce makefile for building test assetsDavid Gibson2022-07-142-1/+28
| | | | | | | | | | | | | | A number of passt/pasta testcases have initial steps which are just about building images or other assets we need for the test proper. Repeating these for each test run can be quite costly. This patch makes a start on moving this sort of test asset building to a separate phase before running the tests proper. For now just add a Makefile to handle the asset building (although it doesn't build anything yet), and make the path where we'll be building the assets available to the tests. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* Invoke specific qemu-system-* binariesDavid Gibson2022-07-146-31/+40
| | | | | | | | | | | | | | | | | | | | | | A lot of tests and examples invoke qemu with the command "kvm". However, as far as I can tell, "kvm" being aliased to the appropriate qemu system binary is Debian specific. The binary names from qemu upstream - qemu-system-$ARCH - also aren't universal, but they are more common (they should be good for both Debian and Fedora at least). In order to still get KVM acceleration when available, we use the option "-M accel=kvm:tcg" to tell qemu to try using either KVM or TCG in that order A number of the places we invoked "kvm" are expecting specifically an x86 guest, and so it's also safer to explicitly invoke qemu-system-x86_64. Some others appear to be independent of the target arch (just wanting the same arch as the host to allow KVM acceleration). Although I suspect there may be more subtle x86 specific options in the qemu command lines, attempt to preserve arch independence by using $(uname -m). Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: qemu-system-ppc64le isn't a thingDavid Gibson2022-07-142-6/+6
| | | | | | | | | | | | Several tests run pp64le guests using "qemu-system-ppc64le". But, at the system level there's no difference between ppc64 and ppc64le - it's the same hardware, just placed into different endian modes by OS early boot code. Reflecting that, qemu only supplies a single "qemu-system-ppc64". Some distros alias qemu-system-ppc64le to qemu-system-ppc64 (Debian does), but it's best not to count on this (Fedora doesn't, for example). Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Embed script for dhclient(8) in mbuto(1) profileStefano Brivio2022-07-144-10/+26
| | | | | | | | | | | | | | | | | | | | | | | | | David reports that dhclient-script(8) on Fedora needs a number of binaries that are not included in PROGS of the current mbuto profile, and we would also need to include hostnamectl(1) there, which will fail without a systemd init. Embed a minimal script for dhclient(8) in the profile itself, written to /sbin/dhclient-script at boot, to just check what we need to check out of DHCP and DHCPv6 functionality. While at it, drop busybox and logger from PROGS, as we don't need them, and add hostname(1). While DHCP option 12 isn't supported yet by the DHCP implementation in passt, we should probably add it soon. Note: owing to the simplicity of this script, we now need to bring up the interface before starting dhclient: add this in test scripts where it's not the case yet. Reported-by: David Gibson <david@gibson.dropbear.id.au> Suggested-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio@redhat.com> note that we need to bring up the interface before starting dhclient
* test: Add external mbuto profile, drop udhcpc, and switch to itStefano Brivio2022-07-063-2/+39
| | | | | | | | | This depends on a future change in mbuto to accept external profile files. Add a file defining what we need for tests and demos, dropping udhcpc and script as they're not needed anymore, and switch to it. Suggested-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* Makefile: Use $(BIN) and $(MANPAGES) variable to simplify several targetsDavid Gibson2022-06-184-4/+4
| | | | | | | | | | | | | | There are several places which explicitly list the various generated binaries, even though a $(BIN) variable already lists them. There are several more places that list all the manpage files, introduce a $(MANPAGES) variable to remove that repetition as well. Tweak the generation of pasta.1 as a link to passt.1 so it's not just made as a side effect of the pasta target. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: add passt.1 and qrap.1 to guest files for distro tests] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Use nmap-ncat instead of openbsd netcat for pasta testsDavid Gibson2022-06-184-47/+48
| | | | | | | | | | | | | | | | A number of the testcases use options specific the OpenBSD version of netcat. That's available in Debian, but not easily available in Fedora. Switch the pasta tests to using the nmap version of netcat (a.k.a. ncat). This is easily available in both Debian and Fedora, and appears to be a bit more modern and maintained as well. ncat generally requires explicit listen addresses (which is good for clarity anywhere). Its default options appear to remove the need for the -N and -q options. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: changed one ncat listening address to IPv6 loopback] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* Use dhclient instead of udhcpcDavid Gibson2022-06-153-6/+6
| | | | | | | | | | | | For some reason, the passt/pasta tests and examples use dhclient for DHCPv6, but in most cases use udhcpc for DHCPv4. Change it to use dhclient for both DHCPv4 and DHCPv6. This means one less tool we need for testing, plus dhclient is easily available on Fedora whereas udhcpc is not. Note that the passt tests still rely on udhcpc indirectly because mbuto wants to put it into the guest images it generates. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* Tweak dhclient arguments for readabilityDavid Gibson2022-06-159-15/+15
| | | | | | | | | | | | | | A number of tests and examples use dhclient in both IPv4 and IPv6 modes. We use "dhclient -6" for IPv6, but usually just "dhclient" for IPv4. Add an explicit "-4" argument to make it more clear and explicit. In addition, when dhclient is run from within pasta it usually won't be "real" root, and so will not have access to write the default global pid file. This results in a mostly harmless but irritating error: Can't create /var/run/dhclient.pid: Permission denied We can avoid that by using the --no-pid flag to dhclient. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* Don't abbreviate ip(8) arguments in examples and testsDavid Gibson2022-06-1520-109/+109
| | | | | | | | ip(8)'s ability to take abbreviated arguments (e.g. "li sh" instead of "link show") is very handy when using it interactively, but it doesn't make for very readable scripts and examples when shown that way. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Use more explicit netcat options for distro/fedora testsDavid Gibson2022-06-151-7/+7
| | | | | | | | | | | | | | | | | | distro/fedora contains two versions of the basic tests, used for different Fedora versions. One uses explicit listening address for netcat in some extra places, the other does not. Apparently the older netcat versions didn't require the explicit addresses. Not supplying addresses doesn't test anything useful though, just a detail in netcat's behaviour. So, it's cleaner to just always supply explicit addresses. In addition, we're explicitly expecting the nmap version of ncat, also known as "ncat". So, it's more explicit what we're after if we invoke it via that name rather than "nc", which will go via an /etc/alternatives link. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: Fix port argument in distro_quick_pasta_test{,_fedora34} too] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Don't check exit code for every command in demo modeStefano Brivio2022-05-191-0/+3
| | | | | | | Having all those 'echo $?' is rather distracting in demos. Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Sefano Brivio <sbrivio@redhat.com>
* tests: Don't count number of test units for demosStefano Brivio2022-05-191-2/+4
| | | | | | | | ...there are no 'test' directives in demo, and this causes a script failure. Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* demo/pasta: Fix bad sleep directiveStefano Brivio2022-05-191-1/+1
| | | | | | | | 'sleep' always needs an argument, this was meant to introduce a 2 seconds delay. Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/run: Return 0 from run(), exit value already reflects failuresStefano Brivio2022-05-191-1/+1
| | | | | | | | | | | There's no need to return non-zero if there have been failures in run(), because the exit value is already determined from the number of failures reported in the log file. Return zero, so that this doesn't cause the script to fail, given we now run it with -e. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/perf/pasta_udp: Drop redundant assignment of ::1 to loopback interfaceStefano Brivio2022-05-191-1/+0
| | | | | | | | | | | There are a few occurrences of this assignment, which are needed to re-add ::1 as loopback address after the MTU has been increased back from a value below 1280 bytes. This one, however, is redundant, and causes an error in the execution. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Simplify explicit checks for command successDavid Gibson2022-05-195-52/+24
| | | | | | | | | A number of individual test cases use '*out' commands to check for success of specific commands they've issued. Now that the test harness is testing for success of all issued commands as a matter of course, we no longer need to do this. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Simplify *tools commands using pane_statusDavid Gibson2022-05-191-15/+10
| | | | | | | | | Now that we have pane_status to check the success of commands issued to panes, we can more easily check for the success of the 'which' commands used to check tool availability, rather than constructing, then parsing special "skip" output. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Add pane_status command to check for success of issued commandsDavid Gibson2022-05-193-50/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | When we use pane_wait to wait for a command issued to a tmux pane to finish we have no idea whether the command succeeded or not. This means that the test scripts can keep running long after the point something vital has failed, making it difficult to work out what went wrong. Add a new pane_status command that checks for success of the issued command and use it in most places instead of pane_wait. We still need explicit pane_wait where we're gathering explicit output with pane_parse, because the way we check the status with 'echo $?' means we lose track of that output. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: - instead of quitting the script, make a test fail if a command issued in a pane fails during a test, and loop until the status code is numeric in pane_status() as a hack to make it a bit more robust - retain usage of pane_wait() in iperf3 and teardown functions as we interrupt iperf3, passt, and pasta, so a non-zero exit code is expected - drop bogus ns_{1,2}_wait() calls in teardown_two_guests(), those functions were never implemented - use pane_status() for "guest" test directives too ] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Don't ignore errors during scriptDavid Gibson2022-05-192-6/+6
| | | | | | | | | | | | | | | | Most commands issued during the testing scripts aren't explicitly checked for errors. Therefore, if they fail, the shell will just keep on executing. This makes it difficult to figure out where things started going wrong if things fall over. Run the whole script with the set -e mode so that it will exit in the case of any (unchecked) failing command. To make this work we do need to add explicit checks / fallbacks for some commands which we expect to fail. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: use sh -e instead of setting -e later, so that we don't miss anything before set -e is issued] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Improve control character filtering in pane_parseDavid Gibson2022-05-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | pane_parse() attempts to grab the output from the last command issued into a tmux pane. It strips out control characters using tr, which in particular includes the final \r\n. However, this won't fully strip out terminal escape sequences. In particular this breaks if the shell in the pane is bash, with enable-bracketed-paste enabled in readline. That issues terminal sequences to enable and disable bracketed paste mode around every shell prompt. We can work around this because these escapes are followed by a \r (CR). More generally, it seems reasonable to assume that any terminal shenanigans followed by a CR, but not an LF is supposed to be hidden. So, use sed to strip everything before the second last CR. We still need the tr to remove the final \r\n from the string (sed processes a line at a time, and doesn't consider the CRLF part of the buffer it's processing). Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: modify regexp to keep foo\r\r\n unchanged, by matching on at least one CR and a non-CR afterwards: that's the usual output pattern for bash on Debian 8 and Debian 9] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Don't globally set tmux default-shellDavid Gibson2022-05-191-6/+7
| | | | | | | | | | | | | | | | | run_term() uses tmux set-option -g to globally set the default shell. Unfortunately this hits a chicken-and-egg problem that's common with many of tmux's session options. If there isn't already a tmux server running, we can't connect to set the option. If we attempt this after starting our session (and therefore the server), then the session will already be started with the previous default shell. In any case it's not a good idea to set tmux global options, since that might interfere with whatever else the user is doing in tmux. So, instead set the default-shell option locally to the session after starting it. To make sure we get the right shell for our initial script, explicitly invoke /bin/sh to interpret it. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Don't use tmux update-environmentDavid Gibson2022-05-191-4/+5
| | | | | | | | | | | | | | | | | The semantics of tmux's update-environment option are a bit confusing. It says it means the given variables are copied into the session environment from the source environment, but it's not entirely clear what the "source" environment means. From my experimentation it appeast to be the environment from which the tmux *server* is launched, not the one issuing the 'new-session' command. That makes it pretty much useles, certainly in our case where we have no way of knowing if the user has pre-existing tmux sessions. Instead use the new-session -e option to explicitly pass in the variables we want to propagate. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Add some debugging output for the test scripts themselvesDavid Gibson2022-05-191-0/+2
| | | | | | | | | | | | The DEBUG option for tests/run enables debugging options to passt/pasta, however that doesn't help with debugging the test scripts themselves, which are fairly fragile. Extend the DEBUG option so it also prints information on each command in the test scripts to make it easier to work out where things are falling over. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Remove unused XVFB variableDavid Gibson2022-05-191-3/+0
| | | | | | | The XVFB variable is initialized at the beginning of test/run then never used again. I'm assuming it's a leftover from some ealier iteration. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Update mbuto git URLsStefano Brivio2022-05-192-2/+2
| | | | | | The project is now at mbuto.sh, and git transport is enabled. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* Add basic .gitignore filesDavid Gibson2022-05-191-0/+2
| | | | | | | Ignore various files generated during build or test. Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* demo: podman: New port forwarding behaviour for pasta, minor fixesStefano Brivio2022-05-021-78/+56
| | | | | | | | | | | | Reflect the recent changes in the Podman adaptation (no port forwarding by default). It turns out that by running two iperf3 processes, sometimes slirp4netns blocks the second connection until the first test is done, thus doubling the throughput. Use a single process for slirp4netns with slirp4netns port handling. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib: Add small delay before trying to parse outputStefano Brivio2022-04-071-0/+1
| | | | | | | Don't fetch the log too early, we might get output from previous commands. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/distro: Set unprivileged_userns_clone on Debian Buster and earlierStefano Brivio2022-04-071-0/+4
| | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib: Consistent cols, rows, poster attributes for asciinema playerStefano Brivio2022-04-072-2/+2
| | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/perf: Work-around for virtio_net hang before long streams from guestStefano Brivio2022-03-292-0/+30
| | | | | | | I didn't have time to investigate the root cause for the virtio_net TX hang yet. Add a quick work-around for the moment being. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test, seccomp, Makefile: Switch to valgrind runs for passt functional testsStefano Brivio2022-03-296-13/+91
| | | | | | | | | | | | | | Pass to seccomp.sh a list of additional syscalls valgrind needs as EXTRA_SYSCALLS in a new 'valgrind' make target, and add corresponding support in seccomp.sh itself. In test setup functions, start passt with valgrind, but not for performance tests. Add tests checking that valgrind exits without errors after all the other tests in the group are done. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test: Add asciinema(1) as requirement for CI in READMEStefano Brivio2022-03-281-1/+1
| | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/video: Fill in href attributes of video shortcutsStefano Brivio2022-03-281-1/+1
| | | | | | ...so that they can be indexed. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* conf, util, tap: Implement --trace option for extra verbose loggingStefano Brivio2022-03-252-1/+15
| | | | | | | | --debug can be a bit too noisy, especially as single packets or socket messages are logged: implement a new option, --trace, implying --debug, that enables all debug messages. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/setup: Unshare PID namespace in pasta_setup()Stefano Brivio2022-03-021-4/+8
| | | | | | ...otherwise, we'll leave processes (dhclient) around. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* demo/pasta: Clean up before rebuilding with -gStefano Brivio2022-03-011-0/+1
| | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* passt, pasta: Run-time selection of AVX2 buildStefano Brivio2022-02-283-12/+2
| | | | | | | | | | | | | Build-time selection of AVX2 flags and routines is not practical for distributions, but limiting AVX2 usage to checksum routines with specific run-time detection doesn't allow for easy performance gains from auto-vectorisation of batched packet handling routines. For x86_64, build non-AVX2 and AVX2 binaries, and implement a simple wrapper replacing the current executable with the AVX2 build if it's available, and if AVX2 is supported by the current CPU. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/distro/opensuse: Add Tumbleweed armv7l testStefano Brivio2022-02-281-0/+29
| | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/term: Don't run demo when started as ./runStefano Brivio2022-02-281-1/+1
| | | | | | | I changed this in a previous commit by mistake, restore the original command. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* demo/pasta: Exit namespace in 'ns' pane before restarting pastaStefano Brivio2022-02-281-0/+1
| | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test: Fix name of CI asciinema player in perf links handlerStefano Brivio2022-02-231-5/+4
| | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* demo, ci: Switch to asciinema(1) for terminal recordingsStefano Brivio2022-02-225-92/+101
| | | | | | | | | | For demos, cool-retro-term(1) looked fancier, but several threads of that and ffmpeg(1) are just messing up with performance testing. The CI videos started getting really big as well, and they were difficult to read. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test: Add demo for Podman with pastaStefano Brivio2022-02-226-3/+951
| | | | | | | | ...showing setup steps, some peculiarities as --net option, and a general side-to-side comparison with slirp4netns(1), including "quick" TCP and UDP throughput and latency benchmarks. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/perf/passt_udp: Drop threshold for 256B testStefano Brivio2022-02-211-2/+2
| | | | | | | | | That test fails sometimes, it looks like iperf3 is still sending initial messages that are too big. I'll need to figure out why, but given that 256 bytes is not really an expected MTU, drop the thresholds to zero for the moment being. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-15 03:23:57 +0000