aboutgitcodebugslistschat
Commit message (Collapse)AuthorAgeFilesLines
* test/distro: Update workarounds for Ubuntu 22.04 on s390x2022_09_23.d6f865aStefano Brivio2022-09-231-1/+3
| | | | | | | | | | | | | | If we use dhclient without creating a complete network configuration, systemd-resolved will stop working after a while, and this sometimes happens while we're still installing packages. Disable it, together with systemd-networkd, while taking care of removing the dhclient hook that prevents overriding /etc/resolv.conf. While at it, it looks like removing snapd and needrestart actually takes more time than keeping them: drop that line. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib: Wait for DHCPv4 before starting DHCPv6 client in two_guests testStefano Brivio2022-09-231-0/+1
| | | | | | | | | | I'm not sure why, but dhclient hangs otherwise. This reflects what we do in the passt_in_ns setup steps. Eventually, this whole block could go away if we let pasta configure this network namespace with --config-net. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/perf: Wait for neper servers in guest to be ready before starting clientStefano Brivio2022-09-232-0/+6
| | | | | | | | Starting tcp_rr, tcp_crr, udp_rr servers in the guest takes a bit longer than starting the corresponding clients on the host, and we end up starting clients before servers unless we add a delay there. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib: Wait for kernel to free up ports used by iperf3 before reusing themStefano Brivio2022-09-221-0/+2
| | | | | | | | If we start another server on the same port right away, we might fail to bind the port. A small delay appears to be needed -- I'm not entirely sure why at this point. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib: Run also iperf3 clients in background, revert to time-based waitStefano Brivio2022-09-221-3/+6
| | | | | | | | | | | | | | | Unfortunately, this partially counters recent efforts by David to speed up these tests, but it looks like iperf3 clients don't reliably terminate, in some rare cases I couldn't isolate yet. For the moment being, reintroduce the time-based wait approach, now using the configurable test duration, and terminate the servers at the end of it, in case they're stuck. There's no point in keeping the 'sleep 2' later, so drop that, and while at it, make sure that the stuck servers have time to flush the JSON output before we use it. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/perf: Disable periodic throughput reports to avoid vhost hangStefano Brivio2022-09-225-5/+5
| | | | | | | | | | | | | | | | | | It appears that if we run throughput tests with one-second periodic reports, the sending side of the vhost channel used for SSH-based command dispatch occasionally stops working altogether. I haven't investigated this further, all I see is that output is truncated at some point, and doesn't resume. If we use gzip compression (ssh -C) this happens less frequently, but it still happens, seemingly indicating the issue is probably related to vhost itself. Disable periodic reports in iperf3 clients. The -i options were actually redundant, so remove them from both test files as well as from test_iperf3(). Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib: Wait on iperf3 clients to be done, then send SIGINT to serversStefano Brivio2022-09-221-8/+8
| | | | | | | | | | | | | | | | An iperf3 client might fail to send the control message indicating the end of the test, if the kernel buffer doesn't accept it, and exit without having sent it, as the control socket is non-blocking. Should this happen, the server will just wait forever for this message, instead of terminating. Restore some of the behaviour that went away with the "test: Rewrite test_iperf3" patch: instead of waiting on servers to terminate, wait on the clients. When they are done, wait 2 seconds, and then send SIGINT to the servers, which make them still write out the JSON report before terminating. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib: Restore IFS while executing directives in def blocksStefano Brivio2022-09-221-1/+1
| | | | | | | If we don't, guest command dispatch will fail altogether, given that we use cat(1) on the enter file, which contains spaces. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* conf, tcp, udp: Arrays for ports need 2^16 values, not 2^16-8Stefano Brivio2022-09-223-5/+5
| | | | | | | | | | | | Reported by David but also by Coverity (CWE-119): In conf_ports: Out-of-bounds access to a buffer ...not in practice, because the allocation size is rounded up anyway, but not nice either. Reported-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tap: Check return value of accept4() before calling getsockopt()Stefano Brivio2022-09-221-2/+4
| | | | | | | | | | | | | | | | Reported by Coverity (CWE-119): Negative value used as argument to a function expecting a positive value (for example, size of buffer or allocation) and harmless, because getsockopt() would return -EBADF if the socket is -1, so we wouldn't print anything. Check if accept4() returns a valid socket before calling getsockopt() on it. Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
* test/perf: Switch performance test duration to 10 seconds instead of 30Stefano Brivio2022-09-224-4/+4
| | | | | | | | | It looks like the workaround for the virtio_net TX hang issue is working less reliably with the new command dispatch mechanism, I'm not sure why. Switch to 10 seconds, at least for the moment. Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
* test/perf: Always use /sbin/sysctl in tcp testStefano Brivio2022-09-222-6/+6
| | | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
* README: Update Availability and Try It sections with new packagesStefano Brivio2022-09-221-25/+32
| | | | | | | | | We now have official packages for Fedora, unofficial (Fedora Copr) for other common RPM-based distributions, and the existing packages with static builds for Debian, and for other RPM-based distributions. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/passt_in_ns: Consistent sleep commands before starting socat clientStefano Brivio2022-09-222-6/+41
| | | | | | | | | | | | | | | There are some 'sleep 1' commands between starting the socat server and its corresponding client to avoid races due to the server not being ready as we start sending data. However, those don't cover all the cases where we might need them, and in some cases the sleep command actually ended up being before the server even starts. This fixes occasional failures in TCP and UDP simple transfer tests, that became apparent with the new command dispatch mechanism. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/perf: Check for /sbin/sysctl with which(1), not simply sysctlStefano Brivio2022-09-223-4/+4
| | | | | | | | | | Otherwise, we're depending on having /sbin in $PATH. For some reason I didn't completely grasp, with the new command dispatch mechanism that's not the case anymore, even if I have /sbin in $PATH in the parent shell. Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
* doc/demo: Clone and use mbuto in init namespaceStefano Brivio2022-09-221-5/+17
| | | | | | | | ...and not in pasta's namespace: the fakeroot(1) version shipping with (at least) Fedora 36 assumes "nested" operation as it sees that the UID is 0, and claims it's not supported. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* doc/demo: Drop /sbin from dhclient command, pass script file explicitlyStefano Brivio2022-09-221-4/+4
| | | | | | | | | | | dhclient might be in /usr/sbin on recent versions of Fedora, and mbuto just adds it to the same location as it was on the host: just call dhclient instead of /sbin/dhclient. This also applies for dhclient-script: given that we create the file on boot, pass its explicit location with -sf. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* Makefile: Include seccomp.h in HEADERS and require it for static checkersStefano Brivio2022-09-221-3/+3
| | | | | | | | Targets running static checkers (cppcheck and clang-tidy) need seccomp.h, but the latter is not included in HEADERS. Add it. Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
* Makefile: Allow define overrides by prepending, not appending, CFLAGSStefano Brivio2022-09-221-24/+25
| | | | | | | | | | | | | | | | | | | If we append CFLAGS to the ones passed via command line (if any), -D options we append will override -D options passed on command line (if any). For example, OpenSUSE build flags include -D_FORTIFY_SOURCE=3, and we want to have -D_FORTIFY_SOURCE=2, if and only if not overridden. The current behaviour implies we redefine _FORTIFY_SOURCE as 2, though. Instead of appending CFLAGS, prepend them by adding all the default build flags to another variable, a simply expanded one (defined with :=), named FLAGS, and pass that *before* CFLAGS in targets, so that defines from command line can override default flags. Reported-by: Dario Faggioli <dfaggioli@suse.com> Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Tested-by: Dario Faggioli <dfaggioli@suse.com>
* test: term: When checking if status line is a number, hide errorsStefano Brivio2022-09-141-1/+1
| | | | | | | | | | We use the [ "$x" -eq "$x" ] syntax to check if $x is a number. The behaviour is clearly implied by POSIX, but some shells might actually report the (intended) error, and dash floods script.log with "Illegal number" error messages. Hide them. Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
* test: Simpler termination handling for UDP testsDavid Gibson2022-09-133-65/+62
| | | | | | | | | | | | | Because UDP is connectionless we don't have an in-built end-of-stream signal for our connectivity tests. We work around this by explicitly adding an end marker to our sample data and killing the listening end once it is seen. However, socat has some built-in options - null-eof and shut-null - which can be used to signal the end of stream with a zero-length UDP packet. Use these to simplify how the UDP tests are implemented. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* udp: Don't drop zero-length outbound UDP packetsDavid Gibson2022-09-131-7/+10
| | | | | | | | | | | | | udp_tap_handler() currently skips outbound packets if they have a payload length of zero. This is not correct, since in a datagram protocol zero length packets still have meaning. Adjust this to correctly forward the zero-length packets by using a msghdr with msg_iovlen == 0. Bugzilla: https://bugs.passt.top/show_bug.cgi?id=19 Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* udp: Don't pre-initialize msghdr arrayDavid Gibson2022-09-131-1/+5
| | | | | | | | | | | | In udp_tap_handler() the array of msghdr structures, mm[], is initialized to zero. Since UIO_MAXIOV is 1024, this can be quite a large zero, which is expensive if we only end up using a few of its entries. It also makes it less obvious how we're setting all the control fields at the point we actually invoke sendmmsg(). Rather than pre-initializing it, just initialize each element as we use it. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move perf.js report file to $LOGDIR/webDavid Gibson2022-09-133-4/+4
| | | | | | | | | The tests generate a performance report in $BASEPATH/perf.js and hooks/pre-push copies it to the website. To avoid cluttering the working directory, instead put perf.js in $LOGDIR/web, since it's a test output artefact. Update hooks/pre-push to copy from its new location. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move video processing files to $STATEBASEDavid Gibson2022-09-134-30/+31
| | | | | | | | | | | | | The asciinema video handling creates a number of temporary files (.uncat, .start, .stop) which currently go into the source tree. Put them in the temporary state directory to avoid clutter. The final processed output is now placed into test_logs/web/ along with the corresponding .js file with links, since they're essentially test artefacts. hooks/pre-push is updated to look for those files in the new location when updating the web site. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* demo: Move pidfiles to state directoryDavid Gibson2022-09-132-7/+8
| | | | | | | Avoiding putting them in bare /tmp means they will be automatically cleaned up with everything else. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move pidfiles and nsholder sockets into state directoryDavid Gibson2022-09-134-36/+35
| | | | | | | Currently they go in the passt source tree with a fixed names, which means their presence can mess with subsequent test runs. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Store pcap files in $LOGDIR instead of /tmpDavid Gibson2022-09-131-8/+8
| | | | | | | The capture files are more or less a different form of log output from the tests, so place them in $LOGDIR. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move pause temporary file to state directoryDavid Gibson2022-09-131-2/+2
| | | | Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use paths in __STATEDIR__ instead of 'temp' and 'tempdir' directivesDavid Gibson2022-09-1315-82/+69
| | | | | | | | | | | | Instead of using the 'temp' and 'tempdir' DSL directives to create temporary files, use fixed paths relative to __STATEDIR__. This has two advantages: 1) The files are automatically cleaned up if the tests fail (and even if that doesn't work they're easier to clean up manuall) 2) When debugging tests it's easier to figure out which of the temporary files are relevant to whatever's going wrong Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Don't redundantly regenerate small test file in pasta/tcpDavid Gibson2022-09-131-3/+0
| | | | | | | In what looks like a copy/paste error, pasta/tcp generates its small test file twice. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move context temporary files to state dirDavid Gibson2022-09-131-29/+32
| | | | | | | | | Currently the context command dispatch subsystem creates a bunch of temporary files in $LOGDIR, which is messy. Store them in $STATEDIR which is for precisely this purpose. The logs from each context still go into $LOGDIR. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move passt_test_log_pipe to state directoryDavid Gibson2022-09-133-19/+16
| | | | | | | We use this fifo to send messages to the information pane. Put it in the state directory so it doesn't need its own cleanup. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Create common state directories for temporary filesDavid Gibson2022-09-134-1/+9
| | | | | | | | | | | | | | | | | | The test scripts create a bunch of temporary files to keep track of internal state. Some are made in /tmp with individual mktemp calls, some go in the passt source directory, and some go in $LOGDIR. This can sometimes make it messy to clean up after failed test runs. Start cleaning this up by creating a single "state" directory ($STATEBASE) in /tmp for all the state or temporary files used by a single test run. Clean it up automatically in cleanup() - except when DEBUG==1, because those files can be useful for debugging test script failures. We create subdirectories under $STATEBASE for each setup function, exposed as $STATESETUP. We also create subdirectories for each test script and expose those to the scripts as __STATEDIR__. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Actually run cleanup functionDavid Gibson2022-09-131-1/+0
| | | | | | | | | | | | | | We install a cleanup() function with 'trap' in order to clean up temporary files we generate during the tests. However, we deinstall it after run_term, which means it won't run in most of the cases where it would be useful. Even if "run from_term" exits with an error, that error will be hidden from the run_term wrapper because it's within a tmux session, so we will return from run_term normally, uninstall the trap and never clean up. In fact there's no reason to uninstall the trap at all, it works just as well on the success exit path as an error exit path. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Remove unused variable FFMPEG_PID_FILEDavid Gibson2022-09-131-1/+0
| | | | | | FFPMPEG_PID_FILE is set (creating a temporary file), then never used. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Group tests by mode then protocol, rather than the reverseDavid Gibson2022-09-1314-32/+32
| | | | | | | | For example, passt/dhcp rather than dhcp/passt. This is more consistent with the two_guests and other test groups, and makes some other cleanups simpler. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use new-style command issue for passt_in_ns testsDavid Gibson2022-09-135-52/+37
| | | | | | | Put the pieces together to use the new style context based dispatch for the passt_in_pasta tests. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use context system for two_guests testsDavid Gibson2022-09-132-73/+57
| | | | | | | | Now that we have all the pieces we need for issuing commands both into namespaces and into guests, we can use those to convert the two_guests to using only the new style context command issue. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use context system for guest commandsDavid Gibson2022-09-1310-16/+90
| | | | | | | | | | | | | | | Extends the context system in the test scripts to allow executing commands within a guest. Do this without requiring an existing network in the guest by using socat to run ssh via a vsock connection. We do need some additional "sleep"s in the tests, because the new faster dispatch means that sometimes we attempt to connect before socat has managed to listen. For now, only use this for the plain "passt" tests. The "passt_in_ns" and other tests have additional complications we still need to deal with. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Extend context system to run commands in namespace for pasta testsDavid Gibson2022-09-133-13/+29
| | | | | | | Extend the context system to allow commands to be run in a namespace created with unshare, and use it for the namespace used in the pasta tests. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Add nsholder utilityDavid Gibson2022-09-133-1/+147
| | | | | | | | | | | | | | | | | | | | | | | | | | | In our test scripts we need to do some ugly parsing of /proc and/or pstree output in order to get the PIDs of processes running in namespaces so that we can connect to those namespaces with nsenter or pasta. This is actually a pretty tricky problem with standard tools. To determine the PID from the outside of the namespace we need to know how the process of interest is related to the unshare or pasta process (child? one of several children? grandchild?) as well as then parsing /proc or ps output. This is slightly awkward now, and will get worse with future changes I'd like to make to have processes are dispatched. The obvious solution would be to have the process of interest (which we control) report its own PID, but that doesn't work easily, because it is in a PID namepace and sees only its local PID not the global PID we need to address it from outside. To handle this, add a small custom tool, "nsholder". This takes a path and a mode parameter. In "hold" mode it will create a unix domain socket bound to the path and listening. In "pid" mode it will get the "hold"ing process's pid via the unix socket using SO_PEERCRED, which translates between PID namespaces. In "stop" mode it will send a message to the socket causing the "hold"ing process to clean up and exit. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use new-style contexts for passt pane in the pasta and passt testsDavid Gibson2022-09-133-19/+12
| | | | | | | | | | Convert the pasta and passt tests to use new-style context execution for the things that run in the "passt" frame. Don't touch the passt_in_ns or two_guests tests yet, because they run passt inside a namespace which introduces some additional complications we have yet to handle. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Issue host commands via context for most testsDavid Gibson2022-09-132-25/+42
| | | | | | | | | Convert most of the tests to use the new-style system for issuing commands for all host commands. We leave the distro tests for now: they use the same pane for both host and guest commands which we'll need some more things to deal with. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Integration of old-style pane execution and new context executionDavid Gibson2022-09-133-81/+121
| | | | | | | | | | | | | | | | We're creating a system for tests to more reliably execute commands in various contexts (e.g. host, guest, namespace). That transition is going to happen over a number of steps though, so in the meantime we need to deal with both the old-style issuing of commands via typing into and screen scraping tmux panels, and the new-style system for executing commands in context. Introduce some transitional helpers which will issue a command via context if the requested context is initialized, but will otherwise fall back to the old style tmux panel based method. Re-implement the various test DSL commands in terms of these new helpers. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Allow a tmux pane to watch commands executed in contextsDavid Gibson2022-09-131-0/+18
| | | | | | | | | | | | | | | | We're moving to a new way of the tests dispatching commands to running in contexts (host, guest, namespace, etc.). As we make this transition, though, we still want the user to be able to watch the commands running in a context, as they previously could from the commands issued in the pane. Add a helper to set up a pane to watch a context's log to allow this. In some cases we currently issue commands from several different logical contexts in the same pane, so allow a pane to watch several contexts at once. Also use tail's --retry option to allow starting the watch before we've initialized the context which will be useful in some cases. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Context execution helpersDavid Gibson2022-09-131-0/+80
| | | | | | | | | | | | | | | | | | For the tests, we need to run commands in various contexts: in the host, in a guest or in a namespace. Currently we do this by running each context in a tmux pane, and using tmux commands to type the commands into the relevant pane, then screen-scrape the output for the results if we need them. This is very fragile, because we have to make various assumptions to parse the output. Those can break if a shell doesn't have the prompt we expect, if the tmux pane is too small or in various other conditions. This starts some library functions for a new "context" system, that provides a common way to invoke commands in a given context, in a way that properly preserves stdout, stderr and the process return code. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Correctly match "background" with "wait" commandsDavid Gibson2022-09-134-8/+6
| | | | | | | | | | | | Our test DSL has a number of paired commands to run something in the background in a pane, then later to wait for it to complete. However, in some of the tests we have these mismatched - starting a command in one pane, then waiting for it in another. We appear to get away with this for some reason, but it's not correct and future changes make it cause more problems. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* Allow --userns when pasta spawns a commandDavid Gibson2022-09-132-7/+0
| | | | | | | | | | | | Currently --userns is only allowed when pasta is attaching to an existing netns or PID, and is prohibited when creating a new netns by spawning a command or shell. With the new handling of userns, this check isn't neccessary. I'm not sure if there's any use case for --userns with a spawned command, but it's strictly more flexible and requires zero extra code, so we might as well. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* Handle userns isolation and dropping root at the same timeDavid Gibson2022-09-138-83/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | passt/pasta can interact with user namespaces in a number of ways: 1) With --netns-only we'll remain in our original user namespace 2) With --userns or a PID option to pasta we'll join either the given user namespace or that of the PID 3) When pasta spawns a shell or command we'll start a new user namespace for the command and then join it 4) With passt we'll create a new user namespace when we sandbox() ourself However (3) and (4) turn out to have essentially the same effect. In both cases we create one new user namespace. The spawned command starts there, and passt/pasta itself will live there from sandbox() onwards. Because of this, we can simplify user namespace handling by moving the userns handling earlier, to the same point we drop root in the original namespace. Extend the drop_user() function to isolate_user() which does both. After switching UID and GID in the original userns, isolate_user() will either join or create the userns we require. When we spawn a command with pasta_start_ns()/pasta_setup_ns() we no longer need to create a userns, because we're already made one. sandbox() likewise no longer needs to create (or join) an userns because we're already in the one we need. We no longer need c->pasta_userns_fd, since the fd is only used locally in isolate_user(). Likewise we can replace c->netns_only with a local in conf(), since it's not used outside there. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-26 04:29:07 +0000