aboutgitcodebugslistschat
path: root/test/lib/term
Commit message (Collapse)AuthorAgeFilesLines
* test: Speed up by cutting on eye candy and performance test durationStefano Brivio2024-08-151-16/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We have a number of delays when we switch to new layouts that were added to make the tests visually easier to follow, together with blinking status bars. Shorten the delays and avoid blinking the status bar if $FAST is set to 1 (no demo mode). Shorten delays in busy loops to 10ms, instead of 100ms, and skip the one-second fixed delay when we wait for the status of a command. Cut the duration of throughput and latency tests to one second, down from ten. Somewhat surprisingly, the results we get are rather consistent, and not significantly different from what we'd get with 10 seconds. This, together with Podman's commit 20f3e8909e3a ("test/system: pasta_test_do add explicit port check"), cuts the time needed on my setup for full test run from approximately 37 minutes to...: $ time ./run [exited] PASS: 165, FAIL: 0 Log at /home/sbrivio/passt/test/test_logs/test.log real 15m34.253s user 0m0.011s sys 0m0.011s Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Tested-by: David Gibson <david@gibson.dropbear.id.au>
* test: Make handling of shell prompts with escapes a little more reliableDavid Gibson2023-12-071-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | When using the old-style "pane" methods of executing commands during the tests, we need to scan the shell output for prompts in order to tell when commands have finished. This is inherently unreliable because commands could output things that look like prompts, and prompts might not look like we expect them to. The only way to really fix this is to use a better way of dispatching commands, like the newer "context" system. However, it's awkward to convert everything to "context" right at the moment, so we're still relying on some tests that do work most of the time. It is, however, particularly sensitive to fancy coloured prompts using escape sequences. Currently we try to handle this by stripping actual ESC characters with tr, then looking for some common variants. We can do a bit better: instead strip all escape sequences using sed before looking for our prompt. Or, at least, any one using [a-zA-Z] as the terminating character. Strictly speaking ANSI escapes can be terminated by any character in 0x40..0x7e, which isn't easily expressed in a regexp. This should capture all common ones, though. With this transformation we can simplify the list of patterns we then look for as a prompt, removing some redundant variants. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* passt: Relicense to GPL 2.0, or any later versionStefano Brivio2023-04-061-1/+1
| | | | | | | | | | | | | | | | | | | In practical terms, passt doesn't benefit from the additional protection offered by the AGPL over the GPL, because it's not suitable to be executed over a computer network. Further, restricting the distribution under the version 3 of the GPL wouldn't provide any practical advantage either, as long as the passt codebase is concerned, and might cause unnecessary compatibility dilemmas. Change licensing terms to the GNU General Public License Version 2, or any later version, with written permission from all current and past contributors, namely: myself, David Gibson, Laine Stump, Andrea Bolognani, Paul Holzinger, Richard W.M. Jones, Chris Kuhn, Florian Weimer, Giuseppe Scrivano, Stefan Hajnoczi, and Vasiliy Ulyanov. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib: Add "td" directive, handled by table_value()Stefano Brivio2022-11-041-0/+28
| | | | | | | This can be used for generic cell values with an arbitrary scale. Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
* test: Add rudimentary support to run selected tests onlyStefano Brivio2022-10-141-1/+1
| | | | | | | | | | | | | | | To keep this simple, only support tests that have corresponding setup and teardown functions implied by their path. For example: ./run passt/ndp will trigger the 'passt' setup and teardown functions. This is not really elegant, but it looks robust, and while David is considering proper alternatives, it should be quite useful. Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
* test: term: When checking if status line is a number, hide errorsStefano Brivio2022-09-141-1/+1
| | | | | | | | | | We use the [ "$x" -eq "$x" ] syntax to check if $x is a number. The behaviour is clearly implied by POSIX, but some shells might actually report the (intended) error, and dash floods script.log with "Illegal number" error messages. Hide them. Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move video processing files to $STATEBASEDavid Gibson2022-09-131-4/+4
| | | | | | | | | | | | | The asciinema video handling creates a number of temporary files (.uncat, .start, .stop) which currently go into the source tree. Put them in the temporary state directory to avoid clutter. The final processed output is now placed into test_logs/web/ along with the corresponding .js file with links, since they're essentially test artefacts. hooks/pre-push is updated to look for those files in the new location when updating the web site. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move pause temporary file to state directoryDavid Gibson2022-09-131-2/+2
| | | | Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Move passt_test_log_pipe to state directoryDavid Gibson2022-09-131-6/+6
| | | | | | | We use this fifo to send messages to the information pane. Put it in the state directory so it doesn't need its own cleanup. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Create common state directories for temporary filesDavid Gibson2022-09-131-1/+1
| | | | | | | | | | | | | | | | | | The test scripts create a bunch of temporary files to keep track of internal state. Some are made in /tmp with individual mktemp calls, some go in the passt source directory, and some go in $LOGDIR. This can sometimes make it messy to clean up after failed test runs. Start cleaning this up by creating a single "state" directory ($STATEBASE) in /tmp for all the state or temporary files used by a single test run. Clean it up automatically in cleanup() - except when DEBUG==1, because those files can be useful for debugging test script failures. We create subdirectories under $STATEBASE for each setup function, exposed as $STATESETUP. We also create subdirectories for each test script and expose those to the scripts as __STATEDIR__. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Use context system for guest commandsDavid Gibson2022-09-131-3/+6
| | | | | | | | | | | | | | | Extends the context system in the test scripts to allow executing commands within a guest. Do this without requiring an existing network in the guest by using socat to run ssh via a vsock connection. We do need some additional "sleep"s in the tests, because the new faster dispatch means that sometimes we attempt to connect before socat has managed to listen. For now, only use this for the plain "passt" tests. The "passt_in_ns" and other tests have additional complications we still need to deal with. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Integration of old-style pane execution and new context executionDavid Gibson2022-09-131-0/+63
| | | | | | | | | | | | | | | | We're creating a system for tests to more reliably execute commands in various contexts (e.g. host, guest, namespace). That transition is going to happen over a number of steps though, so in the meantime we need to deal with both the old-style issuing of commands via typing into and screen scraping tmux panels, and the new-style system for executing commands in context. Introduce some transitional helpers which will issue a command via context if the requested context is initialized, but will otherwise fall back to the old style tmux panel based method. Re-implement the various test DSL commands in terms of these new helpers. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Allow a tmux pane to watch commands executed in contextsDavid Gibson2022-09-131-0/+18
| | | | | | | | | | | | | | | | We're moving to a new way of the tests dispatching commands to running in contexts (host, guest, namespace, etc.). As we make this transition, though, we still want the user to be able to watch the commands running in a context, as they previously could from the commands issued in the pane. Add a helper to set up a pane to watch a context's log to allow this. In some cases we currently issue commands from several different logical contexts in the same pane, so allow a pane to watch several contexts at once. Also use tail's --retry option to allow starting the watch before we've initialized the context which will be useful in some cases. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test: Remove unused *_XTERM variablesDavid Gibson2022-08-201-4/+0
| | | | | | | The DEMO_XTERM and CI_XTERM variables defined in test/lib/term aren't used anywhere. Remove them. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Don't check exit code for every command in demo modeStefano Brivio2022-05-191-0/+3
| | | | | | | Having all those 'echo $?' is rather distracting in demos. Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Sefano Brivio <sbrivio@redhat.com>
* tests: Add pane_status command to check for success of issued commandsDavid Gibson2022-05-191-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | When we use pane_wait to wait for a command issued to a tmux pane to finish we have no idea whether the command succeeded or not. This means that the test scripts can keep running long after the point something vital has failed, making it difficult to work out what went wrong. Add a new pane_status command that checks for success of the issued command and use it in most places instead of pane_wait. We still need explicit pane_wait where we're gathering explicit output with pane_parse, because the way we check the status with 'echo $?' means we lose track of that output. Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: - instead of quitting the script, make a test fail if a command issued in a pane fails during a test, and loop until the status code is numeric in pane_status() as a hack to make it a bit more robust - retain usage of pane_wait() in iperf3 and teardown functions as we interrupt iperf3, passt, and pasta, so a non-zero exit code is expected - drop bogus ns_{1,2}_wait() calls in teardown_two_guests(), those functions were never implemented - use pane_status() for "guest" test directives too ] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Improve control character filtering in pane_parseDavid Gibson2022-05-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | pane_parse() attempts to grab the output from the last command issued into a tmux pane. It strips out control characters using tr, which in particular includes the final \r\n. However, this won't fully strip out terminal escape sequences. In particular this breaks if the shell in the pane is bash, with enable-bracketed-paste enabled in readline. That issues terminal sequences to enable and disable bracketed paste mode around every shell prompt. We can work around this because these escapes are followed by a \r (CR). More generally, it seems reasonable to assume that any terminal shenanigans followed by a CR, but not an LF is supposed to be hidden. So, use sed to strip everything before the second last CR. We still need the tr to remove the final \r\n from the string (sed processes a line at a time, and doesn't consider the CRLF part of the buffer it's processing). Signed-off-by: David Gibson <david@gibson.dropbear.id.au> [sbrivio: modify regexp to keep foo\r\r\n unchanged, by matching on at least one CR and a non-CR afterwards: that's the usual output pattern for bash on Debian 8 and Debian 9] Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* tests: Don't globally set tmux default-shellDavid Gibson2022-05-191-6/+7
| | | | | | | | | | | | | | | | | run_term() uses tmux set-option -g to globally set the default shell. Unfortunately this hits a chicken-and-egg problem that's common with many of tmux's session options. If there isn't already a tmux server running, we can't connect to set the option. If we attempt this after starting our session (and therefore the server), then the session will already be started with the previous default shell. In any case it's not a good idea to set tmux global options, since that might interfere with whatever else the user is doing in tmux. So, instead set the default-shell option locally to the session after starting it. To make sure we get the right shell for our initial script, explicitly invoke /bin/sh to interpret it. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* tests: Don't use tmux update-environmentDavid Gibson2022-05-191-4/+5
| | | | | | | | | | | | | | | | | The semantics of tmux's update-environment option are a bit confusing. It says it means the given variables are copied into the session environment from the source environment, but it's not entirely clear what the "source" environment means. From my experimentation it appeast to be the environment from which the tmux *server* is launched, not the one issuing the 'new-session' command. That makes it pretty much useles, certainly in our case where we have no way of knowing if the user has pre-existing tmux sessions. Instead use the new-session -e option to explicitly pass in the variables we want to propagate. Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
* test/lib: Add small delay before trying to parse outputStefano Brivio2022-04-071-0/+1
| | | | | | | Don't fetch the log too early, we might get output from previous commands. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/term: Don't run demo when started as ./runStefano Brivio2022-02-281-1/+1
| | | | | | | I changed this in a previous commit by mistake, restore the original command. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* demo, ci: Switch to asciinema(1) for terminal recordingsStefano Brivio2022-02-221-19/+8
| | | | | | | | | | For demos, cool-retro-term(1) looked fancier, but several threads of that and ffmpeg(1) are just messing up with performance testing. The CI videos started getting really big as well, and they were difficult to read. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test: Add demo for Podman with pastaStefano Brivio2022-02-221-0/+10
| | | | | | | | ...showing setup steps, some peculiarities as --net option, and a general side-to-side comparison with slirp4netns(1), including "quick" TCP and UDP throughput and latency benchmarks. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/term: Allow for a wider variety of prompt characters in pane_wait()Stefano Brivio2022-01-281-7/+10
| | | | | | | | We might have highlighting and slightly different prompts across different distributions, allow a more reasonable set of prompt strings to be accepted as prompts. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test/lib/term: Export PCAP and DEBUG variables for tmux sessions globallyStefano Brivio2021-10-051-0/+1
| | | | | | Otherwise, this would depend on the local tmux configuration. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test: Record CI and demo videos in Xvfb by default, fix demo setup sequenceStefano Brivio2021-09-291-2/+11
| | | | Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
* test: Add CI/demo scriptsStefano Brivio2021-09-271-0/+610
Not really quick, definitely dirty. Signed-off-by: Stefano Brivio <sbrivio@redhat.com>