passt/test/lib, branch 2025_01_21.4f2c8e7

test: Add tests for passt in vhost-user mode

2024-11-27T15:49:21+00:00

Run functional and performance tests for vhost-user mode as well. For
functional tests, we add passt_vu and passt_vu_in_ns as symbolic links
to their non-vhost-user counterparts, as no differences are intended
but we want to distinguish them in test logs.

For performance tests, instead, we add separate perf/passt_vu_tcp and
perf/passt_vu_udp files, as we need longer test duration, as well as
higher UDP sending bandwidths and larger TCP windows, to actually get
the highest throughput vhost-user mode offers.

For valgrind tests, vhost-user mode needs two extra system calls:
statx and readlink. Add them as EXTRA_SYSCALLS for the valgrind
target.

Signed-off-by: Stefano Brivio 
Signed-off-by: Laurent Vivier 
Reviewed-by: David Gibson

test: Pass TRACE from run_term() into ./run from_term

2024-10-10T03:25:19+00:00

Just like we do for PCAP, DEBUG and KERNEL. Otherwise, running tests
with TRACE=1 will not actually enable tracing output.

Signed-off-by: Stefano Brivio 
Reviewed-by: David Gibson

test/lib/term: Always use printf for messages with escape sequences

2024-10-10T03:25:00+00:00

...instead of echo: otherwise, bash won't handle escape sequences we
use to colour messages (and 'echo -e' is not specified by POSIX).

Signed-off-by: Stefano Brivio 
Reviewed-by: David Gibson

test: Kernel binary can now be passed via the KERNEL environmental variable

2024-10-02T12:50:34+00:00

This is quite useful at least for myself as I'm usually running tests
using a guest kernel that's not the same as the one on the host.

Signed-off-by: Stefano Brivio 
Reviewed-by: David Gibson

test/lib/setup: Transform i686 kernel architecture name into QEMU name (i386)

2024-08-21T10:01:48+00:00

It's qemu-system-i386, but uname -m reports i686. I didn't test i486
and i586.

Signed-off-by: Stefano Brivio 
Reviewed-by: David Gibson

conf: Allow address remapped to host to be configured

2024-08-21T10:00:35+00:00

Because the host and guest share the same IP address with passt/pasta, it's
not possible for the guest to directly address the host.  Therefore we
allow packets from the guest going to a special "NAT to host" address to be
redirected to the host, appearing there as though they have both source and
destination address of loopback.

Currently that special address is always the address of the default
gateway (or none).  That can be a problem if we want that gateway to be
addressable by the guest.  Therefore, allow the special "NAT to host"
address to be overridden on the command line with a new --map-host-loopback
option.

In order to exercise and test it, update the passt_in_ns and perf
tests to use this option and give different mapping addresses for the
two layers of the environment.

Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio

test: Speed up by cutting on eye candy and performance test duration

2024-08-15T07:13:15+00:00

We have a number of delays when we switch to new layouts that were
added to make the tests visually easier to follow, together with
blinking status bars. Shorten the delays and avoid blinking the
status bar if $FAST is set to 1 (no demo mode).

Shorten delays in busy loops to 10ms, instead of 100ms, and skip the
one-second fixed delay when we wait for the status of a command.

Cut the duration of throughput and latency tests to one second, down
from ten. Somewhat surprisingly, the results we get are rather
consistent, and not significantly different from what we'd get with
10 seconds.

This, together with Podman's commit 20f3e8909e3a ("test/system:
pasta_test_do add explicit port check"), cuts the time needed on my
setup for full test run from approximately 37 minutes to...:

  $ time ./run
  [exited]
  PASS: 165, FAIL: 0
  Log at /home/sbrivio/passt/test/test_logs/test.log

  real	15m34.253s
  user	0m0.011s
  sys	0m0.011s

Signed-off-by: Stefano Brivio 
Reviewed-by: David Gibson 
Tested-by: David Gibson

test/lib/perf_report: Fix highlight

2024-07-25T15:56:47+00:00

Signed-off-by: Stefano Brivio

test: iperf3 3.16 introduces multiple threads, drop our own implementation of that

2024-07-25T10:30:38+00:00

Starting from iperf3 version 3.16, -P / --parallel spawns multiple
clients as separate threads, instead of multiple streams serviced by
the same thread.

So we can drop our lib/test implementation to spawn several iperf3
client and server processes and finally simplify things quite a bit.

Adjust number of threads and UDP sending bandwidth to values that seem
to be more or less matching previous throughput tests on my setup.

Signed-off-by: Stefano Brivio 
Reviewed-by: David Gibson 
Tested-by: David Gibson

test: Make handling of shell prompts with escapes a little more reliable

2023-12-07T06:24:48+00:00

When using the old-style "pane" methods of executing commands during the
tests, we need to scan the shell output for prompts in order to tell when
commands have finished.  This is inherently unreliable because commands
could output things that look like prompts, and prompts might not look like
we expect them to.  The only way to really fix this is to use a better way
of dispatching commands, like the newer "context" system.

However, it's awkward to convert everything to "context" right at the
moment, so we're still relying on some tests that do work most of the time.
It is, however, particularly sensitive to fancy coloured prompts using
escape sequences.  Currently we try to handle this by stripping actual
ESC characters with tr, then looking for some common variants.

We can do a bit better: instead strip all escape sequences using sed before
looking for our prompt.  Or, at least, any one using [a-zA-Z] as the
terminating character. Strictly speaking ANSI escapes can be terminated by
any character in 0x40..0x7e, which isn't easily expressed in a regexp.
This should capture all common ones, though.

With this transformation we can simplify the list of patterns we then look
for as a prompt, removing some redundant variants.

Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio