passt/test/lib, branch 2024_08_21.1d6142f

test/lib/setup: Transform i686 kernel architecture name into QEMU name (i386)

2024-08-21T10:01:48+00:00

It's qemu-system-i386, but uname -m reports i686. I didn't test i486
and i586.

Signed-off-by: Stefano Brivio 
Reviewed-by: David Gibson

conf: Allow address remapped to host to be configured

2024-08-21T10:00:35+00:00

Because the host and guest share the same IP address with passt/pasta, it's
not possible for the guest to directly address the host.  Therefore we
allow packets from the guest going to a special "NAT to host" address to be
redirected to the host, appearing there as though they have both source and
destination address of loopback.

Currently that special address is always the address of the default
gateway (or none).  That can be a problem if we want that gateway to be
addressable by the guest.  Therefore, allow the special "NAT to host"
address to be overridden on the command line with a new --map-host-loopback
option.

In order to exercise and test it, update the passt_in_ns and perf
tests to use this option and give different mapping addresses for the
two layers of the environment.

Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio

test: Speed up by cutting on eye candy and performance test duration

2024-08-15T07:13:15+00:00

We have a number of delays when we switch to new layouts that were
added to make the tests visually easier to follow, together with
blinking status bars. Shorten the delays and avoid blinking the
status bar if $FAST is set to 1 (no demo mode).

Shorten delays in busy loops to 10ms, instead of 100ms, and skip the
one-second fixed delay when we wait for the status of a command.

Cut the duration of throughput and latency tests to one second, down
from ten. Somewhat surprisingly, the results we get are rather
consistent, and not significantly different from what we'd get with
10 seconds.

This, together with Podman's commit 20f3e8909e3a ("test/system:
pasta_test_do add explicit port check"), cuts the time needed on my
setup for full test run from approximately 37 minutes to...:

  $ time ./run
  [exited]
  PASS: 165, FAIL: 0
  Log at /home/sbrivio/passt/test/test_logs/test.log

  real	15m34.253s
  user	0m0.011s
  sys	0m0.011s

Signed-off-by: Stefano Brivio 
Reviewed-by: David Gibson 
Tested-by: David Gibson

test/lib/perf_report: Fix highlight

2024-07-25T15:56:47+00:00

Signed-off-by: Stefano Brivio

test: iperf3 3.16 introduces multiple threads, drop our own implementation of that

2024-07-25T10:30:38+00:00

Starting from iperf3 version 3.16, -P / --parallel spawns multiple
clients as separate threads, instead of multiple streams serviced by
the same thread.

So we can drop our lib/test implementation to spawn several iperf3
client and server processes and finally simplify things quite a bit.

Adjust number of threads and UDP sending bandwidth to values that seem
to be more or less matching previous throughput tests on my setup.

Signed-off-by: Stefano Brivio 
Reviewed-by: David Gibson 
Tested-by: David Gibson

test: Make handling of shell prompts with escapes a little more reliable

2023-12-07T06:24:48+00:00

When using the old-style "pane" methods of executing commands during the
tests, we need to scan the shell output for prompts in order to tell when
commands have finished.  This is inherently unreliable because commands
could output things that look like prompts, and prompts might not look like
we expect them to.  The only way to really fix this is to use a better way
of dispatching commands, like the newer "context" system.

However, it's awkward to convert everything to "context" right at the
moment, so we're still relying on some tests that do work most of the time.
It is, however, particularly sensitive to fancy coloured prompts using
escape sequences.  Currently we try to handle this by stripping actual
ESC characters with tr, then looking for some common variants.

We can do a bit better: instead strip all escape sequences using sed before
looking for our prompt.  Or, at least, any one using [a-zA-Z] as the
terminating character. Strictly speaking ANSI escapes can be terminated by
any character in 0x40..0x7e, which isn't easily expressed in a regexp.
This should capture all common ones, though.

With this transformation we can simplify the list of patterns we then look
for as a prompt, removing some redundant variants.

Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio

test/lib/perf_report: Fix up table highlight for pasta's local flows

2023-11-10T05:35:54+00:00

As commit 29269705239f ("test/perf: Small MTUs for spliced TCP
aren't interesting") drops all columns for TCP test MTUs except for
one, in throughput test for pasta's local flows, the first column we
need to highlight in that table is now the second one.

Signed-off-by: Stefano Brivio

test/perf: Start iperf3 server less often

2023-11-07T08:56:10+00:00

Currently we start both the iperf3 server(s) and client(s) afresh each time
we want to make a bandwidth measurement.  That's not really necessary as
usually a whole batch of bandwidth measurements can use the same server.

Split up the iperf3 directive into 3 directives: iperf3s to start the
server, iperf3 to make a measurement and iperf3k to kill the server, so
that we can start the server less often.  This - and more importantly, the
reduced number of waits for the server to be ready - reduces runtime of the
performance tests on my laptop by about 4m (out of ~28minutes).

For now we still restart the server between IPv4 and IPv6 tests.  That's
because in some cases the latency measurements we make in between use the
same ports.

Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio

test/perf: Get iperf3 stats from client side

2023-11-07T08:56:06+00:00

iperf3 generates statistics about its run on both the client and server
sides.  They don't have exactly the same information, but both have the
pieces we need (AFAICT the server communicates some nformation to the
client over the control socket, so the most important information is in the
client side output, even if measured by the server).

Currently we use the server side information for our measurements. Using
the client side information has several advantages though:

 * We can directly wait for the client to complete and we know we'll have
   the output we want.  We don't need to sleep to give the server time to
   write out the results.
 * That in turn means we can wrap up as soon as the client is done, we
   don't need to wait overlong to make sure everything is finished.
 * The slightly different organisation of the data in the client output
   means that we always want the same json value, rather than requiring
   slightly different onces for UDP and TCP.

The fact that we avoid some extra delays speeds up the overal run of the
perf tests by around 7 minutes (out of around 35 minutes) on my laptop.

The fact that we no longer unconditionally kill client and server after
a certain time means that the client could run indefinitely if the server
doesn't respond.  We mitigate that by setting 1s connect timeout on the
client.  This isn't foolproof - if we get an initial response, but then
lose connectivity this could still run indefinitely, however it does cover
by far the most likely failure cases.  --snd-timeout would provide more
robustness, but I've hit odd failures when trying to use it.

Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio

test: Use "nstool exec" to slightly simplify tests

2023-04-07T23:12:08+00:00

Using this, rather than using "nstool info" to get the pid then manually
connecting with nsenter makes things a little simpler.

Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio