<feed xmlns='http://www.w3.org/2005/Atom'>
<title>passt/test/run, branch 2022_09_23.d6f865a</title>
<subtitle>Plug A Simple Socket Transport</subtitle>
<link rel='alternate' type='text/html' href='https://passt.top/passt/'/>
<entry>
<title>test: Move passt_test_log_pipe to state directory</title>
<updated>2022-09-13T09:12:41+00:00</updated>
<author>
<name>David Gibson</name>
<email>david@gibson.dropbear.id.au</email>
</author>
<published>2022-09-13T04:35:20+00:00</published>
<link rel='alternate' type='text/html' href='https://passt.top/passt/commit/?id=e4ecb6d795ef10d26a89e195c23a43bf524e5a41'/>
<id>e4ecb6d795ef10d26a89e195c23a43bf524e5a41</id>
<content type='text'>
We use this fifo to send messages to the information pane.  Put it in the
state directory so it doesn't need its own cleanup.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
We use this fifo to send messages to the information pane.  Put it in the
state directory so it doesn't need its own cleanup.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>test: Create common state directories for temporary files</title>
<updated>2022-09-13T09:12:41+00:00</updated>
<author>
<name>David Gibson</name>
<email>david@gibson.dropbear.id.au</email>
</author>
<published>2022-09-13T04:35:19+00:00</published>
<link rel='alternate' type='text/html' href='https://passt.top/passt/commit/?id=8cb7d5b461f0fc7c49fcb5cd6b0ce8dfcb15fce8'/>
<id>8cb7d5b461f0fc7c49fcb5cd6b0ce8dfcb15fce8</id>
<content type='text'>
The test scripts create a bunch of temporary files to keep track of
internal state.  Some are made in /tmp with individual mktemp calls, some
go in the passt source directory, and some go in $LOGDIR.  This can
sometimes make it messy to clean up after failed test runs.

Start cleaning this up by creating a single "state" directory ($STATEBASE)
in /tmp for all the state or temporary files used by a single test run.
Clean it up automatically in cleanup() - except when DEBUG==1, because
those files can be useful for debugging test script failures.

We create subdirectories under $STATEBASE for each setup function, exposed
as $STATESETUP.  We also create subdirectories for each test script and
expose those to the scripts as __STATEDIR__.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The test scripts create a bunch of temporary files to keep track of
internal state.  Some are made in /tmp with individual mktemp calls, some
go in the passt source directory, and some go in $LOGDIR.  This can
sometimes make it messy to clean up after failed test runs.

Start cleaning this up by creating a single "state" directory ($STATEBASE)
in /tmp for all the state or temporary files used by a single test run.
Clean it up automatically in cleanup() - except when DEBUG==1, because
those files can be useful for debugging test script failures.

We create subdirectories under $STATEBASE for each setup function, exposed
as $STATESETUP.  We also create subdirectories for each test script and
expose those to the scripts as __STATEDIR__.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>test: Actually run cleanup function</title>
<updated>2022-09-13T09:12:41+00:00</updated>
<author>
<name>David Gibson</name>
<email>david@gibson.dropbear.id.au</email>
</author>
<published>2022-09-13T04:35:18+00:00</published>
<link rel='alternate' type='text/html' href='https://passt.top/passt/commit/?id=1e4a15398f2f932e290ab222d2d57a47018f2245'/>
<id>1e4a15398f2f932e290ab222d2d57a47018f2245</id>
<content type='text'>
We install a cleanup() function with 'trap' in order to clean up temporary
files we generate during the tests.  However, we deinstall it after
run_term, which means it won't run in most of the cases where it would be
useful.  Even if "run from_term" exits with an error, that error will be
hidden from the run_term wrapper because it's within a tmux session, so we
will return from run_term normally, uninstall the trap and never clean up.

In fact there's no reason to uninstall the trap at all, it works just as
well on the success exit path as an error exit path.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
We install a cleanup() function with 'trap' in order to clean up temporary
files we generate during the tests.  However, we deinstall it after
run_term, which means it won't run in most of the cases where it would be
useful.  Even if "run from_term" exits with an error, that error will be
hidden from the run_term wrapper because it's within a tmux session, so we
will return from run_term normally, uninstall the trap and never clean up.

In fact there's no reason to uninstall the trap at all, it works just as
well on the success exit path as an error exit path.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>test: Group tests by mode then protocol, rather than the reverse</title>
<updated>2022-09-13T09:12:41+00:00</updated>
<author>
<name>David Gibson</name>
<email>david@gibson.dropbear.id.au</email>
</author>
<published>2022-09-13T04:35:16+00:00</published>
<link rel='alternate' type='text/html' href='https://passt.top/passt/commit/?id=594c2f93716380feb29aeb774ba77704684aa125'/>
<id>594c2f93716380feb29aeb774ba77704684aa125</id>
<content type='text'>
For example, passt/dhcp rather than dhcp/passt.  This is more
consistent with the two_guests and other test groups, and makes some
other cleanups simpler.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
For example, passt/dhcp rather than dhcp/passt.  This is more
consistent with the two_guests and other test groups, and makes some
other cleanups simpler.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>test: Integration of old-style pane execution and new context execution</title>
<updated>2022-09-13T03:32:00+00:00</updated>
<author>
<name>David Gibson</name>
<email>david@gibson.dropbear.id.au</email>
</author>
<published>2022-09-12T10:56:17+00:00</published>
<link rel='alternate' type='text/html' href='https://passt.top/passt/commit/?id=c2f248588b271f083c81f260c0818781e74e3c2d'/>
<id>c2f248588b271f083c81f260c0818781e74e3c2d</id>
<content type='text'>
We're creating a system for tests to more reliably execute commands in
various contexts (e.g. host, guest, namespace).  That transition is going
to happen over a number of steps though, so in the meantime we need to deal
with both the old-style issuing of commands via typing into and screen
scraping tmux panels, and the new-style system for executing commands in
context.

Introduce some transitional helpers which will issue a command via context
if the requested context is initialized, but will otherwise fall back to
the old style tmux panel based method.  Re-implement the various test DSL
commands in terms of these new helpers.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
We're creating a system for tests to more reliably execute commands in
various contexts (e.g. host, guest, namespace).  That transition is going
to happen over a number of steps though, so in the meantime we need to deal
with both the old-style issuing of commands via typing into and screen
scraping tmux panels, and the new-style system for executing commands in
context.

Introduce some transitional helpers which will issue a command via context
if the requested context is initialized, but will otherwise fall back to
the old style tmux panel based method.  Re-implement the various test DSL
commands in terms of these new helpers.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>test: Log debugging output from test script</title>
<updated>2022-08-20T17:07:12+00:00</updated>
<author>
<name>David Gibson</name>
<email>david@gibson.dropbear.id.au</email>
</author>
<published>2022-08-18T06:13:57+00:00</published>
<link rel='alternate' type='text/html' href='https://passt.top/passt/commit/?id=3fdb0747f36d63d6f1a2724181a56e10f7065f4b'/>
<id>3fdb0747f36d63d6f1a2724181a56e10f7065f4b</id>
<content type='text'>
The test scripts run with sh -e, which means they will stop if any commands
return an error.  That's generally desirable, because we won't continue
after things are hopeless due to an earlier step failing.

Unfortunately, the tmux setup we run the script in means it's not obvious
where any error messages related to such a failure will go.  Depending on
exactly where the error occurs they might go to the original terminal
hidden behind tmux, or they might go to a tmux panel that's not visible in
the normal layouts.

To make it easier to find such error message, redirect direct output and
errors from the test script itself to a 'script.log' file in the logs
directory.  When in DEBUG=1 mode, additionaly 'set -x' so we log all the
commands we execute to that file.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The test scripts run with sh -e, which means they will stop if any commands
return an error.  That's generally desirable, because we won't continue
after things are hopeless due to an earlier step failing.

Unfortunately, the tmux setup we run the script in means it's not obvious
where any error messages related to such a failure will go.  Depending on
exactly where the error occurs they might go to the original terminal
hidden behind tmux, or they might go to a tmux panel that's not visible in
the normal layouts.

To make it easier to find such error message, redirect direct output and
errors from the test script itself to a 'script.log' file in the logs
directory.  When in DEBUG=1 mode, additionaly 'set -x' so we log all the
commands we execute to that file.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>test: Use shutdown test for pasta</title>
<updated>2022-08-20T17:07:12+00:00</updated>
<author>
<name>David Gibson</name>
<email>david@gibson.dropbear.id.au</email>
</author>
<published>2022-08-18T06:13:56+00:00</published>
<link rel='alternate' type='text/html' href='https://passt.top/passt/commit/?id=69126d4d48002784562ba44a77df37387968061d'/>
<id>69126d4d48002784562ba44a77df37387968061d</id>
<content type='text'>
For the passt and passt_in_ns tests we have a "shutdown" testcase that
checks for any errors from the passt process we were using (including
valgrind warnings).  Do the same for pasta tests, so that we catch any
error codes from the pasta process.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
For the passt and passt_in_ns tests we have a "shutdown" testcase that
checks for any errors from the passt process we were using (including
valgrind warnings).  Do the same for pasta tests, so that we catch any
error codes from the pasta process.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>test: Rename slightly misleading "valgrind" tests</title>
<updated>2022-08-20T17:07:12+00:00</updated>
<author>
<name>David Gibson</name>
<email>david@gibson.dropbear.id.au</email>
</author>
<published>2022-08-18T06:13:55+00:00</published>
<link rel='alternate' type='text/html' href='https://passt.top/passt/commit/?id=9224af149483e448546db7a63b66992c6003ab3e'/>
<id>9224af149483e448546db7a63b66992c6003ab3e</id>
<content type='text'>
The "valgrind" test cases are designed to pick up errors reported when
passt is running under valgrind.  But what it actually does is just kill
the passt process, then see if it had a non-zero exit code.  That means it
will equally well pick up any other problems which caused passt to exit
with an error status: either something detected within passt or as a result
of passt being killed by an unexpected signal.

The fact that the "valgrind" test is actually responsible for shutting down
the passt process is non-obvious and can lead to problems when selectively
running tests during debugging.

Rename the "valgrind" tests to "shutdown" tests and run it regardless of
whether we're using valgrind or not.  This allows us to remove an ugly
speacial case in the passt_in_ns teardown code.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The "valgrind" test cases are designed to pick up errors reported when
passt is running under valgrind.  But what it actually does is just kill
the passt process, then see if it had a non-zero exit code.  That means it
will equally well pick up any other problems which caused passt to exit
with an error status: either something detected within passt or as a result
of passt being killed by an unexpected signal.

The fact that the "valgrind" test is actually responsible for shutting down
the passt process is non-obvious and can lead to problems when selectively
running tests during debugging.

Rename the "valgrind" tests to "shutdown" tests and run it regardless of
whether we're using valgrind or not.  This allows us to remove an ugly
speacial case in the passt_in_ns teardown code.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>test: Split setup/teardown functions for build and distro tests</title>
<updated>2022-08-20T17:07:12+00:00</updated>
<author>
<name>David Gibson</name>
<email>david@gibson.dropbear.id.au</email>
</author>
<published>2022-08-18T06:13:53+00:00</published>
<link rel='alternate' type='text/html' href='https://passt.top/passt/commit/?id=2fa308ac6e5723241dd433e1610395054f7b7b10'/>
<id>2fa308ac6e5723241dd433e1610395054f7b7b10</id>
<content type='text'>
Currently the build tests and distro tests share a common setup function.
That works for now, but changes we want to make will mean they need
slightly different setup, so split the setup functions in preparation.

Currently, neither build nor distro tests have any teardown function.
Again, future changes are going to mean we need to do some teardown, so
create some empty for now teardown functions in preparation.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Currently the build tests and distro tests share a common setup function.
That works for now, but changes we want to make will mean they need
slightly different setup, so split the setup functions in preparation.

Currently, neither build nor distro tests have any teardown function.
Again, future changes are going to mean we need to do some teardown, so
create some empty for now teardown functions in preparation.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>test: Split cppcheck and clang-tidy tests into different files</title>
<updated>2022-08-20T17:07:12+00:00</updated>
<author>
<name>David Gibson</name>
<email>david@gibson.dropbear.id.au</email>
</author>
<published>2022-08-18T06:13:50+00:00</published>
<link rel='alternate' type='text/html' href='https://passt.top/passt/commit/?id=c8756034b7a86e76040957403f2b67ceb4d2e2a6'/>
<id>c8756034b7a86e76040957403f2b67ceb4d2e2a6</id>
<content type='text'>
Both clang-tidy and cppcheck linting are handled by the same test file,
test/build/static_checkers.  The two linters are independent of each other
though, and each one takes quite a long time.  Split them into separate
files to make it easier to control which are executed from the top level
test script.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Both clang-tidy and cppcheck linting are handled by the same test file,
test/build/static_checkers.  The two linters are independent of each other
though, and each one takes quite a long time.  Split them into separate
files to make it easier to control which are executed from the top level
test script.

Signed-off-by: David Gibson &lt;david@gibson.dropbear.id.au&gt;
</pre>
</div>
</content>
</entry>
</feed>
