passt/tcp_splice.c, branch 2026_05_26.038c51e

tcp_splice: Simplify tracking of read/written bytes

2026-05-26T10:21:49+00:00

For each each direction of each spliced connection, we keep track of how
many bytes we've read from one socket and written to the other.  However,
we never actually care about the absolute values of these, only the
difference between them, which represents how much data is currently "in
flight" in the splicing pipe.

Simplify the handling by having a single variable tracking the number of
bytes in the pipe.

As a bonus, the new scheme makes it clearer that we don't need to worry
about overflows: pending can never become larger than the maximum pipe
bufffer size, well within 32-bits.

I _think_ the old scheme was safe in the case of overflow - again under
the assumption that read/written can never be further apart than the pipe
buffer size.  However, it's much harder to reason about this case.  It's
certainly plausible that an overflow could occur - sending 4GiB through
a local socket is entirely achievable.

Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio

tcp_splice: Clean up flow control path for splice forwarding

2026-05-26T10:21:45+00:00

Splice forwarding can be blocked either waiting for data from one side
or waiting for space on the other.  For that reason,
tcp_splice_sock_handler() on either socket can forward data in either or
both directions, depending on whether we have EPOLLIN, EPOLLOUT or both
events.

The flow control for this is quite hard to follow though, since we forward
in one direction, then sometimes loop back with a goto to do it in the
other direction.  Simplify this by adding a tcp_splice_forward() function
with the logic to forward in one direction and calling it either once or
twice from tcp_splice_sock_handler().

Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio

tcp_splice: Avoid missing EOF recognition while forwarding

2026-05-26T10:21:41+00:00

tcp_splice_sock_handler() has an optimised path for the common case where
the amount we splice(2) into the pipe is exactly the same as the amount we
splice(2) out again.  If the pipe is empty at that point, we stop
forwarding until we get another epoll event.

However, via a subtle chain of events, this can cause a bug for a
half-closed connection.  Suppose the connection is already half-closed in
the other direction - that is, we've already called shutdown(SHUT_WR) on
the socket for which we're getting the event.  In this event we're getting
the last batch of data in the other direction, and also a FIN.  This can
result in EPOLLIN, EPOLLRDHUP and EPOLLHUP events simultaneously.

We read the last data from the socket and successfully splice it to the
other side.  Since there is no data in the pipe, we exit the forwarding
loop.  However, because we did read data, we don't set the eof flag.

Because we don't set eof, we don't (yet) propagate the FIN to the other
side, or set FIN_SENT_(!fromsidei).  Therefore we don't (yet) recognize
this as a clean termination and set the CLOSING flag.  We would correct
this when we get our next event, however before we can do so we process
the EPOLLHUP event.  Because we haven't recognized this as a clean close
we assume it is an abrupt close and send an RST to the other side.

To avoid this, don't stop attempting to forward data on this path.
Continue for at least one more loop.  If we're at EOF, we'll recognize it
on the next splice(2).  If not it gives us an opportunity to forward more
data without returning to the mail epoll loop.

Reported-by: Paul Holzinger 
Link: https://bugs.passt.top/show_bug.cgi?id=202
Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio

tcp_splice: Improve error reporting

2026-05-26T10:21:11+00:00

A number of things can, at least theoretically, go wrong when forwarding
data across a spliced connection.  We generally handle this by resetting
the connection on both sides.  However, in many cases we don't log any
message about why the connection was reset, which can make it hard to
debug why this is happening.

Add a bunch of debug and error logging to make this easier to figure out.
We ratelimit for safety, which requires some tweaks to make the ratelimit
logic work with the flow specific log functions.

Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio

treewide: Spell ASSERT() as assert()

2026-03-20T20:05:29+00:00

The standard library assert(3), at least with glibc, hits our seccomp
filter and dies with SIGSYS before it's able to print a message, making it
near useless.  Therefore, since 7a8ed9459dfe ("Make assertions actually
useful") we've instead used our own implementation, named ASSERT().

This makes our code look slightly odd though - ASSERT() has the same
overall effect as assert(), it's just a different implementation.  More
importantly this makes it awkward to share code between passt/pasta proper
and things that compile in a more typical environment.  We're going to want
that for our upcoming dynamic configuration tool.

Address this by overriding the standard library's assert() implementation
with our own, instead of giving ours its own name.

The standard assert() is supposed to be omitted if NDEBUG is defined,
which ours doesn't do.  Implement that as well, so ours doesn't
unexpectedly differ.  For the -DNDEBUG case we do this by *not* overriding
assert(), since it will be a no-op anyway.  This requires a few places to
add a #include  to let us compile (albeit with warnings) when
-DNDEBUG.

Signed-off-by: David Gibson 
[sbrivio: Fix some conflicts and missing conversions as a result of
 applying "vu_common: Move iovec management into vu_collect()" first]
Signed-off-by: Stefano Brivio

tcp, tcp_splice: Check for failures of shutdown(2)

2026-01-31T02:56:57+00:00

shutdown(2) should never fail, unless we give it bad parameters (e.g.
passing it an fd which isn't a connected socket).  However, if it ever did,
we'd currently ignore the error and carry on which could lead to very
confusing behaviour.

In the interests of debugability, check for failure of shutdown(2), log an
error and:
 - during runtime, reset the affected connection
 - during migration, fail the migration

Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio

tcp_splice: Force TCP RST on abnormal close conditions

2026-01-27T11:40:29+00:00

When we need to prematurely close a spliced connection, we use:
    conn_flag(conn, CLOSING);
This does destroy the flow, but does so in the same way as a clean close
from both ends.  That's not what we want in error conditions, or when one
side of the flow has signalled an abnormal exit with an EPOLLHUP event.

Replace all places where we close the connection - except for the happy
path close - with calls to a new tcp_splice_rst() function, which forces
the sockets to emit a TCP RST on each side.

Link: https://bugs.passt.top/show_bug.cgi?id=193
Signed-off-by: David Gibson 
Signed-off-by: Stefano Brivio

tcp_splice: Register fds with epoll at flow creation

2026-01-20T18:37:39+00:00

Register both splice connection sockets with epoll using empty events
(events=0) in tcp_splice_connect(), before initiating the connection.

This allows tcp_splice_epoll_ctl() to always use EPOLL_CTL_MOD, removing
the need to check whether fds are already registered. As a result, the
conditional ADD/MOD logic is no longer needed, simplifying the function.

If the second flow_epoll_set() fails after the first succeeds, we don't
need explicit rollback: tcp_splice_conn_from_sock() sets the CLOSING
flag on error, and conn_flag() handles it by calling epoll_del() for
both sockets.

Signed-off-by: Laurent Vivier 
Reviewed-by: David Gibson 
Signed-off-by: Stefano Brivio

flow: Introduce flow_epoll_set() to centralize epoll operations

2026-01-14T00:07:51+00:00

Currently, each flow type (TCP, TCP_SPLICE, PING, UDP) has its own
code to add or modify file descriptors in epoll. This leads to
duplicated boilerplate code across icmp.c, tcp.c, tcp_splice.c, and
udp_flow.c, each setting up epoll_ref unions and calling epoll_ctl()
with flow-type-specific details.

Introduce flow_epoll_set() in flow.c to handle epoll operations for
all flow types in a unified way.

This will be needed to migrate queue pair from an epollfd to another.

Signed-off-by: Laurent Vivier 
Reviewed-by: David Gibson 
Signed-off-by: Stefano Brivio

tcp_splice: Refactor tcp_splice_conn_epoll_events() to per-side computation

2026-01-14T00:07:51+00:00

The function tcp_splice_conn_epoll_events() currently takes an array of
struct epoll_event and fills in the .events field for both sides using
flow_foreach_sidei() loops.

This works, but the function is doing two conceptually separate things
at once: computing events for side 0 and computing events for side 1.
The OUT_WAIT handling is particularly subtle, as it has cross-side
effects: when OUT_WAIT(sidei) is set, we add EPOLLOUT to ev[sidei] but
also remove EPOLLIN from ev[!sidei].

Refactor to make the function compute events for a single side at a
time, taking sidei as a parameter and returning uint32_t. This makes
the logic more focused and easier to follow. The cross-side effects of
OUT_WAIT are preserved by checking both OUT_WAIT(sidei) and
OUT_WAIT(!sidei) within each call.

The caller tcp_splice_epoll_ctl() now invokes the function twice, once
for each side, making the two-sided nature of the operation explicit.

No functional change.

Signed-off-by: Laurent Vivier 
Reviewed-by: David Gibson 
Signed-off-by: Stefano Brivio