Udp batching - #8
Open
xl32 wants to merge 2 commits into
Open
Conversation
The ngtcp2 engine spent two syscalls on every datagram: one recvfrom() and one getsockname(), the latter inside the read loop even though the comment above it already noted the bound address cannot change while the engine lives. Read the local address once per pass, and add an optional recv_batch hook to quic_io that quic_io_udp implements with recvmmsg(), so a busy socket costs one syscall per batch of 16 rather than one per datagram. The hook is optional: left NULL on platforms without recvmmsg(), callers keep using recv(). The OpenSSL engine needs none of this, as it drives its own datagram BIO through BIO_recvmmsg() already. Sending is deliberately not batched. Both accumulating packets for sendmmsg() and coalescing them for UDP_SEGMENT/GSO need ngtcp2's write loop restructured to emit equal-sized packets per destination, which is not a change this diff can make safely, and the default engine could not use it. Also add H3SocketBufferSize. A UDP receive buffer at the OS default (commonly 208KB) overflows as soon as one connection runs at speed, and each dropped datagram costs a retransmit and a congestion-window cut. The OS caps what it grants, so the module asks, reports a capped or refused grant at info level, and never fails the socket over it. Removes a per-pass walk of the whole connection list whose two counters were never read.
The bound address cannot change while the engine lives, which is what the comment on the old recv helper already said, so asking the kernel for it on every event-loop pass was avoidable. Read it when the engine is created -- the socket is bound before that point and stays bound -- and keep it in the engine, which takes getsockname() off the pump path entirely rather than merely out of the per-datagram path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Read queued datagrams in one batch with recvmmsg() where the platform has
it, and read the socket's local address once per event-loop pass instead
of once per datagram, halving the syscalls the ngtcp2 engine spends on a
busy socket. The OpenSSL engine already batches inside its own datagram
BIO. Also dropped a per-pass walk of the connection list whose result was
never used.
Added H3SocketBufferSize, which asks for the QUIC socket send and receive
buffer size. A receive buffer left at the OS default overflows once a
single connection runs at speed, and each dropped datagram costs a
retransmit; the OS still caps what it grants, and a capped grant is
logged rather than fatal.