When a process inside a container sends a network packet out, a long bureaucratic chain kicks off inside the Linux kernel. To your code, the call looks instantaneous, but behind the scenes the byte array goes through memory allocation, buffer copying, routing tables, firewall rules, and connection tracking. On an ordinary server, this internal plumbing goes unnoticed. But when the NIC speed climbs to 100 or 200 Gbps and the packet rate hits tens of millions of packets per second, the standard kernel pipeline becomes the main infrastructure bottleneck.
That is the wall Loophole Labs principal engineer Alex Sørlie ran into while building the Architect live migration platform. When moving a running container across clouds, every packet has to be intercepted, modified, encapsulated in a tunnel, and redirected to the new node without dropping the connection. On Linux, XDP (eXpress Data Path, based on eBPF) exists to process traffic at line rate. The catch is that by kernel design, XDP works exclusively on ingress traffic and is unavailable for a container's egress packets. Yet is that limitation really insurmountable if you look at how virtual interfaces work?
Why Traffic Control Chokes at 21.1 Gbps and What a Kernel Buffer Costs
You are probably wondering: why force XDP onto egress when Linux already has a built-in mechanism for intercepting outgoing packets with eBPF—the Traffic Control (TC) subsystem? Kubernetes network plugins rely on TC hooks to filter egress traffic. We covered the basic packet path in a cluster in our breakdown of Kubernetes networking architecture from the Linux network stack to eBPF. Compared to iptables, which in the team's tests on a 200 Gbps link at MTU 1500 delivered only 15.6 Gbps with high latency jitter due to CPU contention, intercepting via TC looks like a step forward.
In practice, however, Traffic Control hits a hard ceiling: on a 200 Gbps testbed between two AWS EC2 instances, a TC program pushed only 21.1 Gbps on egress and 23 Gbps on ingress. For live-migrating busy nodes, that means losing nearly 90% of link capacity.
The reason comes down to when TC runs and what data it operates on. The hook fires late—after routing, the firewall, and the conntrack state table. Even worse is struct sk_buff (the kernel socket buffer). For every frame, the kernel allocates this heavyweight object in memory and copies metadata into it. Imagine a courier service where every letter is placed into a wooden crate before dispatch, plastered with forms for every sorting department, and only at the warehouse door are you asked for permission to ship it. At millions of packets per second, sk_buff allocations burn through CPU resources before the eBPF program even starts.
How XDP's Sprint Works in the NIC Driver and Why Egress Is Closed to It
Unlike a socket buffer, XDP acts like an instant bouncer at the NIC's doorstep. When a packet arrives off the wire, the adapter uses Direct Memory Access (DMA) to write the bytes into the RX ring buffer and raises an interrupt to the driver. The driver places a pointer to the packet into the RX queue—and right there, the XDP program runs. struct sk_buff does not exist yet, the Linux kernel does not even know a frame has arrived, and processing happens on raw bytes in zero-copy mode.
Why does this high-speed mechanism only work on ingress? After digging into the Linux kernel source, the engineers noticed a key detail: the kernel does not tag packets as "incoming" at all. The sole trigger for invoking XDP is the fact that a frame landed in an RX queue. When an application inside a container sends data out, the packet goes straight into a TX queue. No RX queue is involved, so the driver simply has no hook point to launch XDP.
Turning Traffic Around Through a veth Mirror Tunnel and the Cost of Bypassing the Kernel
Since the kernel defines ingress traffic purely by arrival in an RX queue, you need to make the container's outgoing packet pretend to be an incoming packet for the host. And the right tool is already built into every server running Docker, containerd, or Kubernetes.
Each container's network namespace is connected to the host via a veth (Virtual Ethernet) pair, acting like a pass-through pipe. One end (veth0-A) lives inside the container, and the other (veth1-A) sits on the host. When the container sends a packet into the TX queue of veth0-A, the driver simply moves the packet's memory pointer into the RX queue of its peer veth1-A.
From the host's perspective, the packet just arrived in an RX queue! Because the veth driver has supported native XDP since Linux kernel 4.19, we attach an XDP program to veth1-A, rewrite the headers, and use the XDP_REDIRECT verdict to drop the frame straight into the TX queue of the physical eth0 NIC, bypassing the Linux network stack.
On the very first testbed run, though, the packets vanished on the wire: a tcpdump capture on the switch showed it was silently dropping the frames. When you bypass the kernel network stack, you have to do its job manually—recomputing checksums after NAT IP rewrites and resolving the next-hop MAC address via ARP. To avoid recalculating the checksum over the entire buffer and to fill in the next node's MAC address yourself, you embed incremental checksum addition and a custom ARP table inside an eBPF map:
static __always_inline __u16 csum16_add(__u16 csum, __u16 addend) {
csum += addend;
return csum + (csum < addend);
}
struct arp_entry { __u8 mac[ETH_ALEN]; };
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__type(key, __be32);
__type(value, struct arp_entry);
__uint(max_entries, 65535);
} arp_table SEC(".maps");
csum = csum16_add(~tcp->check, ~old_ip_high);
csum = csum16_add(csum, ~old_ip_low);
csum = csum16_add(csum, new_ip_high);
csum = csum16_add(csum, new_ip_low);
tcp->check = ~csum;
struct arp_entry *entry = bpf_map_lookup_elem(&arp_table, &dest_ip);
if (entry) memcpy(eth->h_dest, entry->mac, ETH_ALEN);
Let's see how this code saves the packets at runtime. The csum16_add helper adds 16-bit words while folding the carry bit csum < addend. Instead of reading the entire packet, we take the inverted old checksum ~tcp->check, subtract the halves of the old IP (~old_ip_high, ~old_ip_low), add the new address, and write the inverted result back to tcp->check in a couple of CPU cycles. Meanwhile, the arp_table hash map (BPF_MAP_TYPE_HASH with 65,535 entries) maps a __be32 IP address to an arp_entry struct: bpf_map_lookup_elem looks up dest_ip, and memcpy copies the hardware address entry->mac into the frame's eth->h_dest header. The switch receives a valid Ethernet frame without touching the Linux kernel.
Four Modes Compared on a 200 Gbps Testbed and Limits in Kubernetes
To test the approach under real load, the authors ran iPerf3 inside standard containers using veth pairs between two AWS EC2 instances with 200 Gbps networking at MTU 1500. Only the mechanism forwarding packets from veth to the physical adapter changed:
| Container Egress Mode | Speed (MTU 1500) | What Happens Under the Hood |
|---|---|---|
iptables (PREROUTING) | 15.6 Gbps | Full pass through kernel, NAT, and conntrack; CPU contention and latency spikes |
| Traffic Control (TC egress) | 21.1 Gbps | Runs after struct sk_buff allocation and core stack subsystems |
Generic XDP on veth | Below 21.1 Gbps | Software fallback after sk_buff allocation without TC optimizations |
Native XDP on veth | 194 Gbps | Intercepted in RX queue before the kernel; 12.4× over iptables and 9.2× over TC |
The Generic XDP trap is telling: it turned out slower than Traffic Control because it runs in fallback mode after sk_buff has already been created, losing the benefits of early interception. Only enabling native driver mode (xdpdrv via ip link set dev veth1 xdpgeneric off && ip link set dev veth1 xdpdrv object xdp_egress.o section xdp) pushed throughput to 194 Gbps with minimal latency jitter.
For network-heavy microservices, live migration systems, and Docker environments (the team is preparing an open-source network plugin to replace the default Docker bridge), this offers a way to multiply network speed without kernel patches. In Kubernetes clusters, however, bypassing the stack completely with XDP_REDIRECT means packets disappear from conntrack's radar. While engineers in classic CNIs fight bloated session tables (as we covered in eliminating 80-second delays in Kubernetes caused by the conntrack table in Cilium CNI), moving to pure XDP means you have to handle connection state tracking and background ARP table updates yourself via a userspace daemon.
The takeaway is straightforward: XDP's ingress-only limitation was merely a byproduct of how drivers interact with the RX queue, and a standard veth pair lets you turn container egress into host ingress—accelerating packet transmission from 21.1 to 194 Gbps on a stock Linux kernel.


