한국어English日本語简体中文繁體中文DeutschไทยTiếng ViệtРусскийPortuguês (Brasil)EspañolBahasa Indonesia

Game Lag White Paper › Root causes of TCP retransmission

Middlebox over capacity (firewall, IPS, DDoS protection) Inline appliance PPS / CPU overload

Cause ID rt-appliance-pps · Primary owner Infra team (Network infrastructure) · Also Game team (Server development)

Open the interactive card with figures and simulations →

Firewalls, intrusion prevention systems (IPS), and DDoS protection devices inspect every packet passing through. The moment traffic exceeds their inspection capacity, they drop the packets they can’t process.

Why Hundreds of thousands or more small game packets per second at peak hours or events, or heavy inspection rules → Effect The device maxes out its CPU or packets-per-second limit and drops packets. False positives block legitimate packets too → On screen Freezes and teleporting hit every server behind that device at once, getting worse only when crowds gather

Symptoms
Freeze, Fast-forward, Teleporting, Disconnect
Factors
Packet loss, Latency
Who’s affected
Whole server, Specific region/ISP
When
Evening peak hours, When crowds gather
Owner
Primary owner Infra team (Network infrastructure) · Also Game team (Server development)
Game team action items
Share the game’s traffic pattern (ports, packet sizes, packets per second) with the infra team, batch the small messages for one tick and send them together to cut the packet count.
Infra team action items
Watch the device’s CPU, packets per second, and drop counters alongside game metrics, size devices for small packets, exempt game ports from heavy inspection, tune DDoS protection rules to the game’s traffic pattern.
Ballpark numbers
A “10 Gbps” rating on a spec sheet is often based on large 1,500-byte packets. Game packets are around 100 bytes, so the same bandwidth means more than 10 times as many packets, and the packets-per-second limit fills up first even when the link looks idle.
On the graph
Hits a ceiling · Device packets per second and CPU utilization, device drops
Where to look
The device’s CPU, packets per second, and drop counters, plus packet counts on the switch ports in front of and behind it, compared at the same interval. Overlaid on one screen with concurrent users and server retransmission rate
Confirmed if
At peaks and events, the device’s packets per second or CPU stops at one value and can’t go higher, fewer packets leave the device than enter it, and at the same moment retransmission rates rise across every server behind it
Ruled out if
Same packet counts in front of and behind the device and no device drops: a different cause. NIC drop counters or softnet dropped rising on the server: “Packet drops on the receiving host”
Check with
Infra tools (no game code needed)

Sources

  1. RFC 2544: Benchmarking Methodology for Network Interconnect Devices IETF
    Device performance must be tested at multiple frame sizes, including the minimum and maximum (throughput varies with packet size)

See also

Same layer: Root causes of TCP retransmission

Same symptom (Freeze), other layers

View the interactive card with figures and simulations