한국어English日本語简体中文繁體中文DeutschไทยTiếng ViệtРусскийPortuguês (Brasil)EspañolBahasa Indonesia

Game Lag White Paper › Root causes of TCP retransmission

Policer drops excess traffic Traffic policing

Cause ID rt-policer · Primary owner Infra team (Network infrastructure) · Also Infra team (Server infrastructure), Game team (Server development)

Open the interactive card with figures and simulations →

ISP plans, cloud instance limits, and DDoS protection devices sometimes drop packets over a set rate right away, without queuing them.

Why Momentary send volume exceeds the allowed rate or allowed burst → Effect Packets over the limit are dropped immediately, with no queue (policing) → On screen Each large burst loses several packets, causing a freeze then fast-forward, while the average rate looks below the limit

Symptoms
Freeze, Fast-forward, Teleporting
Factors
Packet loss
Who’s affected
Whole server, Specific region/ISP, Just me
When
When crowds gather, Evening peak hours
Owner
Primary owner Infra team (Network infrastructure) · Also Infra team (Server infrastructure), Game team (Server development)
Game team action items
Spread each tick’s burst across the tick to keep momentary send volume under the allowed burst, and if you hit a packets-per-second limit, combine one tick’s messages into one packet.
Infra team action items
Network: check the device’s policer exceed counters, replace the policer with a shaper, raise the allowed burst. Servers/OS: check cloud limit-exceeded metrics (on AWS, bw_out_allowance_exceeded and pps_allowance_exceeded in ethtool -S), move to a larger instance, pace on the server (Linux fq qdisc).
Ballpark numbers
A shaper (queues packets and delays them) adds latency; a policer (drops them immediately) adds loss. A TCP game connection can freeze for hundreds of ms on a single loss, so when traffic only briefly goes over the limit, the policer usually does more damage.
On the graph
Hits a ceiling · Send volume at short intervals, policer and allowance exceed counters
Where to look
Exceed and drop counters on the device doing the policing; in the cloud, bw_out_allowance_exceeded and pps_allowance_exceeded from ethtool -S. For connections that lost packets, RTT just before the loss from ss -ti rtt or a packet capture
Confirmed if
Exceed counters rise, and send volume at short intervals looks flat, as if cut off at a fixed value. RTT doesn’t rise before the loss, and several packets vanish at once only during large bursts
Ruled out if
RTT rising first, before the loss, points to queue overflow (“Bottleneck queue overflow (congestion loss),” “Send bursts overflow shallow buffers”). Exceed counters unchanged: a different cause
Check with
Infra tools (no game code needed)

Sources

  1. RFC 2475: An Architecture for Differentiated Services IETF
    Definitions: shaping delays packets to fit a traffic profile, and policing drops packets that exceed the profile
  2. An Internet-Wide Analysis of Traffic Policing Google
    Policed transfers see loss rates 6 times higher on average, and pacing or shaping can achieve the same goal. Policing drops the excess with no RTT increase, while queue overflow raises RTT before the loss (SIGCOMM 2016)
  3. Monitor network performance for ENA settings on your EC2 instance AWS
    bw_out_allowance_exceeded, pps_allowance_exceeded in ethtool -S: packets queued or dropped for exceeding instance limits
  4. tc-fq(8) — Linux manual page iproute2
    Per-connection pacing in the Linux fq qdisc
  5. ss(8) — Linux manual page iproute2
    rtt (average round-trip time) in ss -i

See also

Same layer: Root causes of TCP retransmission

Same symptom (Freeze), other layers

View the interactive card with figures and simulations