한국어English日本語简体中文繁體中文DeutschไทยTiếng ViệtРусскийPortuguês (Brasil)EspañolBahasa Indonesia

Game Lag White Paper › L4 Internet path

One faulty ECMP path ECMP / link bundle member fault

Cause ID isp-ecmp · Primary owner Infra team (Network infrastructure) · Also Game team (Server development), External (External)

Open the interactive card with figures and simulations →

ISPs and data centers keep several paths to the same destination and send each connection down one of them. If a single path fails, only the players assigned to it keep lagging.

Why One link or device in a bundle of links is faulty or congested → Effect The path is chosen from the address and port combination (hash), so only connections assigned to that path see loss and delay → On screen Same region and ISP, but only some players keep teleporting. Reconnecting sometimes fixes it

Symptoms
Teleporting, Rubber-banding, Stutter
Factors
Packet loss, Jitter
Who’s affected
Just me, Specific region/ISP
When
Always
Owner
Primary owner Infra team (Network infrastructure) · Also Game team (Server development), External (External)
Game team action items
Record per-connection loss and retransmission stats so you can pull the IP, port, and time for affected players (for TCP, the retransmission count from TCP_INFO; for UDP, compute it from missing packet sequence numbers).
Infra team action items
Collect affected players’ IPs, ports, and times and pass them to the ISP or data center, monitor loss per path, measure paths with the same protocol and port as the game (mtr --tcp or --udp with --port), remove the faulty link or device from the bundle if the path runs over our equipment.
External action items
Ask the ISP to check and replace the faulty path, tell players they can work around it for now by reconnecting (when reconnecting changes the port).
Ballpark numbers
With 4 paths, only about a quarter of players are affected. A ping test may take a different path from the game and come back perfectly fine.
On the graph
Outliers only · Per-connection loss/retransmissions (by IP/port)
Where to look
Split per-connection loss and retransmissions by source IP and port. Run mtr in UDP mode (-u) against the game port (-P) with a fixed source port (-L), and repeat several times with different source ports. With -P and no -L, the source port changes on every probe and several paths get mixed together
Confirmed if
Within the same region and ISP, only certain source port (or address) combinations keep losing packets, and the problem goes away when a reconnect changes the port
Ruled out if
Bad no matter which port: congestion or failure across a whole segment
Check with
Infra tools (no game code needed)
Learn more
To keep a connection’s packets in order, network devices (ECMP, LAG) pin each connection to one path using a value computed from its addresses and ports (or only its addresses, depending on device settings). Where only addresses are used, reconnecting lands on the same path and doesn’t help. So when reports like “ping is fine but the game lags” and “reconnecting fixed it” come in together, suspect this cause.

Sources

  1. RFC 7424: Mechanisms for Optimizing Link Aggregation Group (LAG) and Equal-Cost Multipath (ECMP) Component Link Utilization in Networks IETF
    LAG and ECMP pick one link per flow from a hash of header fields to keep packets in order (many-to-one mapping of flows to links)
  2. RFC 2991: Multipath Issues in Unicast and Multicast Next-Hop Selection IETF
    What defines a flow varies by implementation (destination address only, address pair, or including ports); with multiple paths, ping and traceroute results are hard to trust
  3. mtr(8) manual page source mtr
    The -u (UDP), -P (destination port), and -L (UDP source port) options; with only -P, the probe sequence number goes into the source port, so it changes on every probe

See also

Same layer: L4 Internet path

Same symptom (Teleporting), other layers

View the interactive card with figures and simulations