한국어English日本語简体中文繁體中文DeutschไทยTiếng ViệtРусскийPortuguês (Brasil)EspañolBahasa Indonesia

Game Lag White Paper › L4 Internet path

ISP-shared IP addresses (CGNAT) Carrier-grade NAT

Cause ID isp-cgnat · Primary owner Game team (Client development) · Also Game team (Server development), Infra team (Network infrastructure)

Open the interactive card with figures and simulations →

Mobile networks and some ISPs have many subscribers share one IP address, and they delete the mappings of idle connections after a short time.

Why ISP equipment manages the session table for huge numbers of subscribers → Effect Session table limits, short idle timeouts → On screen Disconnects after sitting idle, and false positives that block everyone sharing the same IP at once

Symptoms
Disconnect, Can’t connect / infinite loading
Factors
Packet loss
Who’s affected
Specific region/ISP
When
After sitting idle
Owner
Primary owner Game team (Client development) · Also Game team (Server development), Infra team (Network infrastructure)
Game team action items
Client: send heartbeats at no more than half the shortest idle timeout (which can be about 30 seconds on mobile networks), send them from the client because ISP CGNAT mappings are reliably refreshed only by outgoing packets, reconnect automatically on disconnect. Server: respond to heartbeats and close the connection proactively if none arrive for a set time, use a session token to keep the same player when a changed mapping gives them a new address and port, be careful with IP-based blocking because many people can share one IP (decide together with account and device signals).
Infra team action items
Adjust per-IP connection limits and new-connections-per-second limits on firewalls and DDoS protection equipment to allow for ISP-shared IPs (raise the thresholds for mobile carrier ranges or exempt them).
Ballpark numbers
On mobile networks the UDP idle timeout can be as short as about 30 seconds.
On the graph
Mass disconnect · Disconnects (heartbeat timeouts), idle time before disconnect (by ISP)
Where to look
In the connection logs, check how many accounts connect from one IP at the same time and their ISP (ASN), and collect per ISP the idle time of connections that dropped after idling. On the player side, check the internet (WAN) address on the router’s admin page
Confirmed if
Multiple accounts connect from one IP in mobile carrier ranges, and idle times before disconnect cluster at short values around 30–60 seconds. The router’s WAN address is in 100.64.0.0/10 (shared address space for carrier NAT) or differs from the address the server sees
Ruled out if
Concentrated among home router users regardless of ISP: “NAT mapping expiry”
Check with
Game server or client logs and metrics

Sources

  1. A Multi-perspective Analysis of Carrier-Grade NAT Deployment (IMC 2016) ACM
    Measured NAT UDP mapping lifetimes of 10–200 seconds, 74% at 1 minute or less; CGN medians of 65 seconds on mobile networks and 35 seconds on wired networks
  2. RFC 6888: Common Requirements for Carrier-Grade NATs (CGNs) IETF
    CGNs must support limits on external ports per subscriber and on the rate of creating new mappings
  3. RFC 6269: Issues with IP Address Sharing IETF
    When many share an address, IP-based blocking (penalty box) also blocks other subscribers on the same address
  4. RFC 6598: IANA-Reserved IPv4 Prefix for Shared Address Space IETF
    100.64.0.0/10 is the shared address space used between carrier NAT (CGN) equipment and subscribers’ routers

See also

Same layer: L4 Internet path

Same symptom (Disconnect), other layers

View the interactive card with figures and simulations