한국어English日本語简体中文繁體中文DeutschไทยTiếng ViệtРусскийPortuguês (Brasil)EspañolBahasa Indonesia

Game Lag White Paper › L7 Server OS (kernel)

Container CPU throttling (CFS quota) Container CPU throttling (CFS quota)

Cause ID so-cpu-quota · Primary owner Infra team (Server infrastructure) · Also Game team (Server development)

Open the interactive card with figures and simulations →

With a CPU limit on a container, the moment it uses up its quota within a set period (usually 100 ms), it is forced to stop for the rest of that period (throttling).

Why A CPU limit is set on the game server container (in Kubernetes, for example) → Effect A burst of tick work uses up the quota, and the server stops for tens of ms until the next period → On screen Average CPU is low, yet ticks spike periodically: stutter, slow motion

Symptoms
Stutter, Slow motion
Factors
Stall, Jitter
Who’s affected
Whole server
When
When crowds gather, Randomly
Owner
Primary owner Infra team (Server infrastructure) · Also Game team (Server development)
Game team action items
Match the worker thread count to the CPU limit (so the runtime doesn’t create as many threads as the host has cores).
Infra team action items
Set a generous CPU limit or remove it and assign dedicated cores, monitor the throttle count (nr_throttled).
Ballpark numbers
On a server limited to 2 cores, 8 threads working at once use up the quota for a 100 ms period in 25 ms and then stop for 75 ms.
On the graph
Rises with load · Throttle count (nr_throttled), server tick time
Where to look
Increase in nr_throttled and throttled_usec (nr_throttled and throttled_time on cgroup v1) in the container cgroup’s cpu.stat, alongside server tick time
Confirmed if
Average CPU utilization stays below the limit, yet nr_throttled and throttled_usec keep rising, lining up with tick spikes
Ruled out if
nr_throttled not rising: not this cause. The VM itself being held back: “CPU steal (virtual machines)”
Check with
Infra tools (no game code needed)

Sources

  1. CFS Bandwidth Control Linux kernel
    Threads that use up the quota for a period stop until the next period (throttling), default period 100 ms, nr_throttled statistic
  2. Control Group v2 Linux kernel
    cpu.max takes the form “$MAX $PERIOD” (quota, period), and the default is “max 100000” (100 ms period)
  3. Resource Management for Pods and Containers Kubernetes
    A container’s CPU limit is a hard limit that the kernel enforces with CPU throttling

See also

Same layer: L7 Server OS (kernel)

Same symptom (Stutter), other layers

View the interactive card with figures and simulations