Game Lag White Paper › L7 Server OS (kernel)

Scheduled jobs Cron jobs (log rotation, backup, scans)

Cause ID so-cron · Primary owner Infra team (Server infrastructure)

Open the interactive card with figures and simulations →

Log compression, backups, and security scans that run at the same time every day take up CPU and disk.

Why An OS job runs at a scheduled time → Effect It shares CPU and disk with the game server → On screen Stutter and slow motion at a fixed time, such as 4 a.m. every day

Symptoms

Stutter, Slow motion

Factors

Stall

Who’s affected

Whole server

When

At regular intervals

Owner

Primary owner Infra team (Server infrastructure)

Infra team action items

Stagger job times, lower their priority (nice, ionice), move them off the game server (run them on a separate server).

On the graph

Periodic spikes · CPU utilization, disk queue, server tick time

Where to look

Run times of scheduled jobs collected from crontab and systemctl list-timers, and which processes use CPU and disk when ticks spike, from pidstat -u -d

Confirmed if

Ticks spike at the same time every day (or every hour), and at that time a scheduled job process takes up CPU and disk

Ruled out if

Spikes don’t line up with a fixed time of day: not this cause. Spikes every few seconds or minutes: “Server GC stop-the-world pause,” “Timers firing all at once”

Check with

Infra tools (no game code needed)

Sources

1. ionice(1) — Linux manual page util-linux
   Jobs in the idle class get disk I/O only when no other program is using the disk
2. systemd.timer(5) — Linux manual page systemd
   RandomizedDelaySec delays scheduled jobs by a random amount to reduce load pileups
3. systemctl(1) — Linux manual page systemd
   list-timers: shows timer units in order of next run time
4. pidstat(1) — Linux manual page sysstat
   -u shows CPU and -d shows disk I/O per process

See also

Same layer: L7 Server OS (kernel)

• Connection queue (listen backlog) overflow
• File descriptor limit
• Kernel socket buffers too small
• Too many threads and context switching
• CPU steal (virtual machines)
• Container CPU throttling (CFS quota)
• Latency spikes from server power management (C-states, frequency scaling)
• OOM killer
• Stalls from memory reclaim and compaction
• System clock jump (NTP step)
• Performance changes after OS, kernel, driver, or firmware updates
• Server conntrack table full
• Ephemeral port exhaustion on server-to-server connections

Same symptom (Stutter), other layers

• Frame time spike L1 Client game process
• Client garbage collection L1 Client game process
• Synchronous loading and shader compilation on the main thread L1 Client game process
• Slow storage delays asset streaming L1 Client game process
• Rendering load from large crowds L1 Client game process
• Missing or too-short interpolation buffer L1 Client game process
• Excessive extrapolation (dead reckoning) L1 Client game process
• Fixed-timestep catch-up spiral L1 Client game process

View the interactive card with figures and simulations