한국어English日本語简体中文繁體中文DeutschไทยTiếng ViệtРусскийPortuguês (Brasil)EspañolBahasa Indonesia

Game Lag White Paper › L10 Memory

Memory fragmentation Heap fragmentation

Cause ID mem-fragment · Primary owner Game team (Server development)

Open the interactive card with figures and simulations →

When repeated allocation and freeing chops free space into small pieces, the process holds far more memory than it actually uses.

Why Many threads allocate and free blocks of varying sizes over a long time → Effect Free space ends up scattered in small pieces that can’t be returned to the OS, so usage keeps growing like a leak → On screen The longer it runs, the slower it gets from swapping and memory shortage, until it gets killed

Symptoms
Slow motion, Disconnect
Factors
Stall
Who’s affected
Whole server
When
The longer it runs
Owner
Primary owner Game team (Server development)
Game team action items
Use per-size memory pools and fragmentation-resistant allocators (jemalloc, mimalloc, and so on).
On the graph
Slow climb · Process memory (RSS)
Where to look
Run two servers on the same build, and on one of them either reduce the number of glibc arenas with the MALLOC_ARENA_MAX environment variable or switch to another allocator such as jemalloc. Compare RSS from pidstat -r over several days
Confirmed if
With similar player and entity counts, only the changed server’s RSS stops growing or grows much more slowly
Ruled out if
Still climbs the same way after switching allocators: memory that is never freed (mem-leak)
Check with
Infra tools (no game code needed)
Learn more
It looks just like a leak on the graph, yet heap analysis finds no leak site. The default Linux allocator (glibc) is especially bad on servers with many threads, and just switching allocators can cut memory use significantly.

Sources

  1. mallopt(3) — Linux manual page Linux man-pages
    To reduce thread contention, glibc malloc creates arenas up to a multiple of the CPU count, and more arenas mean more memory use (limit with M_ARENA_MAX, or set the MALLOC_ARENA_MAX environment variable)
  2. jemalloc memory allocator jemalloc
    A general-purpose malloc implementation that emphasizes fragmentation avoidance and scalable concurrency
  3. pidstat(1) — Linux manual page sysstat
    -r: per-process RSS (memory actually resident in RAM)

See also

Same layer: L10 Memory

Same symptom (Slow motion), other layers

View the interactive card with figures and simulations