Multi-Gen LRU: Stop Linux Thrashing Under Memory Pressure

Every Linux desktop user knows the moment: memory runs low, the mouse starts stuttering, the disk light goes solid and nothing you do matters for the next minute. That is thrashing, and the kernel has a feature aimed squarely at it. The multi-gen LRU (MGLRU) is an alternative page reclaim implementation, and its documentation says it optimizes reclaim and improves performance under memory pressure.

This guide shows how to check that MGLRU is active, how to use its thrashing-prevention valve (min_ttl_ms), and how to use the debugfs interface for working set estimation and proactive reclaim. Everything here comes from the mainline kernel documentation and Kconfig, read while mainline was at 7.3-rc7. It is a companion to my earlier post on reading PSI, which tells you when you are stalling. This one tells you what to change.

Why reclaim policy decides how your machine feels

When RAM fills up, the kernel must decide which pages to throw away or push to swap. A page cache full of files you read once is cheap to drop. The memory of the browser tab you are looking at is not. If reclaim guesses wrong, the kernel evicts a page, you need it 50 ms later, it reads it back from disk and evicts something else you need next. That loop is thrashing.

The kernel documentation puts it in terms of consequences: page reclaim decides the kernel's caching policy and its ability to overcommit memory, and it directly affects kswapd CPU usage and RAM efficiency. The classic approach keeps coarse active and inactive lists. MGLRU instead keeps several generations of pages, so it has a finer sense of how recently each page was used.

Check that MGLRU is on

Per the docs, you need CONFIG_LRU_GEN=y, and CONFIG_LRU_GEN_ENABLED=y if you want it on by default. In mainline Kconfig, LRU_GEN depends on MMU and on either 64-bit, or no SPARSEMEM, or SPARSEMEM_VMEMMAP, so it is generally available on a 64-bit kernel. The runtime control is a stable ABI under /sys/kernel/mm/lru_gen.

zcat /proc/config.gz 2>/dev/null | grep LRU_GEN \
  || grep LRU_GEN /boot/config-$(uname -r)

ls /sys/kernel/mm/lru_gen/
cat /sys/kernel/mm/lru_gen/enabled
# 0x0007 means all three components are on

The enabled file is a bitmask. According to the docs, 0x0001 is the main switch, 0x0002 clears the accessed bit in leaf page table entries in large batches on hardware where the MMU sets it (x86, for example), and 0x0004 does the same for non-leaf entries. The documentation says all components should stay enabled unless one has unforeseen side effects. You can also write y or n to flip everything at once, and a plain number to pick components.

echo y > /sys/kernel/mm/lru_gen/enabled
cat /sys/kernel/mm/lru_gen/enabled
# 0x0007

echo 5 > /sys/kernel/mm/lru_gen/enabled
cat /sys/kernel/mm/lru_gen/enabled
# 0x0005   (main switch + non-leaf batching, leaf batching off)

min_ttl_ms: a pressure relief valve

This is the knob desktop and laptop users care about. Write N to /sys/kernel/mm/lru_gen/min_ttl_ms and the kernel tries to protect the working set of the last N milliseconds from eviction. If it cannot keep that working set in memory, it triggers the OOM killer. The docs call it an adjustable pressure relief valve: when it opens, it terminates applications that are hopefully not being used, instead of letting the whole system grind.

The documentation gives concrete guidance. Based on the average human detectable lag of about 100 ms, N=1000 usually eliminates intolerable janks from thrashing, and larger values such as N=3000 make janks less noticeable at the risk of premature OOM kills. The default is 0, which means disabled. The docs frame this for people without oomd, a userspace OOM daemon, so if you already run systemd-oomd or similar, think about how the two will interact before stacking them.

# current value (0 = off)
cat /sys/kernel/mm/lru_gen/min_ttl_ms

# protect the last second of working set
echo 1000 | sudo tee /sys/kernel/mm/lru_gen/min_ttl_ms

# persist with a oneshot unit
sudo tee /etc/systemd/system/mglru-ttl.service >/dev/null <<'UNIT'
[Unit]
Description=Set MGLRU min_ttl_ms

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo 1000 > /sys/kernel/mm/lru_gen/min_ttl_ms'

[Install]
WantedBy=multi-user.target
UNIT
sudo systemctl enable --now mglru-ttl.service

Working set estimation with debugfs

MGLRU also exposes experimental debugfs files, root-only and not a stable ABI. Reading /sys/kernel/debug/lru_gen gives, for each memory cgroup and NUMA node, a histogram of how many anonymous and file pages were accessed within each age bucket. The generations are listed from min_gen_nr (coldest, oldest age_in_ms) to max_gen_nr (hottest, smallest age_in_ms), and the histograms are noncumulative.

sudo mount -t debugfs none /sys/kernel/debug 2>/dev/null
sudo head -n 20 /sys/kernel/debug/lru_gen
# memcg  memcg_id  memcg_path
#    node  node_id
#        min_gen_nr  age_in_ms  nr_anon_pages  nr_file_pages
#        ...
#        max_gen_nr  age_in_ms  nr_anon_pages  nr_file_pages

The documented use case is a data center job scheduler that needs to know whether a server can fit a new job. You can use the same trick on your own box: ask the kernel to start a new generation, wait, read the file, and see how much memory was touched in that interval. The command is a plus sign followed by memcg_id, node_id and the current max_gen_nr, with optional can_swap and force_scan arguments (force_scan defaults to 1).

F=/sys/kernel/debug/lru_gen
sudo head -n 8 $F                       # note memcg_id, node_id, max_gen_nr
MAXGEN=3                                # replace with what you saw

echo "+ 0 0 $MAXGEN" | sudo tee $F >/dev/null
sleep 60
sudo head -n 12 $F                      # newest generation = pages touched since

Proactive reclaim: evict cold pages on purpose

The same file accepts a minus command that evicts generations up to and including min_gen_nr: a minus sign, then memcg_id, node_id, min_gen_nr, and optionally swappiness and nr_to_reclaim. The docs note that min_gen_nr must be less than max_gen_nr minus 1, because the two newest generations are not fully aged (they are equivalent to the active list) and cannot be evicted. swappiness overrides /proc/sys/vm/swappiness for the call, with a valid range of 0 to 200 or the word max, and max is used exclusively to reclaim anonymous memory.

F=/sys/kernel/debug/lru_gen
MINGEN=0                                # coldest generation from the file

echo "- 0 0 $MINGEN 60 25600" | sudo tee $F >/dev/null
free -m

Gotchas

• Two interfaces, two promises. The sysfs files under /sys/kernel/mm/lru_gen are documented as stable ABI. The debugfs commands are described as experimental, so version-check any tooling that depends on them.
• CONFIG_LRU_GEN_STATS adds the lru_gen_full file with historical stats from evicted generations, but the Kconfig help says not to enable it unless you plan to debug, because it costs memory per memcg and per node.
• On architectures without hardware page-table young bits, the page table walking option is not built (LRU_GEN_WALKS_MMU depends on ARCH_HAS_HW_PTE_YOUNG), and the docs say writes to unsupported components have no effect.
• A hand-edited debugfs command is system-wide. Write a loop that always reads max_gen_nr first, never hardcodes it as the examples above do for brevity.
• Measure before and after. Pair this with PSI: if memory pressure some avg10 falls under load when you set min_ttl_ms, the valve is helping.

Takeaway

If a Linux box freezes instead of failing fast when memory runs out, confirm MGLRU is enabled, try min_ttl_ms at 1000, and watch PSI while you do it. If you run fleets, the debugfs working set and proactive reclaim commands are a ready-made primitive for bin packing. Either way, the documentation is short and worth reading in full.