linux.git/mm/page_alloc.c, branch v7.3-rc2

mm/page_alloc: only update lowmem_reserve_ratio on sysctl write

2026-08-25T01:43:01+00:00

lowmem_reserve_ratio_sysctl_handler() ignores the return value of
proc_dointvec_minmax() and always calls setup_per_zone_lowmem_reserve(),
even for read operations.

Fix three issues:

1. Propagate errors from proc_dointvec_minmax() instead of always
   returning success.  For example, writing non-integer garbage to the
   sysctl now returns an error instead of silently succeeding with
   unchanged values.

2. Only call setup_per_zone_lowmem_reserve() when the sysctl is
   actually written, matching the write-only refresh pattern of
   min_free_kbytes and watermark_scale_factor handlers.

3. On write, parse into a temporary ratio[] array and only copy into
   sysctl_lowmem_reserve_ratio[] and refresh derived state after the
   full vector is validated.  This avoids leaving the ratio array
   partially updated while skipping setup when proc_dointvec_minmax()
   returns an error on a later element (suggested by Andrew Morton).

Drop the manual "< 1 -> 0" sanitization loop and set .extra1 = SYSCTL_ZERO
on the ctl_table entry so proc_dointvec_minmax() enforces the minimum on
write; negative values now return -EINVAL instead of being silently
coerced to 0 (suggested by Vlastimil Babka).

[akpm@linux-foundation.org: add comment, per hannes]
  Link: https://lore.kernel.org/anSRGASe5FIrqwlg@cmpxchg.org
Link: https://lore.kernel.org/linux-mm/tencent_FFD4F4D728AAE8A8AE0AF277A59854A29A06@qq.com/
Link: https://lore.kernel.org/tencent_A860C873956A52E26AD8D309A308A241BA08@qq.com
Signed-off-by: Jianlin Shi 
Acked-by: Vlastimil Babka (SUSE) 
Cc: Brendan Jackman 
Cc: Johannes Weiner 
Cc: Michal Hocko 
Cc: Suren Baghdasaryan 
Cc: Zi Yan 
Cc: Joel Granados 
Signed-off-by: Andrew Morton

mm: page_alloc: fix non-movable reclaim storm in defrag_mode

2026-08-07T01:57:24+00:00

As we deployed defrag_mode into Meta production, pressure spikes and
excessive swapping were observed on some workloads.  Tracing confirmed
that this is unmovable/reclaimable requests spinning in the allocator and
direct reclaim, causing excessive amounts of swap.

The initial plan for defrag_mode was to rely on kswapd/kcompactd to
produce blocks, and if those are overwhelmed under high pressure, let the
allocator fall back (__rmqueue_steal()) after its retry loops.  However,
that retrying results in more reclaim on some of these workloads than we'd
hoped, sometimes excessively so, spurred on by the !costly order
conditions in should_reclaim_retry().

The storms are dependent on the request type.  Reclaim will inevitably
make room in existing movable blocks, since that's where the LRU pages
live.  So if movable requests retry on reclaim, they make progress.

When non-movable requests spin in reclaim that isn't productive.  They
cannot use the individually freed pages, and the process is unlikely to
accidentally free whole blocks to meet the ALLOC_NOFRAGMENT bar.  They
spin and overreclaim excessively, which tanks performance and triggers
userspace guards like swap exhaustion or pressure based OOM.

To fix this, send non-movable requests, regardless of order, into
pageblock reclaim/compaction.  This way, they help move things along to
meet the ALLOC_NOFRAGMENT bar.  After this patch, the reclaim storms and
excess OOM rates are no longer observed in production.

The longer-term plan is still to have all requests, including the movable
ones, help make blocks to spread the cost of defragmenting more evenly and
fairly; combined with proper watermarking to reduce allocation latencies
in the common case.  However, doing this naively unearths scaling and
concurrency limitations in compaction that need to be addressed first. 
Promoting just non-movables for now is the minimally viable bug fix for
the above issue.

[brendan.jackman@linux.dev: fix try_to_compact_pages() kerneldoc]
  Link: https://lore.kernel.org/DK7NM9RPUJOD.11PNJJ5N2OBED@linux.dev
Link: https://lore.kernel.org/20260722150006.3848560-5-hannes@cmpxchg.org
Fixes: e3aa7df331bc ("mm: page_alloc: defrag_mode")
Signed-off-by: Johannes Weiner 
Signed-off-by: "Brendan Jackman" 
Reviewed-by: Vlastimil Babka (SUSE) 
Cc: Brendan Jackman 
Cc: David Hildenbrand 
Cc: Gregory Price 
Cc: Liam R. Howlett 
Cc: Lorenzo Stoakes 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Shakeel Butt 
Cc: Suren Baghdasaryan 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton

mm: page_alloc: move capture_control to the page allocator

2026-08-07T01:57:24+00:00

The compaction capturing code assumes the allocation request order and
compaction target order are the same.  That won't be true once defrag_mode
promotes sub-block allocations to pageblock-order compaction: compaction
targets the larger order, while capture should remain at the original
allocation order.

Move the capture_control to the page allocator and give it its own copies
of what the page freeing path matches against - zone, migratetype and the
allocation order - rather than reaching into compaction's live
compact_control.  __alloc_pages_direct_compact() fills in migratetype and
order, and installs and hides current->capture_control around the whole
compaction call; try_to_compact_pages() aims capc->zone at each zone while
it is being compacted.  compact_zone_order() no longer deals with capture
at all.

Pass the capture_control through try_to_compact_pages() /
compact_zone_order() in place of the bare struct page **.

No functional change.

Link: https://lore.kernel.org/20260722150006.3848560-4-hannes@cmpxchg.org
Fixes: e3aa7df331bc ("mm: page_alloc: defrag_mode")
Signed-off-by: Vlastimil Babka (SUSE) 
Co-developed-by: Johannes Weiner 
Signed-off-by: Johannes Weiner 
Reviewed-by: Gregory Price 
Cc: Brendan Jackman 
Cc: Brendan Jackman 
Cc: David Hildenbrand 
Cc: Liam R. Howlett 
Cc: Lorenzo Stoakes 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Shakeel Butt 
Cc: Suren Baghdasaryan 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton

mm: page_alloc: __GFP_FS lockdep annotation for direct compaction

2026-08-07T01:57:23+00:00

Patch series "mm: fix reclaim storms in defrag_mode", v2.

As we deployed vm.defrag_mode=1 in Meta production, some workloads
regressed with recurring pressure spikes and swap storms (which in turn
triggered userspace OOM rules on pressure and swap utilization levels).

Tracing pinned this to non-movable requests spinning and reclaiming
unproductively when kswapd/kcompactd are overwhelmed.  Direct reclaim
predominantly frees up pages in movable blocks, but those requests cannot
use that space under defrag_mode rules; and it is unlikely to free up
whole blocks incidentally for __rmqueue_claim() to work.

This series fixes it by making non-movable requests participate in
pageblock production in the allocator slowpath - meaning, they will invoke
direct reclaim and direct compaction with pageblock_order.

That requires some small-ish adjustments up front in the allocator and the
compaction code: three prep patches and the fix last.

The series has been in production against one of the affected workloads
for several weeks and restores the OOM kill rate to !defrag_mode baseline.


This patch (of 4):

A subsequent patch will have some order-0 allocations participate in
compaction under defrag_mode, to stave off extfrag events.

Since this is a sprawling expansion of entry points, and compaction can
enter filesystem paths, add lockdep annotations that catches __GFP_FS
passing errors.

Direct reclaim has had this annotation for a while, and since reclaim and
compaction are usually used in conjunction, this is unlikely to unearth
old bugs.  It's more about future proofing and peace of mind.

Link: https://lore.kernel.org/20260722150006.3848560-1-hannes@cmpxchg.org
Link: https://lore.kernel.org/20260722150006.3848560-2-hannes@cmpxchg.org
Fixes: e3aa7df331bc ("mm: page_alloc: defrag_mode")
Signed-off-by: Johannes Weiner 
Reviewed-by: Vlastimil Babka (SUSE) 
Acked-by: Shakeel Butt 
Cc: Brendan Jackman 
Cc: Brendan Jackman 
Cc: David Hildenbrand 
Cc: Liam R. Howlett 
Cc: Lorenzo Stoakes 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Suren Baghdasaryan 
Cc: Zi Yan 
Cc: Brendan Jackman 
Cc: Gregory Price 
Cc: 
Signed-off-by: Andrew Morton

mm/page_alloc: remove a couple of VM_BUG_ON()st

2026-08-07T01:57:11+00:00

VM_BUG_ON() is out of favour and on the way to removal, since I recently
touched alloc_pages_node_noprof() I am removing that invocation, and also
removing the __folio_alloc_node_noprof() one for consistency.  If this
precondition is violated, the system will soon crash anyway.

Link: https://lore.kernel.org/20260715-spin-trylock-followup-v3-4-fc4d246f705d@google.com
Signed-off-by: Brendan Jackman 
Suggested-by: Zi Yan 
Link: https://lore.kernel.org/all/7F866265-3F2E-4765-B9D4-9AB898A9C4AC@nvidia.com/
Acked-by: Zi Yan 
Reviewed-by: Vlastimil Babka (SUSE) 
Cc: Brendan Jackman 
Cc: David Hildenbrand 
Cc: Johannes Weiner 
Cc: Liam R. Howlett 
Cc: Lorenzo Stoakes 
Cc: Michal Hocko 
Cc: Michal Koutný 
Cc: Mike Rapoport 
Cc: Sebastian Andrzej Siewior 
Cc: Steven Rostedt 
Cc: Suren Baghdasaryan 
Cc: Tejun Heo 
Cc: Waiman Long 
Signed-off-by: Andrew Morton

mm/page_alloc: fixup alloc_pages_nolock_noprof() comment

2026-08-07T01:57:11+00:00

Update the comment to reflect the recent change to allow flags in
gfp_nolock.

Link: https://lore.kernel.org/20260715-spin-trylock-followup-v3-3-fc4d246f705d@google.com
Signed-off-by: Brendan Jackman 
Reported-by: sashiko-bot@kernel.org
Link: https://sashiko.dev/#/patchset/20260703-alloc-trylock-v5-0-c87b714e19d3%40google.com?part=6
Reviewed-by: Zi Yan 
Reviewed-by: Vlastimil Babka (SUSE) 
Cc: Brendan Jackman 
Cc: David Hildenbrand 
Cc: Johannes Weiner 
Cc: Liam R. Howlett 
Cc: Lorenzo Stoakes 
Cc: Michal Hocko 
Cc: Michal Koutný 
Cc: Mike Rapoport 
Cc: Sebastian Andrzej Siewior 
Cc: Steven Rostedt 
Cc: Suren Baghdasaryan 
Cc: Tejun Heo 
Cc: Waiman Long 
Signed-off-by: Andrew Morton

mm/page_alloc: rename FPI_TRYLOCK -> FPI_NOLOCK

2026-08-07T01:57:10+00:00

Patch series "mm/page_alloc: couple of followups for recent cleanups", v3.


This patch (of 4):

As discussed in the linked patch, the there is some inconsistency between
"trylock" and "nolock" nomenclature, let's align it.  Since "nolock" is
used in the public API it seems to have more mindshare so do that.

The linked patch did this for the ALLOC_ flag but forgot about FPI_.

Link: https://lore.kernel.org/20260715-spin-trylock-followup-v3-0-fc4d246f705d@google.com
Link: https://lore.kernel.org/all/20260703-alloc-trylock-v5-1-c87b714e19d3@google.com/
Link: https://lore.kernel.org/20260715-spin-trylock-followup-v3-1-fc4d246f705d@google.com
Signed-off-by: Brendan Jackman 
Reviewed-by: Zi Yan 
Reviewed-by: Vlastimil Babka (SUSE) 
Cc: Brendan Jackman 
Cc: David Hildenbrand 
Cc: Johannes Weiner 
Cc: Liam R. Howlett 
Cc: Lorenzo Stoakes 
Cc: Michal Hocko 
Cc: Michal Koutný 
Cc: Mike Rapoport 
Cc: Sebastian Andrzej Siewior 
Cc: Steven Rostedt 
Cc: Suren Baghdasaryan 
Cc: Tejun Heo 
Cc: Waiman Long 
Signed-off-by: Andrew Morton

mm/page_alloc: don't spin_trylock() when disallowed in free_one_page()

2026-08-07T01:57:10+00:00

__free_frozen_pages() checks can_spin_trylock() before entering into the
main free_frozen_page_commit()/free_one_page() path, but before this it
can get to free_one_page() via the !pcp_allowed_order() and
MIGRATE_ISOLATE paths.

The !pcp_allowed_order() path depends on usage by callers so might not be
possible in practice.  The MIGRATE_ISOLATE path probably means kernel
crashes and privilege escalation if anyone ever did memory hotplug and BPF
tracing on a PREEMPT_RT or !SMP build.

Link: https://lore.kernel.org/20260715-alloc-nolock-fixes-v1-2-fadc49952dda@google.com
Signed-off-by: Brendan Jackman 
Fixes: 8c57b687e833 ("mm, bpf: Introduce free_pages_nolock()")
Reported-by: sashiko-bot@kernel.org
Closes: https://sashiko.dev/#/patchset/20260710-spin-trylock-followup-v1-0-affb5fe5ed00%40google.com?part=2
Reviewed-by: Vlastimil Babka (SUSE) 
Reviewed-by: Harry Yoo (Oracle) 
Cc: Brendan Jackman 
Cc: Alexei Starovoitov 
Cc: Johannes Weiner 
Cc: Michal Hocko 
Cc: Sebastian Andrzej Siewior 
Cc: Shakeel Butt 
Cc: Steven Rostedt 
Cc: Suren Baghdasaryan 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton

mm/page_alloc: don't spin_trylock() in NMI on UP

2026-08-07T01:57:10+00:00

Patch series "mm/page_alloc: fixes for free_pages_nolock() on RT/UP".

Pre-existing bugs found by Sashiko during review of this other series:
https://lore.kernel.org/all/20260703-alloc-trylock-v5-0-c87b714e19d3@google.com/

I have not reproduced these bugs, and I suspect there is no real-world
user that is affected by them.


This patch (of 2):

As noted in can_spin_trylock(), using this is unsafe in this context. 
commit 620b46ed6ae17 ("mm/page_alloc: return NULL early from
alloc_frozen_pages_nolock() in NMI on UP") fixed this on the alloc side
but missed the free side.

Impact: If BPF programs using these features in NMI (probably tracing) are
present on non-SMP builds this might crash the kernel and is probably
exploitable by local attackers for privilege escalation.

Link: https://lore.kernel.org/20260715-alloc-nolock-fixes-v1-0-fadc49952dda@google.com
Link: https://lore.kernel.org/20260715-alloc-nolock-fixes-v1-1-fadc49952dda@google.com
Fixes: 8c57b687e833 ("mm, bpf: Introduce free_pages_nolock()")
Signed-off-by: Brendan Jackman 
Reported-by: sashiko-bot@kernel.org
Closes: https://sashiko.dev/#/patchset/20260703-alloc-trylock-v5-0-c87b714e19d3%40google.com?part=18
Reviewed-by: Vlastimil Babka (SUSE) 
Reviewed-by: Harry Yoo (Oracle) 
Cc: Brendan Jackman 
Cc: Alexei Starovoitov 
Cc: Johannes Weiner 
Cc: Michal Hocko 
Cc: Sebastian Andrzej Siewior 
Cc: Shakeel Butt 
Cc: Steven Rostedt 
Cc: Suren Baghdasaryan 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton

mm: let node_reclaim() return the number of pages reclaimed

2026-08-07T01:57:07+00:00

There is only one caller, get_page_from_freelist(), and it does not make
any use of the reason for skipping the reclaim, nor does it make any
distinction between a full and partially successful reclaim.

Therefore, node_reclaim() can simply return the number of pages that have
been reclaimed, same as __node_reclaim(), and the NODE_RECLAIM_xxx macros
can be removed.

There is one small change of behavior when __node_reclaim() was attempted
but returned zero.  The allocation now skips the zone immediately; before
this patch, the zone watermarks were checked first.  I believe it was an
oversight rather than intention, because the chances that zone watermark
is OK after __node_reclaim() did not reclaim any pages are very close to
zero.

Originally, I was looking for occurences of NODE_RECLAIM_SOME and
NODE_RECLAIM_SUCCESS, but I couldn't find any.  That's because they are
typecast from the result of a relational operator.  This seemed a bit
fragile, so I dug a bit deeper and came up with this proposed cleanup.

Link: https://lore.kernel.org/20260714132300.2136018-1-ptesarik@suse.com
Signed-off-by: Petr Tesarik 
Reviewed-by: Vlastimil Babka (SUSE) 
Reviewed-by: Zi Yan 
Acked-by: Johannes Weiner 
Cc: Brendan Jackman 
Cc: Brendan Jackman 
Cc: David Hildenbrand 
Cc: Liam R. Howlett 
Cc: Lorenzo Stoakes 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Suren Baghdasaryan 
Cc: Zi Yan 
Signed-off-by: Andrew Morton