linux-stable.git/include/linux/gfp.h, branch v6.18.2

kho: allocate metadata directly from the buddy allocator

2025-11-10T05:19:42+00:00

KHO allocates metadata for its preserved memory map using the slab
allocator via kzalloc().  This metadata is temporary and is used by the
next kernel during early boot to find preserved memory.

A problem arises when KFENCE is enabled.  kzalloc() calls can be randomly
intercepted by kfence_alloc(), which services the allocation from a
dedicated KFENCE memory pool.  This pool is allocated early in boot via
memblock.

When booting via KHO, the memblock allocator is restricted to a "scratch
area", forcing the KFENCE pool to be allocated within it.  This creates a
conflict, as the scratch area is expected to be ephemeral and
overwriteable by a subsequent kexec.  If KHO metadata is placed in this
KFENCE pool, it leads to memory corruption when the next kernel is loaded.

To fix this, modify KHO to allocate its metadata directly from the buddy
allocator instead of slab.

Link: https://lkml.kernel.org/r/20251021000852.2924827-4-pasha.tatashin@soleen.com
Fixes: fc33e4b44b27 ("kexec: enable KHO support for memory preservation")
Signed-off-by: Pasha Tatashin 
Reviewed-by: Pratyush Yadav 
Reviewed-by: Mike Rapoport (Microsoft) 
Reviewed-by: David Matlack 
Cc: Alexander Graf 
Cc: Christian Brauner 
Cc: Jason Gunthorpe 
Cc: Jonathan Corbet 
Cc: Masahiro Yamada 
Cc: Miguel Ojeda 
Cc: Randy Dunlap 
Cc: Samiullah Khawaja 
Cc: Tejun Heo 
Cc: 
Signed-off-by: Andrew Morton

mm: Allow GFP_ACCOUNT to be used in alloc_pages_nolock().

2025-09-29T07:42:35+00:00

Change alloc_pages_nolock() to default to __GFP_COMP when allocating
pages, since upcoming reentrant alloc_slab_page() needs __GFP_COMP.
Also allow __GFP_ACCOUNT flag to be specified,
since most of BPF infra needs __GFP_ACCOUNT except BPF streams.

Reviewed-by: Vlastimil Babka 
Signed-off-by: Alexei Starovoitov 
Reviewed-by: Shakeel Butt 
Reviewed-by: Harry Yoo 
Signed-off-by: Vlastimil Babka

mm/page_isolation: remove migratetype parameter from more functions

2025-07-13T23:38:18+00:00

migratetype is no longer overwritten during pageblock isolation,
start_isolate_page_range(), has_unmovable_pages(), and
set_migratetype_isolate() no longer need which migratetype to restore
during isolation failure.

For has_unmoable_pages(), it needs to know if the isolation is for CMA
allocation, so adding PB_ISOLATE_MODE_CMA_ALLOC provide the information. 
At the same time change isolation flags to enum pb_isolate_mode
(PB_ISOLATE_MODE_MEM_OFFLINE, PB_ISOLATE_MODE_CMA_ALLOC,
PB_ISOLATE_MODE_OTHER).  Remove REPORT_FAILURE and check
PB_ISOLATE_MODE_MEM_OFFLINE, since only PB_ISOLATE_MODE_MEM_OFFLINE
reports isolation failures.

alloc_contig_range() no longer needs migratetype.  Replace it with a newly
defined acr_flags_t to tell if an allocation is for CMA.  So does
__alloc_contig_migrate_range().  Add ACR_FLAGS_NONE (set to 0) to indicate
ordinary allocations.

Link: https://lkml.kernel.org/r/20250617021115.2331563-7-ziy@nvidia.com
Signed-off-by: Zi Yan 
Reviewed-by: Vlastimil Babka 
Acked-by: David Hildenbrand 
Cc: Baolin Wang 
Cc: Brendan Jackman 
Cc: Johannes Weiner 
Cc: Kirill A. Shuemov 
Cc: Mel Gorman 
Cc: Michal Hocko 
Cc: Oscar Salvador 
Cc: Richard Chang 
Cc: Suren Baghdasaryan 
Signed-off-by: Andrew Morton

mm: rename try_alloc_pages() to alloc_pages_nolock()

2025-05-22T21:55:37+00:00

The "try_" prefix is confusing, since it made people believe that
try_alloc_pages() is analogous to spin_trylock() and NULL return means
EAGAIN.  This is not the case.  If it returns NULL there is no reason to
call it again.  It will most likely return NULL again.  Hence rename it to
alloc_pages_nolock() to make it symmetrical to free_pages_nolock() and
document that NULL means ENOMEM.

Link: https://lkml.kernel.org/r/20250517003446.60260-1-alexei.starovoitov@gmail.com
Signed-off-by: Alexei Starovoitov 
Acked-by: Vlastimil Babka 
Acked-by: Johannes Weiner 
Reviewed-by: Shakeel Butt 
Acked-by: Harry Yoo 
Cc: Andrii Nakryiko 
Cc: Kumar Kartikeya Dwivedi 
Cc: Michal Hocko 
Cc: Peter Zijlstra 
Cc: Sebastian Andrzej Siewior 
Cc: Steven Rostedt 
Signed-off-by: Andrew Morton

mm: Fix the flipped condition in gfpflags_allow_spinning()

2025-03-15T18:18:19+00:00

The function gfpflags_allow_spinning() has a bug that makes it return
the opposite result than intended. This could contribute to deadlocks as
usage profilerates, for now it was noticed as a performance regression
due to try_charge_memcg() not refilling memcg stock when it could. Fix
the flipped condition.

Fixes: 97769a53f117 ("mm, bpf: Introduce try_alloc_pages() for opportunistic page allocation")
Reported-by: kernel test robot 
Acked-by: Shakeel Butt 
Signed-off-by: Vlastimil Babka 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20250310124017.187-1-alexei.starovoitov@gmail.com
Closes: https://lore.kernel.org/oe-lkp/202503101254.cfd454df-lkp@intel.com
Signed-off-by: Alexei Starovoitov

mm, bpf: Introduce free_pages_nolock()

2025-02-27T17:36:18+00:00

Introduce free_pages_nolock() that can free pages without taking locks.
It relies on trylock and can be called from any context.
Since spin_trylock() cannot be used in PREEMPT_RT from hard IRQ or NMI
it uses lockless link list to stash the pages which will be freed
by subsequent free_pages() from good context.

Do not use llist unconditionally. BPF maps continuously
allocate/free, so we cannot unconditionally delay the freeing to
llist. When the memory becomes free make it available to the
kernel and BPF users right away if possible, and fallback to
llist as the last resort.

Acked-by: Vlastimil Babka 
Acked-by: Sebastian Andrzej Siewior 
Reviewed-by: Shakeel Butt 
Signed-off-by: Alexei Starovoitov 
Link: https://lore.kernel.org/r/20250222024427.30294-4-alexei.starovoitov@gmail.com
Signed-off-by: Alexei Starovoitov

mm, bpf: Introduce try_alloc_pages() for opportunistic page allocation

2025-02-27T17:32:27+00:00

Tracing BPF programs execute from tracepoints and kprobes where
running context is unknown, but they need to request additional
memory. The prior workarounds were using pre-allocated memory and
BPF specific freelists to satisfy such allocation requests.
Instead, introduce gfpflags_allow_spinning() condition that signals
to the allocator that running context is unknown.
Then rely on percpu free list of pages to allocate a page.
try_alloc_pages() -> get_page_from_freelist() -> rmqueue() ->
rmqueue_pcplist() will spin_trylock to grab the page from percpu
free list. If it fails (due to re-entrancy or list being empty)
then rmqueue_bulk()/rmqueue_buddy() will attempt to
spin_trylock zone->lock and grab the page from there.
spin_trylock() is not safe in PREEMPT_RT when in NMI or in hard IRQ.
Bailout early in such case.

The support for gfpflags_allow_spinning() mode for free_page and memcg
comes in the next patches.

This is a first step towards supporting BPF requirements in SLUB
and getting rid of bpf_mem_alloc.
That goal was discussed at LSFMM: https://lwn.net/Articles/974138/

Acked-by: Michal Hocko 
Acked-by: Vlastimil Babka 
Acked-by: Sebastian Andrzej Siewior 
Reviewed-by: Shakeel Butt 
Signed-off-by: Alexei Starovoitov 
Link: https://lore.kernel.org/r/20250222024427.30294-3-alexei.starovoitov@gmail.com
Signed-off-by: Alexei Starovoitov

mm: alloc_pages_bulk: rename API

2025-01-26T04:22:31+00:00

The previous commit removed the page_list argument from
alloc_pages_bulk_noprof() along with the alloc_pages_bulk_list() function.

Now that only the *_array() flavour of the API remains, we can do the
following renaming (along with the _noprof() ones):

  alloc_pages_bulk_array -> alloc_pages_bulk
  alloc_pages_bulk_array_mempolicy -> alloc_pages_bulk_mempolicy
  alloc_pages_bulk_array_node -> alloc_pages_bulk_node

Link: https://lkml.kernel.org/r/275a3bbc0be20fbe9002297d60045e67ab3d4ada.1734991165.git.luizcap@redhat.com
Signed-off-by: Luiz Capitulino 
Acked-by: David Hildenbrand 
Cc: Matthew Wilcox (Oracle) 
Cc: Mel Gorman 
Cc: Yunsheng Lin 
Signed-off-by: Andrew Morton

mm: alloc_pages_bulk_noprof: drop page_list argument

2025-01-26T04:22:31+00:00

Patch series "mm: alloc_pages_bulk: small API refactor", v2.

Today, alloc_pages_bulk_noprof() supports two arguments to return
allocated pages: a linked list and an array.  There are also higher level
APIs for both.

However, the linked list API has apparently never been used.  So, this
series removes it along with the list API and also refactors the remaining
API naming for consistency.


This patch (of 2):

commit 387ba26fb1cb ("mm/page_alloc: add a bulk page allocator") added
__alloc_pages_bulk() along with the page_list argument.  The next commit
0f87d9d30f21 ("mm/page_alloc: add an array-based interface to the bulk
page allocator") added the array-based argument.  As it turns out, the
page_list argument has no users in the current tree (if it ever had any). 
Dropping it allows for a slight simplification and eliminates some
unnecessary checks, now that page_array is required.

Also, note that the removal of the page_list argument was proposed before
in the thread below, where Matthew Wilcox mentions that:

  """
  Iterating a linked list is _expensive_.  It is about 10x quicker to
  iterate an array than a linked list.
  """
  (https://lore.kernel.org/linux-mm/20231025093254.xvomlctwhcuerzky@techsingularity.net)

Link: https://lkml.kernel.org/r/cover.1734991165.git.luizcap@redhat.com
Link: https://lkml.kernel.org/r/f1c75db91d08cafd211eca6a3b199b629d4ffe16.1734991165.git.luizcap@redhat.com
Signed-off-by: Luiz Capitulino 
Acked-by: David Hildenbrand 
Cc: Matthew Wilcox (Oracle) 
Cc: Mel Gorman 
Cc: Yunsheng Lin 
Signed-off-by: Andrew Morton

mm: make alloc_pages_mpol() static

2025-01-14T06:40:31+00:00

All callers outside mempolicy.c now use folio_alloc_mpol() thanks to
Kefeng's cleanups, so we can remove this as a visible symbol.

And also remove the alloc_hooks for alloc_pages_mpol(), since all users
in mempolicy.c are using the nonprof version.

Link: https://lkml.kernel.org/r/20241125210149.2976098-3-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) 
Reviewed-by: Zi Yan 
Acked-by: David Hildenbrand 
Reviewed-by: Vlastimil Babka 
Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Cc: Mel Gorman 
Cc: Miaohe Lin 
Cc: Muchun Song 
Cc: William Kucharski 
Signed-off-by: Andrew Morton