linux.git/include/linux/page-flags.h, branch v7.1-rc2

mm: introduce a new page type for page pool in page type

2026-04-05T20:53:19+00:00

Currently, the condition 'page->pp_magic == PP_SIGNATURE' is used to
determine if a page belongs to a page pool.  However, with the planned
removal of @pp_magic, we should instead leverage the page_type in struct
page, such as PGTY_netpp, for this purpose.

Introduce and use the page type APIs e.g.  PageNetpp(), __SetPageNetpp(),
and __ClearPageNetpp() instead, and remove the existing APIs accessing
@pp_magic e.g.  page_pool_page_is_pp(), netmem_or_pp_magic(), and
netmem_clear_pp_magic().

Plus, add @page_type to struct net_iov at the same offset as struct page
so as to use the page_type APIs for struct net_iov as well.  While at it,
reorder @type and @owner in struct net_iov to avoid a hole and increasing
the struct size.

This work was inspired by the following link:

  https://lore.kernel.org/all/582f41c0-2742-4400-9c81-0d46bf4e8314@gmail.com/

While at it, move the sanity check for page pool to on the free path.

[byungchul@sk.com: gate the sanity check, per Johannes]
  Link: https://lkml.kernel.org/r/20260316223113.20097-1-byungchul@sk.com
Link: https://lkml.kernel.org/r/20260224051347.19621-1-byungchul@sk.com
Co-developed-by: Pavel Begunkov 
Signed-off-by: Pavel Begunkov 
Signed-off-by: Byungchul Park 
Suggested-by: David Hildenbrand 
Acked-by: David Hildenbrand 
Acked-by: Zi Yan 
Acked-by: Vlastimil Babka 
Reviewed-by: Toke Høiland-Jørgensen 
Acked-by: Mike Rapoport (Microsoft) 
Acked-by: Johannes Weiner 
Acked-by: Jakub Kicinski 
Acked-by: Jesper Dangaard Brouer 
Acked-by: Ilias Apalodimas 
Cc: Alexei Starovoitov 
Cc: Andrew Lunn 
Cc: Baolin Wang 
Cc: Brendan Jackman 
Cc: Christian Brauner 
Cc: Daniel Borkmann 
Cc: David S. Miller 
Cc: David Wei 
Cc: Dragos Tatulea 
Cc: Eric Dumazet 
Cc: John Fastabend 
Cc: Leon Romanovsky 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes (Oracle) 
Cc: Mark Bloch 
Cc: Matthew Wilcox (Oracle) 
Cc: Michal Hocko 
Cc: Mina Almasry 
Cc: Paolo Abeni 
Cc: Saeed Mahameed 
Cc: Simon Horman 
Cc: Stanislav Fomichev 
Cc: Stehen Rothwell 
Cc: Suren Baghdasaryan 
Cc: Taehee Yoo 
Cc: Tariq Toukan 
Cc: Usama Arif 
Cc: Yu Zhao 
Signed-off-by: Andrew Morton

mm: remove the branch from compound_head()

2026-04-05T20:53:10+00:00

The compound_head() function is a hot path.  For example, the zap path
calls it for every leaf page table entry.

Rewrite the helper function in a branchless manner to eliminate the risk
of CPU branch misprediction.

Link: https://lkml.kernel.org/r/20260227194302.274384-17-kas@kernel.org
Signed-off-by: Kiryl Shutsemau 
Reviewed-by: Muchun Song 
Reviewed-by: Zi Yan 
Acked-by: David Hildenbrand (Arm) 
Reviewed-by: Vlastimil Babka 
Cc: Albert Ou 
Cc: Alexandre Ghiti 
Cc: Baoquan He 
Cc: Christoph Lameter 
Cc: David Rientjes 
Cc: Frank van der Linden 
Cc: Harry Yoo 
Cc: Huacai Chen 
Cc: Johannes Weiner 
Cc: Jonathan Corbet 
Cc: Lorenzo Stoakes 
Cc: Matthew Wilcox (Oracle) 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Oscar Salvador 
Cc: Palmer Dabbelt 
Cc: Paul Walmsley 
Cc: Roman Gushchin 
Cc: Usama Arif 
Cc: WANG Xuerui 
Signed-off-by: Andrew Morton

mm/hugetlb: remove hugetlb_optimize_vmemmap_key static key

2026-04-05T20:53:09+00:00

The hugetlb_optimize_vmemmap_key static key was used to guard fake head
detection in compound_head() and related functions.  It allowed skipping
the fake head checks entirely when HVO was not in use.

With fake heads eliminated and the detection code removed, the static key
serves no purpose.  Remove its definition and all increment/decrement
calls.

Link: https://lkml.kernel.org/r/20260227194302.274384-16-kas@kernel.org
Signed-off-by: Kiryl Shutsemau 
Reviewed-by: Muchun Song 
Acked-by: David Hildenbrand (Arm) 
Cc: Albert Ou 
Cc: Alexandre Ghiti 
Cc: Baoquan He 
Cc: Christoph Lameter 
Cc: David Rientjes 
Cc: Frank van der Linden 
Cc: Harry Yoo 
Cc: Huacai Chen 
Cc: Johannes Weiner 
Cc: Jonathan Corbet 
Cc: Lorenzo Stoakes 
Cc: Matthew Wilcox (Oracle) 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Oscar Salvador 
Cc: Palmer Dabbelt 
Cc: Paul Walmsley 
Cc: Roman Gushchin 
Cc: Usama Arif 
Cc: Vlastimil Babka 
Cc: WANG Xuerui 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm: drop fake head checks

2026-04-05T20:53:09+00:00

With fake head pages eliminated in the previous commit, remove the
supporting infrastructure:

  - page_fixed_fake_head(): no longer needed to detect fake heads;
  - page_is_fake_head(): no longer needed;
  - page_count_writable(): no longer needed for RCU protection;
  - RCU read_lock in page_ref_add_unless(): no longer needed;

This substantially simplifies compound_head() and page_ref_add_unless(),
removing both branches and RCU overhead from these hot paths.

RCU was required to serialize allocation of hugetlb page against
get_page_unless_zero() and prevent writing to read-only fake head.  It is
redundant without fake heads.

See bd225530a4c7 ("mm/hugetlb_vmemmap: fix race with speculative PFN
walkers") for more details.

synchronize_rcu() in mm/hugetlb_vmemmap.c will be removed by a separate
patch.

Link: https://lkml.kernel.org/r/20260227194302.274384-14-kas@kernel.org
Signed-off-by: Kiryl Shutsemau 
Reviewed-by: Muchun Song 
Acked-by: David Hildenbrand (Arm) 
Reviewed-by: Vlastimil Babka 
Cc: Albert Ou 
Cc: Alexandre Ghiti 
Cc: Baoquan He 
Cc: Christoph Lameter 
Cc: David Rientjes 
Cc: Frank van der Linden 
Cc: Harry Yoo 
Cc: Huacai Chen 
Cc: Johannes Weiner 
Cc: Jonathan Corbet 
Cc: Lorenzo Stoakes 
Cc: Matthew Wilcox (Oracle) 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Oscar Salvador 
Cc: Palmer Dabbelt 
Cc: Paul Walmsley 
Cc: Roman Gushchin 
Cc: Usama Arif 
Cc: WANG Xuerui 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm: rework compound_head() for power-of-2 sizeof(struct page)

2026-04-05T20:53:08+00:00

For tail pages, the kernel uses the 'compound_info' field to get to the
head page.  The bit 0 of the field indicates whether the page is a tail
page, and if set, the remaining bits represent a pointer to the head page.

For cases when size of struct page is power-of-2, change the encoding of
compound_info to store a mask that can be applied to the virtual address
of the tail page in order to access the head page.  It is possible because
struct page of the head page is naturally aligned with regards to order of
the page.

The significant impact of this modification is that all tail pages of the
same order will now have identical 'compound_info', regardless of the
compound page they are associated with.  This paves the way for
eliminating fake heads.

The HugeTLB Vmemmap Optimization (HVO) creates fake heads and it is only
applied when the sizeof(struct page) is power-of-2.  Having identical tail
pages allows the same page to be mapped into the vmemmap of all pages,
maintaining memory savings without fake heads.

If sizeof(struct page) is not power-of-2, there is no functional changes.

Limit mask usage to HugeTLB vmemmap optimization (HVO) where it makes a
difference.  The approach with mask would work in the wider set of
conditions, but it requires validating that struct pages are naturally
aligned for all orders up to the MAX_FOLIO_ORDER, which can be tricky.

Link: https://lkml.kernel.org/r/20260227194302.274384-8-kas@kernel.org
Signed-off-by: Kiryl Shutsemau 
Reviewed-by: Muchun Song 
Reviewed-by: Zi Yan 
Acked-by: David Hildenbrand (Arm) 
Acked-by: Usama Arif 
Reviewed-by: Vlastimil Babka 
Cc: Albert Ou 
Cc: Alexandre Ghiti 
Cc: Baoquan He 
Cc: Christoph Lameter 
Cc: David Rientjes 
Cc: Frank van der Linden 
Cc: Harry Yoo 
Cc: Huacai Chen 
Cc: Johannes Weiner 
Cc: Jonathan Corbet 
Cc: Lorenzo Stoakes 
Cc: Matthew Wilcox (Oracle) 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Oscar Salvador 
Cc: Palmer Dabbelt 
Cc: Paul Walmsley 
Cc: Roman Gushchin 
Cc: WANG Xuerui 
Signed-off-by: Andrew Morton

mm: move set/clear_compound_head() next to compound_head()

2026-04-05T20:53:08+00:00

Move set_compound_head() and clear_compound_head() to be adjacent to the
compound_head() function in page-flags.h.

These functions encode and decode the same compound_info field, so keeping
them together makes it easier to verify their logic is consistent,
especially when the encoding changes.

Link: https://lkml.kernel.org/r/20260227194302.274384-5-kas@kernel.org
Signed-off-by: Kiryl Shutsemau 
Reviewed-by: Muchun Song 
Reviewed-by: Zi Yan 
Acked-by: David Hildenbrand (arm) 
Reviewed-by: Vlastimil Babka 
Cc: Albert Ou 
Cc: Alexandre Ghiti 
Cc: Baoquan He 
Cc: Christoph Lameter 
Cc: David Rientjes 
Cc: Frank van der Linden 
Cc: Harry Yoo 
Cc: Huacai Chen 
Cc: Johannes Weiner 
Cc: Jonathan Corbet 
Cc: Lorenzo Stoakes 
Cc: Matthew Wilcox (Oracle) 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Oscar Salvador 
Cc: Palmer Dabbelt 
Cc: Paul Walmsley 
Cc: Roman Gushchin 
Cc: Usama Arif 
Cc: WANG Xuerui 
Signed-off-by: Andrew Morton

mm: rename the 'compound_head' field in the 'struct page' to 'compound_info'

2026-04-05T20:53:08+00:00

The 'compound_head' field in the 'struct page' encodes whether the page is
a tail and where to locate the head page.  Bit 0 is set if the page is a
tail, and the remaining bits in the field point to the head page.

As preparation for changing how the field encodes information about the
head page, rename the field to 'compound_info'.

Link: https://lkml.kernel.org/r/20260227194302.274384-4-kas@kernel.org
Signed-off-by: Kiryl Shutsemau 
Reviewed-by: Muchun Song 
Reviewed-by: Zi Yan 
Acked-by: David Hildenbrand (arm) 
Reviewed-by: Vlastimil Babka 
Cc: Albert Ou 
Cc: Alexandre Ghiti 
Cc: Baoquan He 
Cc: Christoph Lameter 
Cc: David Rientjes 
Cc: Frank van der Linden 
Cc: Harry Yoo 
Cc: Huacai Chen 
Cc: Johannes Weiner 
Cc: Jonathan Corbet 
Cc: Lorenzo Stoakes 
Cc: Matthew Wilcox (Oracle) 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Oscar Salvador 
Cc: Palmer Dabbelt 
Cc: Paul Walmsley 
Cc: Roman Gushchin 
Cc: Usama Arif 
Cc: WANG Xuerui 
Signed-off-by: Andrew Morton

mm: change the interface of prep_compound_tail()

2026-04-05T20:53:07+00:00

Instead of passing down the head page and tail page index, pass the tail
and head pages directly, as well as the order of the compound page.

This is a preparation for changing how the head position is encoded in the
tail page.

Link: https://lkml.kernel.org/r/20260227194302.274384-3-kas@kernel.org
Signed-off-by: Kiryl Shutsemau 
Reviewed-by: Muchun Song 
Reviewed-by: Zi Yan 
Acked-by: David Hildenbrand (arm) 
Reviewed-by: Vlastimil Babka 
Cc: Albert Ou 
Cc: Alexandre Ghiti 
Cc: Baoquan He 
Cc: Christoph Lameter 
Cc: David Rientjes 
Cc: Frank van der Linden 
Cc: Harry Yoo 
Cc: Huacai Chen 
Cc: Johannes Weiner 
Cc: Jonathan Corbet 
Cc: Lorenzo Stoakes 
Cc: Matthew Wilcox (Oracle) 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Oscar Salvador 
Cc: Palmer Dabbelt 
Cc: Paul Walmsley 
Cc: Roman Gushchin 
Cc: Usama Arif 
Cc: WANG Xuerui 
Signed-off-by: Andrew Morton

mm: add folio_test_lazyfree helper

2026-04-05T20:53:03+00:00

Add folio_test_lazyfree() function to identify lazy-free folios to improve
code readability.

Link: https://lkml.kernel.org/r/20260221093918.1456187-4-vernon2gm@gmail.com
Signed-off-by: Vernon Yang 
Acked-by: David Hildenbrand (Red Hat) 
Reviewed-by: Lance Yang 
Reviewed-by: Dev Jain 
Reviewed-by: Barry Song 
Cc: Baolin Wang 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes 
Cc: Masami Hiramatsu 
Cc: Mathieu Desnoyers 
Cc: Nico Pache 
Cc: Ryan Roberts 
Cc: Steven Rostedt 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

slab: Remove folio references from __ksize()

2025-11-13T10:01:08+00:00

In the future, we will separate slab, folio and page from each other
and calling virt_to_folio() on an address allocated from slab will
return NULL.  Delay the conversion from struct page to struct slab
until we know we're not dealing with a large kmalloc allocation.
There's a minor win for large kmalloc allocations as we avoid the
compound_head() hidden in virt_to_folio().

This deprecates calling ksize() on memory allocated by alloc_pages().
Today it becomes a warning and support will be removed entirely in
the future.

Introduce large_kmalloc_size() to abstract how we represent the size
of a large kmalloc allocation.  For now, this is the same as
page_size(), but it will change with separately allocated memdescs.

Signed-off-by: Matthew Wilcox (Oracle) 
Link: https://patch.msgid.link/20251113000932.1589073-3-willy@infradead.org
Acked-by: David Hildenbrand (Red Hat) 
Reviewed-by: Harry Yoo 
Signed-off-by: Vlastimil Babka