linux.git/include/linux/hugetlb.h, branch v7.3-rc2

mm/rmap: add try_to_unmap_poisoned_hugetlb_one

2026-08-05T02:19:06+00:00

Simplify try_to_unmap_one() by separating the hugetlb parts into
try_to_unmap_poisoned_hugetlb_one().

To understand the correctness of the refactoring, the following points
are noted:

1. try_to_unmap() is called for hugetlb folios only when they are
   hwpoisoned.

2. A hugetlb VMA cannot be mlocked.

3. page_vma_mapped_walk() returns at most one hugetlb mapping in a VMA,
   and that mapping points at the head PFN.

4. We won't ever process a softleaf entry that encodes a hugetlb folio;
   hugetlb folios are never swapped out, migration entries will be
   skipped (PVMW_MIGRATION not passed), and device-exclusive does not
   work for hugetlb.

5. The hwpoison entry is constructed from the poisoned folio, just as in
   the pre-refactor code. Any previous uffd-wp state is deliberately not
   preserved for the hwpoison entry.

6. TTU_HWPOISON is always present; for it to not be present, either the
   folio has to be in swapcache, or mapping_can_writeback() is true (see
   unmap_poisoned_folio), none of which is true for hugetlb folios.

7. Hugetlb uses separate counters from normal rss counters, therefore
   update_highwater_rss() need not be called.

While at it:

 - Change VM_BUG_* to VM_WARN_*.

 - Do not declare variables which are only used once.

 - Constify some variables.

 - Add some more VM_WARN_* to assert some invariants.

Except the above 4 points, no functional change intended.

Link: https://lore.kernel.org/20260730094559.418003-3-dev.jain@arm.com
Signed-off-by: Dev Jain 
Reviewed-by: Lorenzo Stoakes (ARM) 
Suggested-by: David Hildenbrand (Arm) 
Acked-by: David Hildenbrand (Arm) 
Cc: Anshuman Khandual 
Cc: Harry Yoo 
Cc: Jann Horn 
Cc: Lance Yang 
Cc: Liam R. Howlett 
Cc: Muchun Song 
Cc: Oscar Salvador 
Cc: Rik van Riel 
Cc: Ryan Roberts 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton

mm/vma: use vma_start_pgoff(), linear_page_index() in mm code

2026-08-05T02:19:00+00:00

There are many instances in which linear_page_index() (as well as
linear_page_delta()) is open-coded, which is confusing and inconsistent.

Additionally, vma->vm_pgoff doesn't necessarily make it clear that this is
the page offset of the start of the VMA range.

Doing so also aids greppability.

So use vma_start_pgoff() in favour of directly accessing vma->vm_pgoff, and
linear_page_index() where we can.

This also lays the ground for future changes which will add an anonymous
page offset in order to be able to index MAP_PRIVATE-file backed anon
folios in terms of their virtual page offset.

No functional change intended.

Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-18-2a5aa403d977@kernel.org
Signed-off-by: Lorenzo Stoakes 
Reviewed-by: Gregory Price 
Reviewed-by: SJ Park 
Reviewed-by: Pedro Falcato 
Reviewed-by: Vlastimil Babka (SUSE) 
Cc: Ackerley Tng 
Cc: David Hildenbrand (Arm) 
Cc: Kai Huang 
Cc: Marek Szyprowski 
Cc: Thomas Zimmermann 
Cc: Liam R. Howlett (Oracle) 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm/rmap: use huge_ptep_get() in try_to_unmap_one()

2026-07-31T02:40:45+00:00

Patch series "Fix incorrect access of hugetlb pte entries", v3.

There are various places which use ptep_get() to get the pte entry
corresponding to a hugetlb folio.  Some arches (like s390) have special
handling to compute the pteval, so they provide huge_ptep_get().  Use this
helper consistently.

Additionally, some code paths may provide huge_ptep_get with an unaligned
address.  This is a problem on arm64 (I checked other arches and it looks
fine for them), which is fixed in patch 1.  The fix is made to be
backport-friendly: the cleaner fix would be to perhaps pass the hstate to
huge_ptep_get() - that is wider churn and we can do that later.


This patch (of 5):

try_to_unmap_one() handles hugetlb folios when memory failure needs to
replace a poisoned hugetlb mapping with a hwpoison entry.  In that case
page_vma_mapped_walk() returns the pte pointer to the hugetlb folio in
pvmw.pte, but the code reads it with ptep_get().

On arches which provide their own huge_ptep_get() to dereference a huge
pte pointer, accessing via ptep_get() would cause pte_pfn(), pte_present()
etc to misbehave.

It is not clear whether this has a trivially visible effect to userspace.

Just use huge_ptep_get() for dereferencing a huge pte pointer.

Link: https://lore.kernel.org/20260703114202.365553-1-dev.jain@arm.com
Link: https://lore.kernel.org/20260703114202.365553-3-dev.jain@arm.com
Fixes: c7ab0d2fdc84 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()")
Signed-off-by: Dev Jain 
Reported-by: David Hildenbrand 
Reviewed-by: Muchun Song 
Acked-by: David Hildenbrand (Arm) 
Cc: Alistair Popple 
Cc: Andi Kleen 
Cc: Anshuman Khandual 
Cc: Byungchul Park 
Cc: Catalin Marinas 
Cc: Dave Hansen 
Cc: Gregory Price 
Cc: Harry Yoo 
Cc: "Huang, Ying" 
Cc: Jann Horn 
Cc: Josh Poimboeuf 
Cc: Joshua Hahn 
Cc: Jun'ichi "Nick" Nomura 
Cc: Kiryl Shutsemau 
Cc: Lance Yang 
Cc: Liam R. Howlett 
Cc: Lorenzo Stoakes 
Cc: Matthew Brost 
Cc: Mel Gorman 
Cc: Naoya Horiguchi 
Cc: Oscar Salvador 
Cc: Pedro Falcato 
Cc: Rakie Kim 
Cc: Ralph Campbell 
Cc: Rik van Riel 
Cc: Ryan Roberts 
Cc: Vlastimil Babka 
Cc: Will Deacon 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton

mm: hugetlb: refactor out hugetlb_alloc_folio()

2026-07-31T02:40:38+00:00

Refactor out hugetlb_alloc_folio() from alloc_hugetlb_folio(), which
handles allocation of a folio and memory and HugeTLB charging to cgroups.

This refactoring decouples the HugeTLB page allocation from VMAs,
specifically:

1. Reservations (as in resv_map) are stored in the vma
2. mpol is stored at vma->vm_policy
3. A vma must be used for allocation even if the pages are not meant to be
   used by host process.

Without this coupling, VMAs are no longer a requirement for allocation. 
This opens up the allocation routine for usage without VMAs, which will
allow guest_memfd to use HugeTLB as a more generic allocator of huge
pages, since guest_memfd memory may not have any associated VMAs by
design.  In addition, direct allocations from HugeTLB could possibly be
refactored to avoid the use of a pseudo-VMA.

Also, this decouples HugeTLB page allocation from HugeTLBfs, where the
subpool is stored at the fs mount.  This is also a requirement for
guest_memfd, where the plan is to have a subpool created per-fd and stored
on the inode.

Provide and use alloc_flags to allow more allocation knobs in future
without expanding the number of parameters in hugetlb_alloc_folio().

No functional change intended.

Link: https://lore.kernel.org/20260702-hugetlb-open-up-v4-6-d53cefcccf34@google.com
Signed-off-by: Ackerley Tng 
Cc: Alistair Popple 
Cc: Byungchul Park 
Cc: David Hildenbrand 
Cc: David Rientjes 
Cc: "Edgecombe, Rick P" 
Cc: Frank van der Linden 
Cc: Gregory Price 
Cc: "Huang, Ying" 
Cc: James Houghton 
Cc: Jason Gunthorpe 
Cc: Jiaqi Yan 
Cc: Joshua Hahn 
Cc: Matthew Brost 
Cc: Michael Roth 
Cc: Michal Hocko 
Cc: Muchun Song 
Cc: Oscar Salvador 
Cc: Paolo Bonzini 
Cc: Pasha Tatashin 
Cc: Peter Xu 
Cc: Pratyush Yadav 
Cc: Rakie Kim 
Cc: Roman Gushchin 
Cc: Sean Christopherson 
Cc: Shakeel Butt 
Cc: Shivank Garg 
Cc: Vishal Annapurve 
Cc: Yan Zhao 
Cc: Zi Yan 
Cc: Qi Zheng 
Signed-off-by: Andrew Morton

mm/hugetlb: remove obsolete bootmem cross-zone checks

2026-07-29T04:12:00+00:00

Bootmem gigantic HugeTLB pages used to be validated again during
gather_bootmem_prealloc_node() and any cross-zone pages were discarded
there.

That validation is no longer needed.  Cross-zone bootmem gigantic pages
are now detected during allocation and freed before they reach the later
bootmem gathering path, so the remaining pages are already zone-valid.

Remove the obsolete cross-zone validation, invalid-page freeing, and the
associated discarded-page accounting.

Link: https://lore.kernel.org/20260612035903.2468601-17-songmuchun@bytedance.com
Signed-off-by: Muchun Song 
Acked-by: Mike Rapoport (Microsoft) 
Cc: "Aneesh Kumar K.V" 
Cc: David Hildenbrand 
Cc: Frank van der Linden 
Cc: Liam R. Howlett 
Cc: Lorenzo Stoakes 
Cc: Madhavan Srinivasan 
Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Oscar Salvador (SUSE) 
Cc: "Ritesh Harjani (IBM)" 
Cc: Usama Arif 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton

mm/hugetlb: refactor early boot gigantic hugepage allocation

2026-07-29T04:11:59+00:00

The early boot gigantic hugepage allocation helpers currently mix
allocation with huge_bootmem_page setup, and leave part of the
initialization flow in architecture code.

Refactor the interface to return the allocated huge page pointer and move
the huge_bootmem_page setup into the generic hugetlb code.  This makes the
architecture-specific paths focus only on finding memory, while the common
code handles node placement and early page metadata setup in one place.

This also lets powerpc benefit from memblock_reserved_mark_noinit(), which
it did not enable before.

In addition, upcoming cross-zone validation for boot-time gigantic hugetlb
reservation is common logic.  With this refactoring, that logic can stay
in the generic code instead of being duplicated in architecture-specific
paths.

Link: https://lore.kernel.org/20260612035903.2468601-14-songmuchun@bytedance.com
Signed-off-by: Muchun Song 
Reviewed-by: Mike Rapoport (Microsoft) 
Reviewed-by: Oscar Salvador (SUSE) 
Cc: "Aneesh Kumar K.V" 
Cc: David Hildenbrand 
Cc: Frank van der Linden 
Cc: Liam R. Howlett 
Cc: Lorenzo Stoakes 
Cc: Madhavan Srinivasan 
Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Cc: Oscar Salvador (SUSE) 
Cc: "Ritesh Harjani (IBM)" 
Cc: Usama Arif 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton

mm/hugetlb: initialize gigantic bootmem hugepage struct pages earlier

2026-07-29T04:11:57+00:00

Gigantic bootmem HugeTLB pages are currently initialized from
hugetlb_init(), but page_alloc_init_late() runs earlier and walks
pageblocks to determine zone contiguity.

If a bootmem HugeTLB region is marked noinit, set_zone_contiguous() can
observe still-uninitialized struct pages through
__pageblock_pfn_to_page().  This may not trigger an immediate failure, but
it can make set_zone_contiguous() compute the wrong zone contiguity state.
If extra poisoned-page checks are added in this path, such as
PF_POISONED_CHECK() in page_zone_id(), it can also trigger an early boot
panic.

Initialize gigantic bootmem HugeTLB struct pages from
page_alloc_init_late(), before zone contiguity is evaluated, so later page
allocator setup only sees valid struct page state.  This also makes the
initialization order more natural, as struct pages should be initialized
before later code inspects them.

Link: https://lore.kernel.org/20260612035903.2468601-5-songmuchun@bytedance.com
Fixes: fde1c4ecf916 ("mm: hugetlb: skip initialization of gigantic tail struct pages if freed by HVO")
Signed-off-by: Muchun Song 
Acked-by: Mike Rapoport (Microsoft) 
Acked-by: Oscar Salvador 
Cc: "Aneesh Kumar K.V" 
Cc: David Hildenbrand 
Cc: Frank van der Linden 
Cc: Liam R. Howlett 
Cc: Lorenzo Stoakes 
Cc: Madhavan Srinivasan 
Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Cc: Oscar Salvador (SUSE) 
Cc: "Ritesh Harjani (IBM)" 
Cc: Usama Arif 
Cc: Vlastimil Babka 
Cc: 
Signed-off-by: Andrew Morton

mm: use enum migrate_reason instead of int for migration reason parameters

2026-07-29T04:11:50+00:00

Replace all 'int reason' function parameters that carry migrate_reason
values with the proper 'enum migrate_reason' type.  This makes the intent
explicit and leverages compiler type checking.  The affected subsystems
are:

  - page_owner: __folio_set_owner_migrate_reason(),
                folio_set_owner_migrate_reason()
  - migrate: migrate_pages(), migrate_pages_sync(),
             migrate_pages_batch(), migrate_folios_move(),
             migrate_hugetlbs(), unmap_and_move_huge_page()
  - hugetlb: move_hugetlb_state(), htlb_allow_alloc_fallback()
  - trace: mm_migrate_pages and mm_migrate_pages_start events

The 'short last_migrate_reason' struct field and internal helper parameter
in page_owner are intentionally left as 'short' since they store per-page
metadata where size matters.

No functional change.

Link: https://lore.kernel.org/20260714015117.78351-4-ye.liu@linux.dev
Signed-off-by: Ye Liu 
Reviewed-by: Zi Yan 
Reviewed-by: Vlastimil Babka (SUSE) 
Reviewed-by: Lorenzo Stoakes 
Acked-by: David Hildenbrand (Arm) 
Cc: Brendan Jackman 
Cc: Johannes Weiner 
Cc: Michal Hocko 
Cc: Suren Baghdasaryan 
Signed-off-by: Andrew Morton

mm/memory-failure: fix hugetlb_lock AA deadlock in get_huge_page_for_hwpoison

2026-05-29T03:50:34+00:00

Two concurrent madvise(MADV_HWPOISON) calls on the same hugetlb page can
trigger a recursive spinlock self-deadlock (AA deadlock) on hugetlb_lock
when racing with a concurrent unmap:

  thread#0                              thread#1
  --------                              --------
  madvise(folio, MADV_HWPOISON)
    -> poisons the folio successfully
  madvise(folio, MADV_HWPOISON)         unmap(folio)
    try_memory_failure_hugetlb
      get_huge_page_for_hwpoison
        spin_lock_irq(&hugetlb_lock)    <- held
        __get_huge_page_for_hwpoison
          hugetlb_update_hwpoison()
            -> MF_HUGETLB_FOLIO_PRE_POISONED
          goto out:
            folio_put()
              refcount: 1 -> 0
              free_huge_folio()
                spin_lock_irqsave(&hugetlb_lock)
                  -> AA DEADLOCK!

The out: path in __get_huge_page_for_hwpoison() calls folio_put() to drop
the GUP reference while the hugetlb_lock is still held by the hugetlb.c
wrapper get_huge_page_for_hwpoison().  If concurrent unmap has released
the page table mapping reference, folio_put() drops the folio refcount to
zero, triggering free_huge_folio() which attempts to re-acquire the
non-recursive hugetlb_lock.

Fix this by moving hugetlb_lock acquisition from the hugetlb.c wrapper
into get_huge_page_for_hwpoison().  Place spin_unlock_irq() before the
folio_put() at the out: label so the folio is always released outside the
lock.

[akpm@linux-foundation.org: fix race, rename label per Miaohe]
  Link: https://sashiko.dev/#/patchset/20260522010305.4099834-1-mawupeng1@huawei.com
  Link: https://lore.kernel.org/f39f405e-4b4b-8f79-70fe-a2b5b62114eb@huawei.com
Link: https://lore.kernel.org/20260522010305.4099834-1-mawupeng1@huawei.com
Fixes: 405ce051236c ("mm/hwpoison: fix race between hugetlb free/demotion and memory_failure_hugetlb()")
Signed-off-by: Wupeng Ma 
Acked-by: Oscar Salvador (SUSE) 
Acked-by: Muchun Song 
Reviewed-by: Kefeng Wang 
Acked-by: Miaohe Lin 
Cc: David Hildenbrand 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Naoya Horiguchi 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: 
Signed-off-by: Andrew Morton

Revert "mm/hugetlbfs: update hugetlbfs to use mmap_prepare"

2026-05-22T02:06:11+00:00

This reverts commit ea52cb24cd3f ("mm/hugetlbfs: update hugetlbfs to use
mmap_prepare") with conflict resolution to account for changes in commit
ea52cb24cd3f ("mm/hugetlbfs: update hugetlbfs to use mmap_prepare").

The patch incorrectly handled hugetlb VMA lock allocation at the
mmap_prepare stage, where a failed allocation occurring after mmap_prepare
is called might result in the lock leaking.

There is no risk of a merge causing a similar issues, as
VMA_DONTEXPAND_BIT is set for hugetlb mappings.

As a first step in addressing this issue, simply revert the change so we
can rework how we do this having corrected the underlying issues.

We maintain the VMA flags changes as best we can, accounting for the fact
that we were working with a VMA descriptor previously and propagating
like-for-like changes for this.

Note that we invoke vma_set_flags() and do not call vma_start_write() as
vm_flags_set() does.  This is OK as it's being done in an .mmap hook where
the VMA is not yet linked into the tree so nobody else can be accessing
it.

Link: https://lore.kernel.org/20260512160643.266960-1-ljs@kernel.org
Fixes: ea52cb24cd3f ("mm/hugetlbfs: update hugetlbfs to use mmap_prepare")
Signed-off-by: Lorenzo Stoakes 
Reported-by: Mingyu Wang <25181214217@stu.xidian.edu.cn>
Closes: https://lore.kernel.org/linux-mm/20260425070700.562229-1-25181214217@stu.xidian.edu.cn/
Acked-by: Muchun Song 
Acked-by: Oscar Salvador 
Cc: David Hildenbrand 
Cc: Liam R. Howlett 
Cc: Pedro Falcato 
Cc: 
Signed-off-by: Andrew Morton