linux-stable.git/mm/huge_memory.c, branch linux-rolling-stable

mm/huge_memory: preserve pmd_swp_uffd_wp on device-private PMD downgrade

2026-07-24T14:21:01+00:00

commit f7e2c21bd1f57cd5350eecdfdb5d6025ca6afbab upstream.

change_non_present_huge_pmd() rewrites a writable device-private PMD swap
entry into a readable one without carrying pmd_swp_uffd_wp() across.  The
PTE-level change_softleaf_pte() does this correctly; mirror that here,
matching what copy_huge_pmd() does for the fork path.  Without the carry,
a plain mprotect() over a UFFD_WP-marked device-private THP strips the bit
and the trap is bypassed on swap-in.

Link: https://lore.kernel.org/20260529172331.356655-5-kas@kernel.org
Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations")
Signed-off-by: Kiryl Shutsemau 
Reported-by: Sashiko AI review 
Reviewed-by: Balbir Singh 
Cc: David Hildenbrand 
Cc: Lorenzo Stoakes 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Peter Xu 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Greg Kroah-Hartman

mm: preserve PG_dropbehind flag during folio split

2026-07-24T14:19:31+00:00

[ Upstream commit 0ad186cf167fdbeb6b29a8005c71973781036bd3 ]

__split_folio_to_order() copies page flags from the original folio to
newly created sub-folios using an explicit allowlist, but PG_dropbehind
is not included. When a large folio with PG_dropbehind set is split,
only the head sub-folio retains the flag; all tail sub-folios silently
lose it and will not be reclaimed eagerly after writeback completes.

Add PG_dropbehind to the flag copy mask so that the drop-behind hint
is preserved across folio splits.

Fixes: a323281cdfec ("mm: add PG_dropbehind folio flag")
Assisted-by: Claude:claude-opus-4-6
Signed-off-by: Jeff Layton 
Link: https://patch.msgid.link/20260511-dontcache-v7-1-2848ddce8090@kernel.org
Acked-by: David Hildenbrand (Arm) 
Reviewed-by: Jan Kara 
Signed-off-by: Christian Brauner (Amutable) 
Signed-off-by: Sasha Levin

mm/huge_memory: use correct flags for device private PMD entry

2026-06-03T23:25:51+00:00

Commit 65edfda6f3f2 ("mm/rmap: extend rmap and migration support
device-private entries") updated set_pmd_migration_entry() to use
pmdp_huge_get_and_clear() in the softleaf case, but made no further
adjustments to the function itself.

Therefore this function continues to incorrectly use pmd_write(),
pmd_soft_dirty() and pmd_uffd_wp() to determine whether the installed
migration entry should be marked writable, softdirty or uffd-wp
respectively.

Whilst all are incorrect, the most problematic of these is pmd_write(), as
this can lead to corrupted rmap state.

On x86-64 _PAGE_SWP_SOFT_DIRTY is aliased to _PAGE_RW.  So calling
pmd_write() on a softleaf will return the softdirty state encoded in the
entry, assuming CONFIG_MEM_SOFT_DIRTY was enabled.

This was observed when running the hmm.hmm_device_private.anon_write_child
selftest:

1. The test faults in a range then migrates it such that a device-private
   THP range is established.

2. The parent then migrates it to a device-private writable PMD entry whose
   folio is entirely AnonExclusive with entire_mapcount=1, softdirty set
   (accidentally correct write state).

3. The parent forks and the PMD entries are set to device-private read only
   entries, entire_mapcount=2, softdirty still set.

4. [BUG] The child writes to the range then migrates to RAM - intending to
   install non-writable migration entries - but replacing parent and child
   PMD mappings with WRITABLE entries due to misinterpreting the softdirty
   bit.

5. In remove_migration_pmd(), if !softleaf_is_migration_read(entry) we
   set the RMAP_EXCLUSIVE flag when calling folio_add_anon_rmap_pmd() for
   both parent and child, which are therefore AnonExclusive.

6. [SPLAT] Child sets migrated folio entire_mapcount=1, parent sets
   entire_mapcount=2 and we end up with an AnonExclusive folio with
   entire_mapcount=2! Assert fires in __folio_add_anon_rmap():

		VM_WARN_ON_FOLIO(folio_test_large(folio) &&
				 folio_entire_mapcount(folio) > 1 &&
				 PageAnonExclusive(cur_page), folio)

This patch fixes the issue by correctly referencing the softleaf entry
fields for writable, softdirty and uffd-wp in set_pmd_migration_entry().

It also only updates A/D flags if the entry is present as these are
otherwise not meaningful for a softleaf entry.

This patch also flips the if (!present) { ...  } else { ...  } logic in
set_pmd_migration_entry() so it is easier to understand, and adds some
comments to make things clearer.

I was able to bisect this to commit 775465fd26a3 ("lib/test_hmm: add zone
device private THP test infrastructure") which first exposes this bug as
it was the commit that permitted test_hmm to generate the test.

However commit 65edfda6f3f2 ("mm/rmap: extend rmap and migration support
device-private entries") is the commit that actually enabled this
behaviour.

Link: https://lore.kernel.org/20260601083044.57132-1-ljs@kernel.org
Fixes: 65edfda6f3f2 ("mm/rmap: extend rmap and migration support device-private entries")
Signed-off-by: Lorenzo Stoakes 
Acked-by: David Hildenbrand (Arm) 
Reviewed-by: Dev Jain 
Reviewed-by: Balbir Singh 
Reviewed-by: Baolin Wang 
Reviewed-by: Oscar Salvador (SUSE) 
Reviewed-by: Barry Song 
Reviewed-by: Lance Yang 
Reviewed-by: Zi Yan 
Cc: Baolin Wang 
Cc: Liam R. Howlett 
Cc: Nico Pache 
Cc: Ryan Roberts 
Cc: SeongJae Park 
Cc: Wei Yang 
Cc: 
Signed-off-by: Andrew Morton

mm/huge_memory: update file PMD counter before folio_put()

2026-06-01T04:50:25+00:00

__split_huge_pmd_locked() updates the file/shmem RSS counter after
dropping the PMD mapping's folio reference.  If folio_put() drops the last
reference, mm_counter_file() can later read freed folio state via
folio_test_swapbacked().

Move the counter update before folio_put().

Link: https://lore.kernel.org/20260526101337.1984081-1-yintirui@huawei.com
Fixes: fadae2953072 ("thp: use mm_file_counter to determine update which rss counter")
Signed-off-by: Yin Tirui 
Reviewed-by: Lorenzo Stoakes 
Acked-by: David Hildenbrand (arm) 
Reviewed-by: Lance Yang 
Reviewed-by: Dev Jain 
Cc: Baolin Wang 
Cc: Barry Song 
Cc: Chen Jun 
Cc: Kefeng Wang 
Cc: Liam R. Howlett 
Cc: Nico Pache 
Cc: Ryan Roberts 
Cc: Vlastimil Babka 
Cc: Yang Shi 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton

mm/huge_memory: update file PUD counter before folio_put()

2026-06-01T04:50:25+00:00

__split_huge_pud_locked() updates the file/shmem RSS counter after
dropping the PUD mapping's folio reference.  If folio_put() drops the last
reference, mm_counter_file() can later read freed folio state via
folio_test_swapbacked().

Move the counter update before folio_put().

Link: https://lore.kernel.org/20260526101355.1984244-1-yintirui@huawei.com
Fixes: dbe54153296d ("mm/huge_memory: add vmf_insert_folio_pud()")
Signed-off-by: Yin Tirui 
Reviewed-by: Lorenzo Stoakes 
Acked-by: David Hildenbrand (arm) 
Reviewed-by: Lance Yang 
Reviewed-by: Dev Jain 
Cc: Alistair Popple 
Cc: Baolin Wang 
Cc: Barry Song 
Cc: Chen Jun 
Cc: Kefeng Wang 
Cc: Liam R. Howlett 
Cc: Nico Pache 
Cc: Ryan Roberts 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton

mm: thp: prevent memory cgroup release in folio_split_queue_lock{_irqsave}()

2026-04-18T07:10:46+00:00

In the near future, a folio will no longer pin its corresponding memory
cgroup.  To ensure safety, it will only be appropriate to hold the rcu
read lock or acquire a reference to the memory cgroup returned by
folio_memcg(), thereby preventing it from being released.

In the current patch, the rcu read lock is employed to safeguard against
the release of the memory cgroup in folio_split_queue_lock{_irqsave}().

Link: https://lore.kernel.org/ca2957c0df1126b2c71b40c738018fd5255525a6.1772711148.git.zhengqi.arch@bytedance.com
Signed-off-by: Qi Zheng 
Reviewed-by: Harry Yoo 
Acked-by: Johannes Weiner 
Acked-by: Shakeel Butt 
Acked-by: David Hildenbrand (Red Hat) 
Acked-by: Muchun Song 
Cc: Allen Pais 
Cc: Axel Rasmussen 
Cc: Baoquan He 
Cc: Chengming Zhou 
Cc: Chen Ridong 
Cc: Hamza Mahfooz 
Cc: Hugh Dickins 
Cc: Imran Khan 
Cc: Kamalesh Babulal 
Cc: Lance Yang 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes (Oracle) 
Cc: Michal Hocko 
Cc: Michal Koutný 
Cc: Mike Rapoport 
Cc: Muchun Song 
Cc: Nhat Pham 
Cc: Roman Gushchin 
Cc: Suren Baghdasaryan 
Cc: Usama Arif 
Cc: Vlastimil Babka 
Cc: Wei Xu 
Cc: Yosry Ahmed 
Cc: Yuanchu Xie 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm: rename unlock_page_lruvec_irq and its variants

2026-04-18T07:10:44+00:00

It is inappropriate to use folio_lruvec_lock() variants in conjunction
with unlock_page_lruvec() variants, as this involves the inconsistent
operation of locking a folio while unlocking a page.  To rectify this, the
functions unlock_page_lruvec{_irq, _irqrestore} are renamed to
lruvec_unlock{_irq,_irqrestore}.

Link: https://lore.kernel.org/4e5e05271a250df4d1812e1832be65636a78c957.1772711148.git.zhengqi.arch@bytedance.com
Signed-off-by: Muchun Song 
Signed-off-by: Qi Zheng 
Acked-by: Roman Gushchin 
Acked-by: Johannes Weiner 
Reviewed-by: Harry Yoo 
Reviewed-by: Chen Ridong 
Acked-by: David Hildenbrand (Red Hat) 
Acked-by: Shakeel Butt 
Cc: Allen Pais 
Cc: Axel Rasmussen 
Cc: Baoquan He 
Cc: Chengming Zhou 
Cc: Hamza Mahfooz 
Cc: Hugh Dickins 
Cc: Imran Khan 
Cc: Kamalesh Babulal 
Cc: Lance Yang 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes (Oracle) 
Cc: Michal Hocko 
Cc: Michal Koutný 
Cc: Mike Rapoport 
Cc: Muchun Song 
Cc: Nhat Pham 
Cc: Suren Baghdasaryan 
Cc: Usama Arif 
Cc: Vlastimil Babka 
Cc: Wei Xu 
Cc: Yosry Ahmed 
Cc: Yuanchu Xie 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm: fix deferred split queue races during migration

2026-04-05T20:53:47+00:00

migrate_folio_move() records the deferred split queue state from src and
replays it on dst.  Replaying it after remove_migration_ptes(src, dst, 0)
makes dst visible before it is requeued, so a concurrent rmap-removal path
can mark dst partially mapped and trip the WARN in deferred_split_folio().

Move the requeue before remove_migration_ptes() so dst is back on the
deferred split queue before it becomes visible again.

Because migration still holds dst locked at that point, teach
deferred_split_scan() to requeue a folio when folio_trylock() fails. 
Otherwise a fully mapped underused folio can be dequeued by the shrinker
and silently lost from split_queue.

[ziy@nvidia.com: move the comment]
  Link: https://lkml.kernel.org/r/FB71A764-0F10-4E5A-B4A0-BA4C7F138408@nvidia.com
Link: https://syzkaller.appspot.com/bug?extid=a7067a757858ac8eb085
Link: https://lkml.kernel.org/r/20260401131032.13011-1-lance.yang@linux.dev
Fixes: 8a8ca142a488 ("mm: migrate: requeue destination folio on deferred split queue")
Signed-off-by: Lance Yang 
Signed-off-by: Zi Yan 
Reported-by: syzbot+a7067a757858ac8eb085@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/linux-mm/69ccb65b.050a0220.183828.003a.GAE@google.com/
Suggested-by: David Hildenbrand (Arm) 
Acked-by: David Hildenbrand (Arm) 
Acked-by: Zi Yan 
Cc: Alistair Popple 
Cc: Baolin Wang 
Cc: Barry Song 
Cc: Byungchul Park 
Cc: David Hildenbrand 
Cc: Deepanshu Kartikey 
Cc: Dev Jain 
Cc: Gregory Price 
Cc: "Huang, Ying" 
Cc: Joshua Hahn 
Cc: Lance Yang 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes (Oracle) 
Cc: Matthew Brost 
Cc: Nico Pache 
Cc: Rakie Kim 
Cc: Ryan Roberts 
Cc: Wei Yang 
Cc: Ying Huang 
Cc: Usama Arif 
Cc: 
Signed-off-by: Andrew Morton

mm/huge_memory: add and use has_deposited_pgtable()

2026-04-05T20:53:47+00:00

Rather than thread has_deposited through zap_huge_pmd(), make things
clearer by adding has_deposited_pgtable() with comments describing why in
each case.

[ljs@kernel.org: fix folio_put()-before-recheck issue, per Sashiko]
  Link: https://lkml.kernel.org/r/0a917f80-902f-49b0-a75f-1bbaf23d7f94@lucifer.local
Link: https://lkml.kernel.org/r/f9db59ca90937e39913d50ecb4f662e2bad17bbb.1774029655.git.ljs@kernel.org
Signed-off-by: Lorenzo Stoakes (Oracle) 
Reviewed-by: Suren Baghdasaryan 
Cc: Baolin Wang 
Cc: Barry Song 
Cc: David Hildenbrand 
Cc: Dev Jain 
Cc: Lance Yang 
Cc: Liam Howlett 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Nico Pache 
Cc: Qi Zheng 
Cc: Ryan Roberts 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm/huge_memory: add and use normal_or_softleaf_folio_pmd()

2026-04-05T20:53:46+00:00

Now we have pmd_to_softleaf_folio() available to us which also raises a
CONFIG_DEBUG_VM warning if unexpectedly an invalid softleaf entry, we can
now abstract folio handling altogether.

vm_normal_folio() deals with the huge zero page (which is present), as well
as PFN map/mixed map mappings in both cases returning NULL.

Otherwise, we try to obtain the softleaf folio.

This makes the logic far easier to comprehend and has it use the standard
vm_normal_folio_pmd() path for decoding of present entries.

Finally, we have to update the flushing logic to only do so if a folio is
established.

This patch also makes the 'is_present' value more accurate - because PFN
map, mixed map and zero huge pages are present, just not present and
'normal'.

[ljs@kernel.org: avoid bisection hazard]
  Link: https://lkml.kernel.org/r/d0cc6161-77a4-42ba-a411-96c23c78df1b@lucifer.local
Link: https://lkml.kernel.org/r/c2be872d64ef9573b80727d9ab5446cf002f17b5.1774029655.git.ljs@kernel.org
Signed-off-by: Lorenzo Stoakes (Oracle) 
Reviewed-by: Suren Baghdasaryan 
Cc: Baolin Wang 
Cc: Barry Song 
Cc: David Hildenbrand 
Cc: Dev Jain 
Cc: Lance Yang 
Cc: Liam Howlett 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Nico Pache 
Cc: Qi Zheng 
Cc: Ryan Roberts 
Cc: Zi Yan 
Signed-off-by: Andrew Morton