linux.git/mm/huge_memory.c, branch v6.16

mm: convert track_pfn_insert() to pfnmap_setup_cachemode*()

2025-05-22T21:55:36+00:00

...  by factoring it out from track_pfn_remap() into
pfnmap_setup_cachemode() and provide pfnmap_setup_cachemode_pfn() as a
replacement for track_pfn_insert().

For PMDs/PUDs, we keep checking a single pfn only.  Add some
documentation, and also document why it is valid to not check the whole
pfn range.

We'll reuse pfnmap_setup_cachemode() from core MM next.

Link: https://lkml.kernel.org/r/20250512123424.637989-3-david@redhat.com
Signed-off-by: David Hildenbrand 
Acked-by: Ingo Molnar 	[x86 bits]
Reviewed-by: Lorenzo Stoakes 
Reviewed-by: Liam R. Howlett 
Cc: Andy Lutomirski 
Cc: Borislav Betkov 
Cc: Dave Airlie 
Cc: "H. Peter Anvin" 
Cc: Jani Nikula 
Cc: Jann Horn 
Cc: Jonas Lahtinen 
Cc: "Masami Hiramatsu (Google)" 
Cc: Mathieu Desnoyers 
Cc: Peter Xu 
Cc: Peter Zijlstra 
Cc: Rodrigo Vivi 
Cc: Steven Rostedt 
Cc: Thomas Gleinxer 
Cc: Tvrtko Ursulin 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton

mm/huge_memory: remove useless folio pointers passing

2025-05-13T06:50:34+00:00

Since the previous commit "mm/huge_memory: Adjust try_to_migrate_one() and
split_huge_pmd_locked()" has simplified the logic by leveraging the folio
verification in page_vma_mapped_walk(), this patch removes the unnecessary
folio pointers passing.

Link: https://lkml.kernel.org/r/20250425103859.825879-3-gavinguo@igalia.com
Link: https://lore.kernel.org/all/98d1d195-7821-4627-b518-83103ade56c0@redhat.com/
Link: https://lore.kernel.org/all/91599a3c-e69e-4d79-bac5-5013c96203d7@redhat.com/
Signed-off-by: Gavin Guo 
Suggested-by: David Hildenbrand 
Acked-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Baolin Wang 
Cc: Florent Revest 
Cc: Gavin Shan 
Cc: Hugh Dickins 
Cc: Matthew Wilcox (Oracle) 
Cc: Miaohe Lin 
Signed-off-by: Andrew Morton

mm/huge_memory: adjust try_to_migrate_one() and split_huge_pmd_locked()

2025-05-13T06:50:33+00:00

Patch series "Clean up split_huge_pmd_locked() and remove unnecessary
folio pointers", v2.

The patch series enhances the folio verification by leveraging the
existing page_vma_mapped_walk() mechanism and removing redundant folio
pointer passing.


This patch (of 2):

split_huge_pmd_locked() currently performs redundant checks for migration
entries and folio validation that are already handled by the
page_vma_mapped_walk mechanism in try_to_migrate_one.

Specifically, page_vma_mapped_walk already ensures that:
- The folio is properly mapped in the given VMA area
- pmd_trans_huge, pmd_devmap, and migration entry validation are
  performed

To leverage page_vma_mapped_walk's work, moving TTU_SPLIT_HUGE_PMD
handling to the while loop checking and removing these duplicate checks
from split_huge_pmd_locked.

Link: https://lkml.kernel.org/r/20250425103859.825879-1-gavinguo@igalia.com
Link: https://lkml.kernel.org/r/20250425103859.825879-2-gavinguo@igalia.com
Link: https://lore.kernel.org/all/98d1d195-7821-4627-b518-83103ade56c0@redhat.com/
Link: https://lore.kernel.org/all/91599a3c-e69e-4d79-bac5-5013c96203d7@redhat.com/
Signed-off-by: Gavin Guo 
Suggested-by: David Hildenbrand 
Acked-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Baolin Wang 
Cc: Florent Revest 
Cc: Gavin Shan 
Cc: Hugh Dickins 
Cc: Matthew Wilcox (Oracle) 
Cc: Miaohe Lin 
Signed-off-by: Andrew Morton

mm: convert free_page_and_swap_cache() to free_folio_and_swap_cache()

2025-05-12T00:48:32+00:00

free_page_and_swap_cache() takes a struct page pointer as input parameter,
but it will immediately convert it to folio and all operations following
within use folio instead of page.  It makes more sense to pass in folio
directly.

Convert free_page_and_swap_cache() to free_folio_and_swap_cache() to
consume folio directly.

Link: https://lkml.kernel.org/r/20250416201720.41678-1-nifan.cxl@gmail.com
Signed-off-by: Fan Ni 
Acked-by: Davidlohr Bueso 
Acked-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Vishal Moola (Oracle) 
Reviewed-by: Matthew Wilcox (Oracle) 
Cc: Adam Manzanares 
Cc: "Aneesh Kumar K.V" 
Cc: Heiko Carstens 
Cc: Luis Chamberalin 
Cc: Vasily Gorbik 
Cc: Will Deacon 
Signed-off-by: Andrew Morton

mm: huge_memory: add folio_mark_accessed() when zapping file THP

2025-05-12T00:48:26+00:00

When investigating performance issues during file folio unmap, I noticed
some behavioral differences in handling non-PMD-sized folios and PMD-sized
folios.  For non-PMD-sized file folios, it will call folio_mark_accessed()
to mark the folio as having seen activity, but this is not done for
PMD-sized folios.

This might not cause obvious issues, but a potential problem could be
that, it might lead to reclaim of hot file folios under memory pressure,
as quoted from Johannes:

: Sometimes file contents are only accessed through relatively short-lived
: mappings. But they can nevertheless be accessed a lot and be hot. It's
: important to not lose that information on unmap, and end up kicking out a
: frequently used cache page.

Therefore, we should also add folio_mark_accessed() for PMD-sized file
folios when unmapping.

[baolin.wang@linux.alibaba.com: add comment]
  Link: https://lkml.kernel.org/r/23fdc11d-e983-4627-89a8-79e9ecf9a45a@linux.alibaba.com
Link: https://lkml.kernel.org/r/fc117f60d7b686f87067f36a0ef7cdbc3a78109c.1744190345.git.baolin.wang@linux.alibaba.com
Signed-off-by: Baolin Wang 
Acked-by: Johannes Weiner 
Acked-by: Zi Yan 
Acked-by: David Hildenbrand 
Reviewed-by: Oscar Salvador 
Cc: Barry Song <21cnbao@gmail.com>
Cc: Matthew Wilcox (Oracle) 
Cc: Ryan Roberts 
Signed-off-by: Andrew Morton

mm: add folio_mk_pmd()

2025-05-12T00:48:04+00:00

Removes five conversions from folio to page.  Also removes both callers of
mk_pmd() that aren't part of mk_huge_pmd(), getting us a step closer to
removing the confusion between mk_pmd(), mk_huge_pmd() and pmd_mkhuge().

Link: https://lkml.kernel.org/r/20250402181709.2386022-11-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) 
Reviewed-by: Zi Yan 
Cc: Alexander Gordeev 
Cc: Andreas Larsson 
Cc: Anton Ivanov 
Cc: Dave Hansen 
Cc: David Hildenbrand 
Cc: "David S. Miller" 
Cc: Geert Uytterhoeven 
Cc: Johannes Berg 
Cc: Muchun Song 
Cc: Richard Weinberger 
Cc: 
Signed-off-by: Andrew Morton

mm/huge_memory: fix dereferencing invalid pmd migration entry

2025-05-08T06:39:38+00:00

When migrating a THP, concurrent access to the PMD migration entry during
a deferred split scan can lead to an invalid address access, as
illustrated below.  To prevent this invalid access, it is necessary to
check the PMD migration entry and return early.  In this context, there is
no need to use pmd_to_swp_entry and pfn_swap_entry_to_page to verify the
equality of the target folio.  Since the PMD migration entry is locked, it
cannot be served as the target.

Mailing list discussion and explanation from Hugh Dickins: "An anon_vma
lookup points to a location which may contain the folio of interest, but
might instead contain another folio: and weeding out those other folios is
precisely what the "folio != pmd_folio((*pmd)" check (and the "risk of
replacing the wrong folio" comment a few lines above it) is for."

BUG: unable to handle page fault for address: ffffea60001db008
CPU: 0 UID: 0 PID: 2199114 Comm: tee Not tainted 6.14.0+ #4 NONE
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:split_huge_pmd_locked+0x3b5/0x2b60
Call Trace:

try_to_migrate_one+0x28c/0x3730
rmap_walk_anon+0x4f6/0x770
unmap_folio+0x196/0x1f0
split_huge_page_to_list_to_order+0x9f6/0x1560
deferred_split_scan+0xac5/0x12a0
shrinker_debugfs_scan_write+0x376/0x470
full_proxy_write+0x15c/0x220
vfs_write+0x2fc/0xcb0
ksys_write+0x146/0x250
do_syscall_64+0x6a/0x120
entry_SYSCALL_64_after_hwframe+0x76/0x7e

The bug is found by syzkaller on an internal kernel, then confirmed on
upstream.

Link: https://lkml.kernel.org/r/20250421113536.3682201-1-gavinguo@igalia.com
Link: https://lore.kernel.org/all/20250414072737.1698513-1-gavinguo@igalia.com/
Link: https://lore.kernel.org/all/20250418085802.2973519-1-gavinguo@igalia.com/
Fixes: 84c3fc4e9c56 ("mm: thp: check pmd migration entry in common path")
Signed-off-by: Gavin Guo 
Acked-by: David Hildenbrand 
Acked-by: Hugh Dickins 
Acked-by: Zi Yan 
Reviewed-by: Gavin Shan 
Cc: Florent Revest 
Cc: Matthew Wilcox (Oracle) 
Cc: Miaohe Lin 
Cc: 
Signed-off-by: Andrew Morton

mm: separate folio_split_memcg_refs() from split_page_memcg()

2025-03-22T05:03:11+00:00

Patch series "Minor memcg cleanups & prep for memdescs", v2.

Separate the handling of accounted folios and GFP_ACCOUNT pages for easier
to understand code.  For more detail, see
https://lore.kernel.org/linux-mm/Z9LwTOudOlCGny3f@casper.infradead.org/


This patch (of 5):

Folios always use memcg_data to refer to the mem_cgroup while pages
allocated with GFP_ACCOUNT have a pointer to the obj_cgroup.  Since the
caller already knows what it has, split the function into two and then we
don't need to check.

Move the assignment of split folio memcg_data to the point where we set up
the other parts of the new folio.  That leaves folio_split_memcg_refs()
just handling the memcg accounting.

Link: https://lkml.kernel.org/r/20250314133617.138071-1-willy@infradead.org
Link: https://lkml.kernel.org/r/20250314133617.138071-2-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) 
Acked-by: Johannes Weiner 
Acked-by: Shakeel Butt 
Acked-by: Zi Yan 
Acked-by: Roman Gushchin 
Cc: David Hildenbrand 
Cc: Matthew Wilcow (Oracle) 
Cc: Michal Hocko 
Cc: Muchun Song 
Signed-off-by: Andrew Morton

mm: convert lru_add_page_tail() to lru_add_split_folio()

2025-03-18T05:07:06+00:00

Remove three hidden calls to compound_head() and accesses to page->lru.

Link: https://lkml.kernel.org/r/20250313151458.4145978-1-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) 
Reviewed-by: Zi Yan 
Signed-off-by: Andrew Morton

mm/truncate: use folio_split() in truncate operation

2025-03-18T05:07:00+00:00

Instead of splitting the large folio uniformly during truncation, try to
use buddy allocator like folio_split() at the start and the end of a
truncation range to minimize the number of resulting folios if it is
supported.  try_folio_split() is introduced to use folio_split() if
supported and it falls back to uniform split otherwise.

For example, to truncate a order-4 folio
[0, 1, 2, 3, 4, 5, ..., 15]
between [3, 10] (inclusive), folio_split() splits the folio at 3 to
[0,1], [2], [3], [4..7], [8..15] and [3], [4..7] can be dropped and
[8..15] is kept with zeros in [8..10], then another folio_split() is
done at 10, so [8..10] can be dropped.

One possible optimization is to make folio_split() to split a folio based
on a given range, like [3..10] above.  But that complicates folio_split(),
so it will be investigated when necessary.

Link: https://lkml.kernel.org/r/20250226210032.2044041-8-ziy@nvidia.com
Link: https://lkml.kernel.org/r/20250307174001.242794-8-ziy@nvidia.com
Signed-off-by: Zi Yan 
Cc: Baolin Wang 
Cc: David Hildenbrand 
Cc: Hugh Dickins 
Cc: John Hubbard 
Cc: Kefeng Wang 
Cc: Kirill A. Shuemov 
Cc: Matthew Wilcox 
Cc: Miaohe Lin 
Cc: Ryan Roberts 
Cc: Yang Shi 
Cc: Yu Zhao 
Cc: Kairui Song 
Signed-off-by: Andrew Morton