linux.git/mm/memory.c, branch v6.13

mm: use clear_user_(high)page() for arch with special user folio handling

2024-12-19T03:04:43+00:00

Some architectures have special handling after clearing user folios:
architectures, which set cpu_dcache_is_aliasing() to true, require
flushing dcache; arc, which sets cpu_icache_is_aliasing() to true, changes
folio->flags to make icache coherent to dcache.  So __GFP_ZERO using only
clear_page() is not enough to zero user folios and clear_user_(high)page()
must be used.  Otherwise, user data will be corrupted.

Fix it by always clearing user folios with clear_user_(high)page() when
cpu_dcache_is_aliasing() is true or cpu_icache_is_aliasing() is true. 
Rename alloc_zeroed() to user_alloc_needs_zeroing() and invert the logic
to clarify its intend.

Link: https://lkml.kernel.org/r/20241209182326.2955963-2-ziy@nvidia.com
Fixes: 5708d96da20b ("mm: avoid zeroing user movable page twice with init_on_alloc=1")
Signed-off-by: Zi Yan 
Reported-by: Geert Uytterhoeven 
Closes: https://lore.kernel.org/linux-mm/CAMuHMdV1hRp_NtR5YnJo=HsfgKQeH91J537Gh4gKk3PFZhSkbA@mail.gmail.com/
Tested-by: Geert Uytterhoeven 
Acked-by: Vlastimil Babka 
Cc: Alexander Potapenko 
Cc: David Hildenbrand 
Cc: John Hubbard 
Cc: Kees Cook 
Cc: Kefeng Wang 
Cc: Mathieu Desnoyers 
Cc: Matthew Wilcox 
Cc: Miaohe Lin 
Cc: Ryan Roberts 
Cc: Vineet Gupta 
Signed-off-by: Andrew Morton

mm: use aligned address in copy_user_gigantic_page()

2024-12-19T03:04:42+00:00

In current kernel, hugetlb_wp() calls copy_user_large_folio() with the
fault address.  Where the fault address may be not aligned with the huge
page size.  Then, copy_user_large_folio() may call
copy_user_gigantic_page() with the address, while
copy_user_gigantic_page() requires the address to be huge page size
aligned.  So, this may cause memory corruption or information leak,
addtional, use more obvious naming 'addr_hint' instead of 'addr' for
copy_user_gigantic_page().

Link: https://lkml.kernel.org/r/20241028145656.932941-2-wangkefeng.wang@huawei.com
Fixes: 530dd9926dc1 ("mm: memory: improve copy_user_large_folio()")
Signed-off-by: Kefeng Wang 
Reviewed-by: David Hildenbrand 
Cc: Huang Ying 
Cc: Matthew Wilcox (Oracle) 
Cc: Muchun Song 
Cc: 
Signed-off-by: Andrew Morton

mm: use aligned address in clear_gigantic_page()

2024-12-19T03:04:42+00:00

In current kernel, hugetlb_no_page() calls folio_zero_user() with the
fault address.  Where the fault address may be not aligned with the huge
page size.  Then, folio_zero_user() may call clear_gigantic_page() with
the address, while clear_gigantic_page() requires the address to be huge
page size aligned.  So, this may cause memory corruption or information
leak, addtional, use more obvious naming 'addr_hint' instead of 'addr' for
clear_gigantic_page().

Link: https://lkml.kernel.org/r/20241028145656.932941-1-wangkefeng.wang@huawei.com
Fixes: 78fefd04c123 ("mm: memory: convert clear_huge_page() to folio_zero_user()")
Signed-off-by: Kefeng Wang 
Reviewed-by: "Huang, Ying" 
Reviewed-by: David Hildenbrand 
Cc: Matthew Wilcox (Oracle) 
Cc: Muchun Song 
Cc: 
Signed-off-by: Andrew Morton

mm: add PTE_MARKER_GUARD PTE marker

2024-11-11T08:26:44+00:00

Add a new PTE marker that results in any access causing the accessing
process to segfault.

This is preferable to PTE_MARKER_POISONED, which results in the same
handling as hardware poisoned memory, and is thus undesirable for cases
where we simply wish to 'soft' poison a range.

This is in preparation for implementing the ability to specify guard pages
at the page table level, i.e.  ranges that, when accessed, should cause
process termination.

Additionally, rename zap_drop_file_uffd_wp() to zap_drop_markers() - the
function checks the ZAP_FLAG_DROP_MARKER flag so naming it for this single
purpose was simply incorrect.

We then reuse the same logic to determine whether a zap should clear a
guard entry - this should only be performed on teardown and never on
MADV_DONTNEED or MADV_FREE.

We additionally add a WARN_ON_ONCE() in hugetlb logic should a guard
marker be encountered there, as we explicitly do not support this
operation and this should not occur.

Link: https://lkml.kernel.org/r/f47f3d5acca2dcf9bbf655b6d33f3dc713e4a4a0.1730123433.git.lorenzo.stoakes@oracle.com
Signed-off-by: Lorenzo Stoakes 
Acked-by: Vlastimil Babka 
Suggested-by: Vlastimil Babka 
Suggested-by: Jann Horn 
Suggested-by: David Hildenbrand 
Cc: Arnd Bergmann 
Cc: Christian Brauner 
Cc: Christoph Hellwig 
Cc: Chris Zankel 
Cc: Helge Deller 
Cc: James E.J. Bottomley 
Cc: Jeff Xu 
Cc: John Hubbard 
Cc: Liam R. Howlett 
Cc: Matthew Wilcox (Oracle) 
Cc: Matt Turner 
Cc: Max Filippov 
Cc: Muchun Song 
Cc: Paul E. McKenney 
Cc: Richard Henderson 
Cc: Shuah Khan 
Cc: Shuah Khan 
Cc: Sidhartha Kumar 
Cc: Suren Baghdasaryan 
Cc: Thomas Bogendoerfer 
Signed-off-by: Andrew Morton

mm/memory.c: simplify pfnmap_lockdep_assert

2024-11-07T04:11:17+00:00

Use local `mapping' to reduce the pointer chasing.

akpm: extracted from a bugfix which Linus fixed with b1b46751671be ("mm:
fix follow_pfnmap API lockdep assert").

Link: https://lkml.kernel.org/r/20241004-fix-null-deref-v4-1-d0a8ec01ac85@iiitd.ac.in
Signed-off-by: Manas 
Reviewed-by: Peter Xu 
Cc: Anup Sharma 
Cc: Shuah Khan 
Cc: Matthew Wilcox 
Signed-off-by: Andrew Morton

mm: avoid zeroing user movable page twice with init_on_alloc=1

2024-11-07T04:11:13+00:00

Commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and
init_on_free=1 boot options") forces allocated page to be zeroed in
post_alloc_hook() when init_on_alloc=1.

For order-0 folios, if arch does not define
vma_alloc_zeroed_movable_folio(), the default implementation again zeros
the page return from the buddy allocator.  So the page is zeroed twice. 
Fix it by passing __GFP_ZERO instead to avoid double page zeroing.  At the
moment, s390,arm64,x86,alpha,m68k are not impacted since they define their
own vma_alloc_zeroed_movable_folio().

For >0 order folios (mTHP and PMD THP), folio_zero_user() is called to
zero the folio again.  Fix it by calling folio_zero_user() only if
init_on_alloc is set.  All arch are impacted.

Add alloc_zeroed() helper to encapsulate the init_on_alloc check.

[ziy@nvidia.com: comment fixes, per David]
  Link: https://lkml.kernel.org/r/97DB52E1-C594-49B5-9736-89AC302FAB01@nvidia.com
Link: https://lkml.kernel.org/r/20241011150304.709590-1-ziy@nvidia.com
Signed-off-by: Zi Yan 
Acked-by: Vlastimil Babka 
Acked-by: David Hildenbrand 
Cc: Alexander Potapenko 
Cc: "Huang, Ying" 
Cc: John Hubbard 
Cc: Kees Cook 
Cc: Kefeng Wang 
Cc: Matthew Wilcox 
Cc: Miaohe Lin 
Cc: Ryan Roberts 
Signed-off-by: Andrew Morton

mm: remove unused hugepage for vma_alloc_folio()

2024-11-07T04:11:12+00:00

The hugepage parameter was deprecated since commit ddc1a5cbc05d
("mempolicy: alloc_pages_mpol() for NUMA policy without vma"), for
PMD-sized THP, it still tries only preferred node if possible in
vma_alloc_folio() by checking the order of the folio allocation.

Link: https://lkml.kernel.org/r/20241010061556.1846751-1-wangkefeng.wang@huawei.com
Signed-off-by: Kefeng Wang 
Acked-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Barry Song 
Cc: Hugh Dickins 
Cc: Matthew Wilcox 
Cc: Ryan Roberts 
Signed-off-by: Andrew Morton

mm/memory.c: remove stray newline at top of file

2024-11-07T04:11:11+00:00

Fixes: d61ea1cb0095 ("userfaultfd: UFFD_FEATURE_WP_ASYNC")
Reported-by: Jeongjun Park 
Closes: https://lkml.kernel.org/r/20241007065307.4158-1-aha310510@gmail.com
Cc: Muhammad Usama Anjum 
Cc: Peter Xu 
Cc: Greg KH 
Signed-off-by: Andrew Morton

mm: copy_pte_range() use pte_offset_map_rw_nolock()

2024-11-06T00:56:28+00:00

In copy_pte_range(), we may modify the src_pte entry after holding the
src_ptl, so convert it to using pte_offset_map_rw_nolock().  Since we
already hold the exclusive mmap_lock, and the copy_pte_range() and
retract_page_tables() are using vma->anon_vma to be exclusive, so the PTE
page is stable, there is no need to get pmdval and do pmd_same() check.

Link: https://lkml.kernel.org/r/9166f6fad806efbca72e318ab6f0f8af458056a9.1727332572.git.zhengqi.arch@bytedance.com
Signed-off-by: Qi Zheng 
Reviewed-by: Muchun Song 
Cc: David Hildenbrand 
Cc: Hugh Dickins 
Cc: Matthew Wilcox 
Cc: Mike Rapoport (Microsoft) 
Cc: Peter Xu 
Cc: Ryan Roberts 
Cc: Vishal Moola (Oracle) 
Signed-off-by: Andrew Morton

mm: handle_pte_fault() use pte_offset_map_rw_nolock()

2024-11-06T00:56:27+00:00

In handle_pte_fault(), we may modify the vmf->pte after acquiring the
vmf->ptl, so convert it to using pte_offset_map_rw_nolock().  But since we
will do the pte_same() check, so there is no need to get pmdval to do
pmd_same() check, just pass a dummy variable to it.

Link: https://lkml.kernel.org/r/af8d694853b44c5a6018403ae435440e275854c7.1727332572.git.zhengqi.arch@bytedance.com
Signed-off-by: Qi Zheng 
Acked-by: David Hildenbrand 
Reviewed-by: Muchun Song 
Cc: Hugh Dickins 
Cc: Matthew Wilcox 
Cc: Mike Rapoport (Microsoft) 
Cc: Peter Xu 
Cc: Ryan Roberts 
Cc: Vishal Moola (Oracle) 
Signed-off-by: Andrew Morton