linux.git/mm/util.c, branch v6.17

mm/util: introduce snapshot_page()

2025-07-25T02:12:35+00:00

This commit refactors __dump_page() into snapshot_page().

snapshot_page() tries to take a faithful snapshot of a page and its folio
representation.  The snapshot is returned in the struct page_snapshot
parameter along with additional flags that are best retrieved at snapshot
creation time to reduce race windows.

This function is intended to be used by callers that need a stable
representation of a struct page and struct folio so that pointers or page
information doesn't change while working on a page.

The idea and original implementation of snapshot_page() comes from Matthew
Wilcox with suggestions for improvements from David Hildenbrand.  All bugs
and misconceptions are mine.

[luizcap@redhat.com: fix set_ps_flags() commentary]
  Link: https://lkml.kernel.org/r/d5c75701-b353-4536-a306-187fab0655b3@redhat.com
Link: https://lkml.kernel.org/r/637a03a05cb2e3df88f84ff9e9f9642374ef813a.1752499009.git.luizcap@redhat.com
Signed-off-by: Luiz Capitulino 
Reviewed-by: Shivank Garg 
Tested-by: Harry Yoo 
Acked-by: David Hildenbrand 
Cc: Matthew Wilcox (Oracle) 
Cc: Oscar Salvador 
Cc: SeongJae Park 
Signed-off-by: Andrew Morton

mm: remove boolean output parameters from folio_pte_batch_ext()

2025-07-20T01:59:46+00:00

Instead, let's just allow for specifying through flags whether we want to
have bits merged into the original PTE.

For the madvise() case, simplify by having only a single parameter for
merging young+dirty.  For madvise_cold_or_pageout_pte_range() merging the
dirty bit is not required, but also not harmful.  This code is not that
performance critical after all to really force all micro-optimizations.

As we now have two pte_t * parameters, use PageTable() to make sure we are
actually given a pointer at a copy of the PTE, not a pointer into an
actual page table.

Link: https://lkml.kernel.org/r/20250702104926.212243-5-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Dev Jain 
Reviewed-by: Oscar Salvador 
Cc: Alistair Popple 
Cc: Byungchul Park 
Cc: Gregory Price 
Cc: "Huang, Ying" 
Cc: Jann Horn 
Cc: Joshua Hahn 
Cc: Lance Yang 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes 
Cc: Mathew Brost 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Rakie Kim 
Cc: Rik van Riel 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm: split folio_pte_batch() into folio_pte_batch() and folio_pte_batch_flags()

2025-07-20T01:59:45+00:00

Many users (including upcoming ones) don't really need the flags etc, and
can live with the possible overhead of a function call.

So let's provide a basic, non-inlined folio_pte_batch(), to avoid code
bloat while still providing a variant that optimizes out all flag checks
at runtime.  folio_pte_batch_flags() will get inlined into
folio_pte_batch(), optimizing out any conditionals that depend on input
flags.

folio_pte_batch() will behave like folio_pte_batch_flags() when no flags
are specified.  It's okay to add new users of folio_pte_batch_flags(), but
using folio_pte_batch() if applicable is preferred.

So, before this change, folio_pte_batch() was inlined into the C file
optimized by propagating constants within the resulting object file.

With this change, we now also have a folio_pte_batch() that is optimized
by propagating all constants.  But instead of having one instance per
object file, we have a single shared one.

In zap_present_ptes(), where we care about performance, the compiler
already seem to generate a call to a common inlined folio_pte_batch()
variant, shared with fork() code.  So calling the new non-inlined variant
should not make a difference.

While at it, drop the "addr" parameter that is unused.

Link: https://lkml.kernel.org/r/20250702104926.212243-4-david@redhat.com
Signed-off-by: David Hildenbrand 
Suggested-by: Andrew Morton 
Link: https://lore.kernel.org/linux-mm/20250503182858.5a02729fcffd6d4723afcfc2@linux-foundation.org/
Reviewed-by: Oscar Salvador 
Reviewed-by: Zi Yan 
Reviewed-by: Dev Jain 
Cc: Alistair Popple 
Cc: Byungchul Park 
Cc: Gregory Price 
Cc: "Huang, Ying" 
Cc: Jann Horn 
Cc: Joshua Hahn 
Cc: Lance Yang 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes 
Cc: Mathew Brost 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Rakie Kim 
Cc: Rik van Riel 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton

mm: rename PAGE_MAPPING_* to FOLIO_MAPPING_*

2025-07-13T23:38:31+00:00

Now that the mapping flags are only used for folios, let's rename the
defines.

Link: https://lkml.kernel.org/r/20250704102524.326966-27-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Lorenzo Stoakes 
Reviewed-by: Harry Yoo 
Cc: Alistair Popple 
Cc: Al Viro 
Cc: Arnd Bergmann 
Cc: Brendan Jackman 
Cc: Byungchul Park 
Cc: Chengming Zhou 
Cc: Christian Brauner 
Cc: Christophe Leroy 
Cc: Eugenio Pé rez 
Cc: Greg Kroah-Hartman 
Cc: Gregory Price 
Cc: "Huang, Ying" 
Cc: Jan Kara 
Cc: Jason Gunthorpe 
Cc: Jason Wang 
Cc: Jerrin Shaji George 
Cc: Johannes Weiner 
Cc: John Hubbard 
Cc: Jonathan Corbet 
Cc: Joshua Hahn 
Cc: Liam Howlett 
Cc: Madhavan Srinivasan 
Cc: Mathew Brost 
Cc: Matthew Wilcox (Oracle) 
Cc: Miaohe Lin 
Cc: Michael Ellerman 
Cc: "Michael S. Tsirkin" 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Minchan Kim 
Cc: Naoya Horiguchi 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Peter Xu 
Cc: Qi Zheng 
Cc: Rakie Kim 
Cc: Rik van Riel 
Cc: Sergey Senozhatsky 
Cc: Shakeel Butt 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Xuan Zhuo 
Cc: xu xin 
Signed-off-by: Andrew Morton

mm: add mmap_prepare() compatibility layer for nested file systems

2025-06-13T04:39:02+00:00

Nested file systems, that is those which invoke call_mmap() within their
own f_op->mmap() handlers, may encounter underlying file systems which
provide the f_op->mmap_prepare() hook introduced by commit c84bf6dd2b83
("mm: introduce new .mmap_prepare() file callback").

We have a chicken-and-egg scenario here - until all file systems are
converted to using .mmap_prepare(), we cannot convert these nested
handlers, as we can't call f_op->mmap from an .mmap_prepare() hook.

So we have to do it the other way round - invoke the .mmap_prepare() hook
from an .mmap() one.

in order to do so, we need to convert VMA state into a struct vm_area_desc
descriptor, invoking the underlying file system's f_op->mmap_prepare()
callback passing a pointer to this, and then setting VMA state accordingly
and safely.

This patch achieves this via the compat_vma_mmap_prepare() function, which
we invoke from call_mmap() if f_op->mmap_prepare() is specified in the
passed in file pointer.

We place the fundamental logic into mm/vma.h where VMA manipulation
belongs.  We also update the VMA userland tests to accommodate the
changes.

The compat_vma_mmap_prepare() function and its associated machinery is
temporary, and will be removed once the conversion of file systems is
complete.

We carefully place this code so it can be used with CONFIG_MMU and also
with cutting edge nommu silicon.

[akpm@linux-foundation.org: export compat_vma_mmap_prepare tp fix build]
[lorenzo.stoakes@oracle.com: remove unused declarations]
  Link: https://lkml.kernel.org/r/ac3ae324-4c65-432a-8c6d-2af988b18ac8@lucifer.local
Link: https://lkml.kernel.org/r/20250609165749.344976-1-lorenzo.stoakes@oracle.com
Fixes: c84bf6dd2b83 ("mm: introduce new .mmap_prepare() file callback").
Signed-off-by: Lorenzo Stoakes 
Reported-by: Jann Horn 
Closes: https://lore.kernel.org/linux-mm/CAG48ez04yOEVx1ekzOChARDDBZzAKwet8PEoPM4Ln3_rk91AzQ@mail.gmail.com/
Reviewed-by: Pedro Falcato 
Reviewed-by: Vlastimil Babka 
Cc: Al Viro 
Cc: Christian Brauner 
Cc: Jan Kara 
Cc: Jann Horn 
Cc: Liam Howlett 
Signed-off-by: Andrew Morton

Merge tag 'sysctl-6.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl

2025-03-27T04:02:05+00:00

Pull sysctl updates from Joel Granados:

 - Move vm_table members out of kernel/sysctl.c

   All vm_table array members have moved to their respective subsystems
   leading to the removal of vm_table from kernel/sysctl.c. This
   increases modularity by placing the ctl_tables closer to where they
   are actually used and at the same time reducing the chances of merge
   conflicts in kernel/sysctl.c.

 - ctl_table range fixes

   Replace the proc_handler function that checks variable ranges in
   coredump_sysctls and vdso_table with the one that actually uses the
   extra{1,2} pointers as min/max values. This tightens the range of the
   values that users can pass into the kernel effectively preventing
   {under,over}flows.

 - Misc fixes

   Correct grammar errors and typos in test messages. Update sysctl
   files in MAINTAINERS. Constified and removed array size in
   declaration for alignment_tbl

* tag 'sysctl-6.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl: (22 commits)
  selftests/sysctl: fix wording of help messages
  selftests: fix spelling/grammar errors in sysctl/sysctl.sh
  MAINTAINERS: Update sysctl file list in MAINTAINERS
  sysctl: Fix underflow value setting risk in vm_table
  coredump: Fixes core_pipe_limit sysctl proc_handler
  sysctl: remove unneeded include
  sysctl: remove the vm_table
  sh: vdso: move the sysctl to arch/sh/kernel/vsyscall/vsyscall.c
  x86: vdso: move the sysctl to arch/x86/entry/vdso/vdso32-setup.c
  fs: dcache: move the sysctl to fs/dcache.c
  sunrpc: simplify rpcauth_cache_shrink_count()
  fs: drop_caches: move sysctl to fs/drop_caches.c
  fs: fs-writeback: move sysctl to fs/fs-writeback.c
  mm: nommu: move sysctl to mm/nommu.c
  security: min_addr: move sysctl to security/min_addr.c
  mm: mmap: move sysctl to mm/mmap.c
  mm: util: move sysctls to mm/util.c
  mm: vmscan: move vmscan sysctls to mm/vmscan.c
  mm: swap: move sysctl to mm/swap.c
  mm: filemap: move sysctl to mm/filemap.c
  ...

Merge tag 'slab-for-6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab

2025-03-24T23:15:47+00:00

Pull slab updates from Vlastimil Babka:

 - Move the TINY_RCU kvfree_rcu() implementation from RCU to SLAB
   subsystem and cleanup its integration (Vlastimil Babka)

   Following the move of the TREE_RCU batching kvfree_rcu()
   implementation in 6.14, move also the simpler TINY_RCU variant.
   Refactor the #ifdef guards so that the simple implementation is also
   used with SLUB_TINY.

   Remove the need for RCU to recognize fake callback function pointers
   (__is_kvfree_rcu_offset()) when handling call_rcu() by implementing a
   callback that calculates the object's address from the embedded
   rcu_head address without knowing its offset.

 - Improve kmalloc cache randomization in kvmalloc (GONG Ruiqi)

   Due to an extra layer of function call, all kvmalloc() allocations
   used the same set of random caches. Thanks to moving the kvmalloc()
   implementation to slub.c, this is improved and randomization now
   works for kvmalloc.

 - Various improvements to debugging, testing and other cleanups (Hyesoo
   Yu, Lilith Gkini, Uladzislau Rezki, Matthew Wilcox, Kevin Brodsky, Ye
   Bin)

* tag 'slab-for-6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab:
  slub: Handle freelist cycle in on_freelist()
  mm/slab: call kmalloc_noprof() unconditionally in kmalloc_array_noprof()
  slab: Mark large folios for debugging purposes
  kunit, slub: Add test_kfree_rcu_wq_destroy use case
  mm, slab: cleanup slab_bug() parameters
  mm: slub: call WARN() when detecting a slab corruption
  mm: slub: Print the broken data before restoring them
  slab: Achieve better kmalloc caches randomization in kvmalloc
  slab: Adjust placement of __kvmalloc_node_noprof
  mm/slab: simplify SLAB_* flag handling
  slab: don't batch kvfree_rcu() with SLUB_TINY
  rcu, slab: use a regular callback function for kvfree_rcu
  rcu: remove trace_rcu_kvfree_callback
  slab, rcu: move TINY_RCU variant of kvfree_rcu() to SLAB

fsnotify: add pre-content hooks on mmap()

2025-03-12T16:57:32+00:00

Pre-content hooks in page faults introduces potential deadlock of HSM
handler in userspace with filesystem freezing.

The requirement with pre-content event is that for every accessed file
range an event covering at least this range will be generated at least
once before the file data is accesses.

In preparation to disabling pre-content event hooks on page faults,
add pre-content hooks at mmap() variants for the entire mmaped range,
so HSM can fill content when user requests to map a portion of the file.

Note that exec() variant also calls vm_mmap_pgoff() internally to map
code sections, so pre-content hooks are also generated in this case.

Link: https://lore.kernel.org/linux-fsdevel/7ehxrhbvehlrjwvrduoxsao5k3x4aw275patsb3krkwuq573yv@o2hskrfawbnc/
Suggested-by: Josef Bacik 
Signed-off-by: Amir Goldstein 
Signed-off-by: Jan Kara 
Link: https://patch.msgid.link/20250312073852.2123409-2-amir73il@gmail.com

slab: Adjust placement of __kvmalloc_node_noprof

2025-03-04T07:53:50+00:00

Move __kvmalloc_node_noprof (as well as kvfree*, kvrealloc_noprof and
kmalloc_gfp_adjust for consistency) into mm/slub.c so that it can
directly invoke __do_kmalloc_node, which is needed for the next patch.

No functional changes intended.

Signed-off-by: GONG Ruiqi 
Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Signed-off-by: Vlastimil Babka

mm: util: move sysctls to mm/util.c

2025-02-07T15:53:04+00:00

This moves all util related sysctls to mm/util.c, as part of the
kernel/sysctl.c cleaning, also removes redundant external
variable declarations and function declarations.

Signed-off-by: Kaixiong Yu 
Reviewed-by: Kees Cook 
Reviewed-by: Jeff Layton 
Signed-off-by: Joel Granados