linux-stable.git/mm/vmalloc.c, branch linux-6.15.y

mm/vmalloc: leave lazy MMU mode on PTE mapping error

2025-07-17T16:43:52+00:00

commit fea18c686320a53fce7ad62a87a3e1d10ad02f31 upstream.

vmap_pages_pte_range() enters the lazy MMU mode, but fails to leave it in
case an error is encountered.

Link: https://lkml.kernel.org/r/20250623075721.2817094-1-agordeev@linux.ibm.com
Fixes: 2ba3e6947aed ("mm/vmalloc: track which page-table levels were modified")
Signed-off-by: Alexander Gordeev 
Reported-by: kernel test robot 
Reported-by: Dan Carpenter 
Closes: https://lore.kernel.org/r/202506132017.T1l1l6ME-lkp@intel.com/
Reviewed-by: Ryan Roberts 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Greg Kroah-Hartman

mm/vmalloc: fix data race in show_numa_info()

2025-07-10T14:08:54+00:00

commit 5c5f0468d172ddec2e333d738d2a1f85402cf0bc upstream.

The following data-race was found in show_numa_info():

==================================================================
BUG: KCSAN: data-race in vmalloc_info_show / vmalloc_info_show

read to 0xffff88800971fe30 of 4 bytes by task 8289 on cpu 0:
 show_numa_info mm/vmalloc.c:4936 [inline]
 vmalloc_info_show+0x5a8/0x7e0 mm/vmalloc.c:5016
 seq_read_iter+0x373/0xb40 fs/seq_file.c:230
 proc_reg_read_iter+0x11e/0x170 fs/proc/inode.c:299
....

write to 0xffff88800971fe30 of 4 bytes by task 8287 on cpu 1:
 show_numa_info mm/vmalloc.c:4934 [inline]
 vmalloc_info_show+0x38f/0x7e0 mm/vmalloc.c:5016
 seq_read_iter+0x373/0xb40 fs/seq_file.c:230
 proc_reg_read_iter+0x11e/0x170 fs/proc/inode.c:299
....

value changed: 0x0000008f -> 0x00000000
==================================================================

According to this report,there is a read/write data-race because
m->private is accessible to multiple CPUs.  To fix this, instead of
allocating the heap in proc_vmalloc_init() and passing the heap address to
m->private, vmalloc_info_show() should allocate the heap.

Link: https://lkml.kernel.org/r/20250508165620.15321-1-aha310510@gmail.com
Fixes: 8e1d743f2c26 ("mm: vmalloc: support multiple nodes in vmallocinfo")
Signed-off-by: Jeongjun Park 
Suggested-by: Eric Dumazet 
Suggested-by: Andrew Morton 
Reviewed-by: "Uladzislau Rezki (Sony)" 
Signed-off-by: Andrew Morton 
Signed-off-by: Greg Kroah-Hartman

mm: vmalloc: only zero-init on vrealloc shrink

2025-05-25T07:53:48+00:00

The common case is to grow reallocations, and since init_on_alloc will
have already zeroed the whole allocation, we only need to zero when
shrinking the allocation.

Link: https://lkml.kernel.org/r/20250515214217.619685-2-kees@kernel.org
Fixes: a0309faf1cb0 ("mm: vmalloc: support more granular vrealloc() sizing")
Signed-off-by: Kees Cook 
Tested-by: Pawan Gupta 
Cc: Danilo Krummrich 
Cc: Eduard Zingerman 
Cc: "Erhard F." 
Cc: Shung-Hsi Yu 
Cc: "Uladzislau Rezki (Sony)" 
Cc: 
Signed-off-by: Andrew Morton

mm: vmalloc: actually use the in-place vrealloc region

2025-05-25T07:53:48+00:00

Patch series "mm: vmalloc: Actually use the in-place vrealloc region".

This fixes a performance regression[1] with vrealloc()[1].


The refactoring to not build a new vmalloc region only actually worked
when shrinking.  Actually return the resized area when it grows.  Ugh.

Link: https://lkml.kernel.org/r/20250515214217.619685-1-kees@kernel.org
Fixes: a0309faf1cb0 ("mm: vmalloc: support more granular vrealloc() sizing")
Signed-off-by: Kees Cook 
Reported-by: Shung-Hsi Yu 
Closes: https://lore.kernel.org/all/20250515-bpf-verifier-slowdown-vwo2meju4cgp2su5ckj@6gi6ssxbnfqg [1]
Tested-by: Eduard Zingerman 
Tested-by: Pawan Gupta 
Tested-by: Shung-Hsi Yu 
Reviewed-by: "Uladzislau Rezki (Sony)" 
Reviewed-by: Danilo Krummrich 
Cc: "Erhard F." 
Cc: 
Signed-off-by: Andrew Morton

mm: vmalloc: support more granular vrealloc() sizing

2025-05-08T06:39:41+00:00

Introduce struct vm_struct::requested_size so that the requested
(re)allocation size is retained separately from the allocated area size. 
This means that KASAN will correctly poison the correct spans of requested
bytes.  This also means we can support growing the usable portion of an
allocation that can already be supported by the existing area's existing
allocation.

Link: https://lkml.kernel.org/r/20250426001105.it.679-kees@kernel.org
Fixes: 3ddc2fefe6f3 ("mm: vmalloc: implement vrealloc()")
Signed-off-by: Kees Cook 
Reported-by: Erhard Furtner 
Closes: https://lore.kernel.org/all/20250408192503.6149a816@outsider.home/
Reviewed-by: Danilo Krummrich 
Cc: Michal Hocko 
Cc: "Uladzislau Rezki (Sony)" 
Cc: Vlastimil Babka 
Cc: 
Signed-off-by: Andrew Morton

mm/vmalloc: refactor __vmalloc_node_range_noprof()

2025-03-18T05:06:58+00:00

According to the code logic, the first parameter of the sub-function
__get_vm_area_node() should be size instead of real_size.

Then in __get_vm_area_node(), the size will be aligned, so the redundant
alignment operation is deleted.

The use of the real_size variable causes code redundancy, so it is removed
to simplify the code.

The real prefix is generally used to indicate the adjusted value of a
parameter, but according to the code logic, it should indicate the
original value, so it is recommended to rename it to original_align.

Link: https://lkml.kernel.org/r/20250306072131.800499-1-liuye@kylinos.cn
Signed-off-by: Liu Ye 
Reviewed-by: "Uladzislau Rezki (Sony)" 
Cc: Christop Hellwig 
Signed-off-by: Andrew Morton

mm: don't skip arch_sync_kernel_mappings() in error paths

2025-03-06T05:36:18+00:00

Fix callers that previously skipped calling arch_sync_kernel_mappings() if
an error occurred during a pgtable update.  The call is still required to
sync any pgtable updates that may have occurred prior to hitting the error
condition.

These are theoretical bugs discovered during code review.

Link: https://lkml.kernel.org/r/20250226121610.2401743-1-ryan.roberts@arm.com
Fixes: 2ba3e6947aed ("mm/vmalloc: track which page-table levels were modified")
Fixes: 0c95cba49255 ("mm: apply_to_pte_range warn and fail if a large pte is encountered")
Signed-off-by: Ryan Roberts 
Reviewed-by: Anshuman Khandual 
Reviewed-by: Catalin Marinas 
Cc: Christop Hellwig 
Cc: "Uladzislau Rezki (Sony)" 
Cc: 
Signed-off-by: Andrew Morton

mm: alloc_pages_bulk: rename API

2025-01-26T04:22:31+00:00

The previous commit removed the page_list argument from
alloc_pages_bulk_noprof() along with the alloc_pages_bulk_list() function.

Now that only the *_array() flavour of the API remains, we can do the
following renaming (along with the _noprof() ones):

  alloc_pages_bulk_array -> alloc_pages_bulk
  alloc_pages_bulk_array_mempolicy -> alloc_pages_bulk_mempolicy
  alloc_pages_bulk_array_node -> alloc_pages_bulk_node

Link: https://lkml.kernel.org/r/275a3bbc0be20fbe9002297d60045e67ab3d4ada.1734991165.git.luizcap@redhat.com
Signed-off-by: Luiz Capitulino 
Acked-by: David Hildenbrand 
Cc: Matthew Wilcox (Oracle) 
Cc: Mel Gorman 
Cc: Yunsheng Lin 
Signed-off-by: Andrew Morton

vmalloc: fix accounting with i915

2024-12-19T03:04:45+00:00

If the caller of vmap() specifies VM_MAP_PUT_PAGES (currently only the
i915 driver), we will decrement nr_vmalloc_pages and MEMCG_VMALLOC in
vfree().  These counters are incremented by vmalloc() but not by vmap() so
this will cause an underflow.  Check the VM_MAP_PUT_PAGES flag before
decrementing either counter.

Link: https://lkml.kernel.org/r/20241211202538.168311-1-willy@infradead.org
Fixes: b944afc9d64d ("mm: add a VM_MAP_PUT_PAGES flag for vmap")
Signed-off-by: Matthew Wilcox (Oracle) 
Acked-by: Johannes Weiner 
Reviewed-by: Shakeel Butt 
Reviewed-by: Balbir Singh 
Acked-by: Michal Hocko 
Cc: Christoph Hellwig 
Cc: Muchun Song 
Cc: Roman Gushchin 
Cc: "Uladzislau Rezki (Sony)" 
Cc: 
Signed-off-by: Andrew Morton

mm: fix vrealloc()'s KASAN poisoning logic

2024-12-06T03:54:44+00:00

When vrealloc() reuses already allocated vmap_area, we need to re-annotate
poisoned and unpoisoned portions of underlying memory according to the new
size.

This results in a KASAN splat recorded at [1].  A KASAN mis-reporting
issue where there is none.

Note, hard-coding KASAN_VMALLOC_PROT_NORMAL might not be exactly correct,
but KASAN flag logic is pretty involved and spread out throughout
__vmalloc_node_range_noprof(), so I'm using the bare minimum flag here and
leaving the rest to mm people to refactor this logic and reuse it here.

Link: https://lkml.kernel.org/r/20241126005206.3457974-1-andrii@kernel.org
Link: https://lore.kernel.org/bpf/67450f9b.050a0220.21d33d.0004.GAE@google.com/ [1]
Fixes: 3ddc2fefe6f3 ("mm: vmalloc: implement vrealloc()")
Signed-off-by: Andrii Nakryiko 
Cc: Alexei Starovoitov 
Cc: Christoph Hellwig 
Cc: Michal Hocko 
Cc: Uladzislau Rezki (Sony) 
Cc: Vlastimil Babka 
Cc: 
Signed-off-by: Andrew Morton