linux.git/mm/mm_init.c, branch v6.14

mm/memmap: prevent double scanning of memmap by kmemleak

2025-01-26T04:22:30+00:00

kmemleak explicitly scans the mem_map through the valid struct page
objects.  However, memmap_alloc() was also adding this memory to the gray
object list, causing it to be scanned twice.  Remove memmap_alloc() from
the scan list and add a comment to clarify the behavior.

Link: https://lore.kernel.org/lkml/CAOm6qn=FVeTpH54wGDFMHuCOeYtvoTx30ktnv9-w3Nh8RMofEA@mail.gmail.com/
Link: https://lkml.kernel.org/r/20250106021126.1678334-1-guoweikang.kernel@gmail.com
Signed-off-by: Guo Weikang 
Reviewed-by: Catalin Marinas 
Cc: Mike Rapoport (Microsoft) 
Signed-off-by: Andrew Morton

Merge tag 'memblock-v6.13-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rppt/memblock

2024-11-27T19:13:25+00:00

Pull memblock updates from Mike Rapoport:

 - replace hardcoded strings with str_on_off() in report_meminit()

 - initialize reserved pages to MIGRATE_MOVABLE when deferred struct
   page initialization is enabled so that if the reserved pages are
   freed they are put on movable free lists like it is done now when
   deferred struct page initialization is disabled

* tag 'memblock-v6.13-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rppt/memblock:
  memblock: uniformly initialize all reserved pages to MIGRATE_MOVABLE
  mm: Use str_on_off() helper function in report_meminit()

alloc_tag: support for page allocation tag compression

2024-11-07T22:25:16+00:00

Implement support for storing page allocation tag references directly in
the page flags instead of page extensions.  sysctl.vm.mem_profiling boot
parameter it extended to provide a way for a user to request this mode. 
Enabling compression eliminates memory overhead caused by page_ext and
results in better performance for page allocations.  However this mode
will not work if the number of available page flag bits is insufficient to
address all kernel allocations.  Such condition can happen during boot or
when loading a module.  If this condition is detected, memory allocation
profiling gets disabled with an appropriate warning.  By default
compression mode is disabled.

Link: https://lkml.kernel.org/r/20241023170759.999909-7-surenb@google.com
Signed-off-by: Suren Baghdasaryan 
Reviewed-by: Pasha Tatashin 
Cc: Ard Biesheuvel 
Cc: Arnd Bergmann 
Cc: Borislav Petkov (AMD) 
Cc: Christoph Hellwig 
Cc: Daniel Gomez 
Cc: David Hildenbrand 
Cc: Davidlohr Bueso 
Cc: David Rientjes 
Cc: Dennis Zhou 
Cc: Johannes Weiner 
Cc: John Hubbard 
Cc: Jonathan Corbet 
Cc: Joonsoo Kim 
Cc: Kalesh Singh 
Cc: Kees Cook 
Cc: Kent Overstreet 
Cc: Liam R. Howlett 
Cc: Luis Chamberlain 
Cc: Matthew Wilcox 
Cc: Michal Hocko 
Cc: Mike Rapoport (Microsoft) 
Cc: Minchan Kim 
Cc: Paul E. McKenney 
Cc: Petr Pavlu 
Cc: Roman Gushchin 
Cc: Sami Tolvanen 
Cc: Sourav Panda 
Cc: Steven Rostedt (Google) 
Cc: Thomas Gleixner 
Cc: Thomas Huth 
Cc: Uladzislau Rezki (Sony) 
Cc: Vlastimil Babka 
Cc: Xiongwei Song 
Cc: Yu Zhao 
Signed-off-by: Andrew Morton

memblock: uniformly initialize all reserved pages to MIGRATE_MOVABLE

2024-10-28T10:16:06+00:00

Currently when CONFIG_DEFERRED_STRUCT_PAGE_INIT is not set, the reserved
pages are initialized to MIGRATE_MOVABLE by default in memmap_init.

Reserved memory mainly store the metadata of struct page. When
HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON=Y and hugepages are allocated,
the HVO will remap the vmemmap virtual address range to the page which
vmemmap_reuse is mapped to. The pages previously mapping the range will
be freed to the buddy system.

Before this patch:
when CONFIG_DEFERRED_STRUCT_PAGE_INIT is not set, the freed memory was
placed on the Movable list;
When CONFIG_DEFERRED_STRUCT_PAGE_INIT=Y, the freed memory was placed on
the Unmovable list.

After this patch, the freed memory is placed on the Movable list
regardless of whether CONFIG_DEFERRED_STRUCT_PAGE_INIT is set.

Eg:
Tested on a virtual machine(1000GB):
Intel(R) Xeon(R) Platinum 8358P CPU

After vm start:
echo 500000 > /proc/sys/vm/nr_hugepages
cat /proc/meminfo | grep -i huge
HugePages_Total:   500000
HugePages_Free:    500000
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:        1024000000 kB

cat /proc/pagetypeinfo
before：
Free pages count per migrate type at order       0      1      2      3      4      5      6      7      8      9     10
…
Node    0, zone   Normal, type    Unmovable     51      2      1     28     53     35     35     43     40     69   3852
Node    0, zone   Normal, type      Movable   6485   4610    666    202    200    185    208     87     54      2    240
Node    0, zone   Normal, type  Reclaimable      2      2      1     23     13      1      2      1      0      1      0
Node    0, zone   Normal, type   HighAtomic      0      0      0      0      0      0      0      0      0      0      0
Node    0, zone   Normal, type      Isolate      0      0      0      0      0      0      0      0      0      0      0
Unmovable ≈ 15GB

after：
Free pages count per migrate type at order       0      1      2      3      4      5      6      7      8      9     10
…
Node    0, zone   Normal, type    Unmovable      0      1      1      0      0      0      0      1      1      1      0
Node    0, zone   Normal, type      Movable   1563   4107   1119    189    256    368    286    132    109      4   3841
Node    0, zone   Normal, type  Reclaimable      2      2      1     23     13      1      2      1      0      1      0
Node    0, zone   Normal, type   HighAtomic      0      0      0      0      0      0      0      0      0      0      0
Node    0, zone   Normal, type      Isolate      0      0      0      0      0      0      0      0      0      0      0

Signed-off-by: Hua Su 
Reviewed-by: Wei Yang 
Link: https://lore.kernel.org/r/20241021051151.4664-1-suhua.tanke@gmail.com
Signed-off-by: Mike Rapoport (Microsoft)

mm: Use str_on_off() helper function in report_meminit()

2024-10-18T16:05:58+00:00

Remove hard-coded strings by using the helper function str_on_off().

Signed-off-by: Thorsten Blum 
Link: https://lore.kernel.org/r/20241018103150.96824-2-thorsten.blum@linux.dev
Signed-off-by: Mike Rapoport (Microsoft)

mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION

2024-09-04T04:15:28+00:00

There are no users of HAVE_ARCH_NODEDATA_EXTENSION left, so
arch_alloc_nodedata() and arch_refresh_nodedata() are not needed anymore.

Replace the call to arch_alloc_nodedata() in free_area_init() with a new
helper alloc_offline_node_data(), remove arch_refresh_nodedata() and
cleanup include/linux/memory_hotplug.h from the associated ifdefery.

Link: https://lkml.kernel.org/r/20240807064110.1003856-9-rppt@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) 
Tested-by: Zi Yan  # for x86_64 and arm64
Acked-by: Dan Williams 
Cc: Alexander Gordeev 
Cc: Andreas Larsson 
Cc: Arnd Bergmann 
Cc: Borislav Petkov 
Cc: Catalin Marinas 
Cc: Christophe Leroy 
Cc: Dave Hansen 
Cc: David Hildenbrand 
Cc: Davidlohr Bueso 
Cc: David S. Miller 
Cc: Greg Kroah-Hartman 
Cc: Heiko Carstens 
Cc: Huacai Chen 
Cc: Ingo Molnar 
Cc: Jiaxun Yang 
Cc: John Paul Adrian Glaubitz 
Cc: Jonathan Cameron 
Cc: Jonathan Corbet 
Cc: Michael Ellerman 
Cc: Palmer Dabbelt 
Cc: Rafael J. Wysocki 
Cc: Rob Herring (Arm) 
Cc: Samuel Holland 
Cc: Thomas Bogendoerfer 
Cc: Thomas Gleixner 
Cc: Vasily Gorbik 
Cc: Will Deacon 
Signed-off-by: Andrew Morton

mm: rework accept memory helpers

2024-09-02T03:26:07+00:00

Make accept_memory() and range_contains_unaccepted_memory() take 'start'
and 'size' arguments instead of 'start' and 'end'.

Remove accept_page(), replacing it with direct calls to accept_memory(). 
The accept_page() name is going to be used for a different function.

Link: https://lkml.kernel.org/r/20240809114854.3745464-6-kirill.shutemov@linux.intel.com
Signed-off-by: Kirill A. Shutemov 
Suggested-by: David Hildenbrand 
Cc: Borislav Petkov 
Cc: Johannes Weiner 
Cc: Matthew Wilcox 
Cc: Mel Gorman 
Cc: Mike Rapoport (Microsoft) 
Cc: Tom Lendacky 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton

alloc_tag: mark pages reserved during CMA activation as not tagged

2024-08-16T05:16:16+00:00

During CMA activation, pages in CMA area are prepared and then freed
without being allocated.  This triggers warnings when memory allocation
debug config (CONFIG_MEM_ALLOC_PROFILING_DEBUG) is enabled.  Fix this by
marking these pages not tagged before freeing them.

Link: https://lkml.kernel.org/r/20240813150758.855881-2-surenb@google.com
Fixes: d224eb0287fb ("codetag: debug: mark codetags for reserved pages as empty")
Signed-off-by: Suren Baghdasaryan 
Acked-by: David Hildenbrand 
Cc: Kees Cook 
Cc: Kent Overstreet 
Cc: Pasha Tatashin 
Cc: Sourav Panda 
Cc: Vlastimil Babka 
Cc: 	[6.10]
Signed-off-by: Andrew Morton

alloc_tag: introduce clear_page_tag_ref() helper function

2024-08-16T05:16:16+00:00

In several cases we are freeing pages which were not allocated using
common page allocators.  For such cases, in order to keep allocation
accounting correct, we should clear the page tag to indicate that the page
being freed is expected to not have a valid allocation tag.  Introduce
clear_page_tag_ref() helper function to be used for this.

Link: https://lkml.kernel.org/r/20240813150758.855881-1-surenb@google.com
Fixes: d224eb0287fb ("codetag: debug: mark codetags for reserved pages as empty")
Signed-off-by: Suren Baghdasaryan 
Suggested-by: David Hildenbrand 
Acked-by: David Hildenbrand 
Reviewed-by: Pasha Tatashin 
Cc: Kees Cook 
Cc: Kent Overstreet 
Cc: Sourav Panda 
Cc: Vlastimil Babka 
Cc: 	[6.10]
Signed-off-by: Andrew Morton

mm: don't account memmap per-node

2024-08-16T05:16:14+00:00

Fix invalid access to pgdat during hot-remove operation:
ndctl users reported a GPF when trying to destroy a namespace:
$ ndctl destroy-namespace all -r all -f
 Segmentation fault
 dmesg:
 Oops: general protection fault, probably for
 non-canonical address 0xdffffc0000005650: 0000 [#1] PREEMPT SMP KASAN
 PTI
 KASAN: probably user-memory-access in range
 [0x000000000002b280-0x000000000002b287]
 CPU: 26 UID: 0 PID: 1868 Comm: ndctl Not tainted 6.11.0-rc1 #1
 Hardware name: Dell Inc. PowerEdge R640/08HT8T, BIOS
 2.20.1 09/13/2023
 RIP: 0010:mod_node_page_state+0x2a/0x110

cxl-test users report a GPF when trying to unload the test module:
$ modrpobe -r cxl-test
 dmesg
 BUG: unable to handle page fault for address: 0000000000004200
 #PF: supervisor read access in kernel mode
 #PF: error_code(0x0000) - not-present page
 PGD 0 P4D 0
 Oops: Oops: 0000 [#1] PREEMPT SMP PTI
 CPU: 0 UID: 0 PID: 1076 Comm: modprobe Tainted: G O N 6.11.0-rc1 #197
 Tainted: [O]=OOT_MODULE, [N]=TEST
 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/15
 RIP: 0010:mod_node_page_state+0x6/0x90

Currently, when memory is hot-plugged or hot-removed the accounting is
done based on the assumption that memmap is allocated from the same node
as the hot-plugged/hot-removed memory, which is not always the case.

In addition, there are challenges with keeping the node id of the memory
that is being remove to the time when memmap accounting is actually
performed: since this is done after remove_pfn_range_from_zone(), and
also after remove_memory_block_devices(). Meaning that we cannot use
pgdat nor walking though memblocks to get the nid.

Given all of that, account the memmap overhead system wide instead.

For this we are going to be using global atomic counters, but given that
memmap size is rarely modified, and normally is only modified either
during early boot when there is only one CPU, or under a hotplug global
mutex lock, therefore there is no need for per-cpu optimizations.

Also, while we are here rename nr_memmap to nr_memmap_pages, and
nr_memmap_boot to nr_memmap_boot_pages to be self explanatory that the
units are in page count.

[pasha.tatashin@soleen.com: address a few nits from David Hildenbrand]
  Link: https://lkml.kernel.org/r/20240809191020.1142142-4-pasha.tatashin@soleen.com
Link: https://lkml.kernel.org/r/20240809191020.1142142-4-pasha.tatashin@soleen.com
Link: https://lkml.kernel.org/r/20240808213437.682006-4-pasha.tatashin@soleen.com
Fixes: 15995a352474 ("mm: report per-page metadata information")
Signed-off-by: Pasha Tatashin 
Reported-by: Yi Zhang 
Closes: https://lore.kernel.org/linux-cxl/CAHj4cs9Ax1=CoJkgBGP_+sNu6-6=6v=_L-ZBZY0bVLD3wUWZQg@mail.gmail.com
Reported-by: Alison Schofield 
Closes: https://lore.kernel.org/linux-mm/Zq0tPd2h6alFz8XF@aschofie-mobl2/#t
Tested-by: Dan Williams 
Tested-by: Alison Schofield 
Acked-by: David Hildenbrand 
Acked-by: David Rientjes 
Tested-by: Yi Zhang 
Cc: Domenico Cerasuolo 
Cc: Fan Ni 
Cc: Joel Granados 
Cc: Johannes Weiner 
Cc: Li Zhijian 
Cc: Matthew Wilcox (Oracle) 
Cc: Mike Rapoport 
Cc: Muchun Song 
Cc: Nhat Pham 
Cc: Sourav Panda 
Cc: Vlastimil Babka 
Cc: Yosry Ahmed 
Signed-off-by: Andrew Morton