linux.git/mm/Makefile, branch v6.15

mm: rename GENERIC_PTDUMP and PTDUMP_CORE

2025-03-17T07:05:32+00:00

Platforms subscribe into generic ptdump implementation via GENERIC_PTDUMP.
But generic ptdump gets enabled via PTDUMP_CORE.  These configs
combination is confusing as they sound very similar and does not
differentiate between platform's feature subscription and feature
enablement for ptdump.  Rename the configs as ARCH_HAS_PTDUMP and PTDUMP
making it more clear and improve readability.

Link: https://lkml.kernel.org/r/20250226122404.1927473-6-anshuman.khandual@arm.com
Signed-off-by: Anshuman Khandual 
Reviewed-by: Christophe Leroy  (powerpc)
Acked-by: Catalin Marinas 	[arm64]
Cc: Will Deacon 
Cc: Jonathan Corbet 
Cc: Marc Zyngier 
Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Cc: Paul Walmsley 
Cc: Palmer Dabbelt 
Cc: Heiko Carstens 
Cc: Vasily Gorbik 
Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: Christophe Leroy 
Cc: Madhavan Srinivasan 
Cc: Mark Rutland 
Cc: Steven Price 
Signed-off-by: Andrew Morton

mm, swap: remove swap slot cache

2025-03-17T05:06:43+00:00

Slot cache is no longer needed now, removing it and all related code.

- vm-scalability with: `usemem --init-time -O -y -x -R -31 1G`,
12G memory cgroup using simulated pmem as SWAP (32G pmem, 32 CPUs),
16 test runs for each case, measuring the total throughput:

                      Before (KB/s) (stdev)  After (KB/s) (stdev)
Random (4K):          424907.60 (24410.78)   414745.92  (34554.78)
Random (64K):         163308.82 (11635.72)   167314.50  (18434.99)
Sequential (4K, !-R): 6150056.79 (103205.90) 6321469.06 (115878.16)

The performance changes are below noise level.

- Build linux kernel with make -j96, using 4K folio with 1.5G memory
cgroup limit and 64K folio with 2G memory cgroup limit, on top of tmpfs,
12 test runs, measuring the system time:

                  Before (s) (stdev)  After (s) (stdev)
make -j96 (4K):   6445.69 (61.95)     6408.80 (69.46)
make -j96 (64K):  6841.71 (409.04)    6437.99 (435.55)

Similar to above, 64k mTHP case showed a slight improvement.

Link: https://lkml.kernel.org/r/20250313165935.63303-7-ryncsn@gmail.com
Signed-off-by: Kairui Song 
Reviewed-by: Baoquan He 
Cc: Baolin Wang 
Cc: Barry Song 
Cc: Chris Li 
Cc: "Huang, Ying" 
Cc: Hugh Dickins 
Cc: Johannes Weiner 
Cc: Kalesh Singh 
Cc: Matthew Wilcow (Oracle) 
Cc: Nhat Pham 
Cc: Yosry Ahmed 
Signed-off-by: Andrew Morton

mm/hugetlb: move hugetlb CMA code in to its own file

2025-03-17T05:06:31+00:00

hugetlb.c contained a number of CONFIG_CMA ifdefs, and the code inside
them was large enough to merit being in its own file, so move it, cleaning
up things a bit.

Hide some direct variable access behind functions to accommodate the move.

No functional change intended.

Link: https://lkml.kernel.org/r/20250228182928.2645936-28-fvdl@google.com
Signed-off-by: Frank van der Linden 
Cc: Alexander Gordeev 
Cc: Andy Lutomirski 
Cc: Arnd Bergmann 
Cc: Dan Carpenter 
Cc: Dave Hansen 
Cc: David Hildenbrand 
Cc: Heiko Carstens 
Cc: Joao Martins 
Cc: Johannes Weiner 
Cc: Madhavan Srinivasan 
Cc: Michael Ellerman 
Cc: Muchun Song 
Cc: Oscar Salvador 
Cc: Peter Zijlstra 
Cc: Roman Gushchin (Cruise) 
Cc: Usama Arif 
Cc: Vasily Gorbik 
Cc: Yu Zhao 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm: zbud: remove zbud

2025-03-17T05:06:01+00:00

The zbud compressed pages allocator is rarely used, most users use
zsmalloc.  zbud consumes much more memory (only stores 1 or 2 compressed
pages per physical page).  The only advantage of zbud is a marginal
performance improvement that by no means justify the memory overhead.

Historically, zsmalloc had significantly worse latency than zbud and
z3fold but offered better memory savings.  This is no longer the case as
shown by a simple recent analysis [1].  In a kernel build test on tmpfs in
a limited cgroup, zbud 2-3% less time than zsmalloc, but at the cost of
using ~32% more memory (1.5G vs 1.13G).  The tradeoff does not make sense
for zbud in any practical scenario.

The only alleged advantage of zbud is not having the dependency on
CONFIG_MMU, but CONFIG_SWAP already depends on CONFIG_MMU anyway, and zbud
is only used by zswap.

Remove zbud after z3fold's removal, leaving zsmalloc as the one and only
zpool allocator.  Leave the removal of the zpool API (and its associated
config options) to a followup cleanup after no more allocators show up.

Deprecating zbud for a few cycles before removing it was initially
proposed [2], like z3fold was marked as deprecated for 2 cycles [3]. 
However, Johannes rightfully pointed out that the 2 cycles is too short
for most downstream consumers, and z3fold was deprecated first only as a
courtesy anyway.

[1]https://lore.kernel.org/lkml/CAJD7tkbRF6od-2x_L8-A1QL3=2Ww13sCj4S3i4bNndqF+3+_Vg@mail.gmail.com/
[2]https://lore.kernel.org/lkml/Z5gdnSX5Lv-nfjQL@google.com/
[3]https://lore.kernel.org/lkml/20240904233343.933462-1-yosryahmed@google.com/

Link: https://lkml.kernel.org/r/20250129180633.3501650-3-yosry.ahmed@linux.dev
Signed-off-by: Yosry Ahmed 
Reviewed-by: Shakeel Butt 
Acked-by: Johannes Weiner 
Acked-by: Nhat Pham 
Cc: Alexander Gordeev 
Cc: Chengming Zhou 
Cc: Christian Borntraeger 
Cc: Dan Streetman 
Cc: Heiko Carstens 
Cc: Huacai Chen 
Cc: Miaohe Lin 
Cc: Seth Jennings 
Cc: Sven Schnelle 
Cc: Vasily Gorbik 
Cc: Vitaly Wool 
Cc: Vlastimil Babka 
Cc: WANG Xuerui 
Signed-off-by: Andrew Morton

mm: z3fold: remove z3fold

2025-03-17T05:06:01+00:00

Patch series "mm: zswap: remove z3fold and zbud", v2.

After 2 cycles of deprecating z3fold, remove it as well as zbud (rationale
in specific patches).


This patch (of 2):

Z3fold has been marked as deprecated for 2 cycles and no one complained,
as expected.  As there are no known users, remove the code now.

Link: https://lkml.kernel.org/r/20250129180633.3501650-1-yosry.ahmed@linux.dev
Link: https://lkml.kernel.org/r/20250129180633.3501650-2-yosry.ahmed@linux.dev
Signed-off-by: Yosry Ahmed 
Acked-by: Johannes Weiner 
Reviewed-by: Shakeel Butt 
Acked-by: Nhat Pham 
Cc: Alexander Gordeev 
Cc: Chengming Zhou 
Cc: Christian Borntraeger 
Cc: Dan Streetman 
Cc: Heiko Carstens 
Cc: Huacai Chen 
Cc: Miaohe Lin 
Cc: Seth Jennings 
Cc: Sven Schnelle 
Cc: Vasily Gorbik 
Cc: Vitaly Wool 
Cc: Vlastimil Babka 
Cc: WANG Xuerui 
Signed-off-by: Andrew Morton

mm: pgtable: reclaim empty PTE page in madvise(MADV_DONTNEED)

2025-01-14T06:40:48+00:00

Now in order to pursue high performance, applications mostly use some
high-performance user-mode memory allocators, such as jemalloc or
tcmalloc.  These memory allocators use madvise(MADV_DONTNEED or MADV_FREE)
to release physical memory, but neither MADV_DONTNEED nor MADV_FREE will
release page table memory, which may cause huge page table memory usage.

The following are a memory usage snapshot of one process which actually
happened on our server:

        VIRT:  55t
        RES:   590g
        VmPTE: 110g

In this case, most of the page table entries are empty.  For such a PTE
page where all entries are empty, we can actually free it back to the
system for others to use.

As a first step, this commit aims to synchronously free the empty PTE
pages in madvise(MADV_DONTNEED) case.  We will detect and free empty PTE
pages in zap_pte_range(), and will add zap_details.reclaim_pt to exclude
cases other than madvise(MADV_DONTNEED).

Once an empty PTE is detected, we first try to hold the pmd lock within
the pte lock.  If successful, we clear the pmd entry directly (fast path).
Otherwise, we wait until the pte lock is released, then re-hold the pmd
and pte locks and loop PTRS_PER_PTE times to check pte_none() to re-detect
whether the PTE page is empty and free it (slow path).

For other cases such as madvise(MADV_FREE), consider scanning and freeing
empty PTE pages asynchronously in the future.

The following code snippet can show the effect of optimization:

        mmap 50G
        while (1) {
                for (; i < 1024 * 25; i++) {
                        touch 2M memory
                        madvise MADV_DONTNEED 2M
                }
        }

As we can see, the memory usage of VmPTE is reduced:

                        before                          after
VIRT                   50.0 GB                        50.0 GB
RES                     3.1 MB                         3.1 MB
VmPTE                102640 KB                         240 KB

[zhengqi.arch@bytedance.com: fix uninitialized symbol 'ptl']
  Link: https://lkml.kernel.org/r/20241206112348.51570-1-zhengqi.arch@bytedance.com
  Link: https://lore.kernel.org/linux-mm/224e6a4e-43b5-4080-bdd8-b0a6fb2f0853@stanley.mountain/
Link: https://lkml.kernel.org/r/92aba2b319a734913f18ba41e7d86a265f0b84e2.1733305182.git.zhengqi.arch@bytedance.com
Signed-off-by: Qi Zheng 
Cc: Andy Lutomirski 
Cc: Catalin Marinas 
Cc: Dave Hansen 
Cc: David Hildenbrand 
Cc: David Rientjes 
Cc: Hugh Dickins 
Cc: Jann Horn 
Cc: Lorenzo Stoakes 
Cc: Matthew Wilcox 
Cc: Mel Gorman 
Cc: Muchun Song 
Cc: Peter Xu 
Cc: Peter Zijlstra 
Cc: Will Deacon 
Cc: Zach O'Keefe 
Cc: Dan Carpenter 
Signed-off-by: Andrew Morton

mm: move the page fragment allocator from page_alloc into its own file

2024-11-11T18:56:26+00:00

Inspired by [1], move the page fragment allocator from page_alloc
into its own c file and header file, as we are about to make more
change for it to replace another page_frag implementation in
sock.c

As this patchset is going to replace 'struct page_frag' with
'struct page_frag_cache' in sched.h, including page_frag_cache.h
in sched.h has a compiler error caused by interdependence between
mm_types.h and mm.h for asm-offsets.c, see [2]. So avoid the compiler
error by moving 'struct page_frag_cache' to mm_types_task.h as
suggested by Alexander, see [3].

1. https://lore.kernel.org/all/20230411160902.4134381-3-dhowells@redhat.com/
2. https://lore.kernel.org/all/15623dac-9358-4597-b3ee-3694a5956920@gmail.com/
3. https://lore.kernel.org/all/CAKgT0UdH1yD=LSCXFJ=YM_aiA4OomD-2wXykO42bizaWMt_HOA@mail.gmail.com/
CC: David Howells 
CC: Linux-MM 
Signed-off-by: Yunsheng Lin 
Acked-by: Andrew Morton 
Reviewed-by: Alexander Duyck 
Link: https://patch.msgid.link/20241028115343.3405838-3-linyunsheng@huawei.com
Signed-off-by: Jakub Kicinski

mm: introduce numa_emulation

2024-09-04T04:15:31+00:00

Move numa_emulation code from arch/x86 to mm/numa_emulation.c

This code will be later reused by arch_numa.

No functional changes.

Link: https://lkml.kernel.org/r/20240807064110.1003856-20-rppt@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) 
Tested-by: Zi Yan  # for x86_64 and arm64
Reviewed-by: Jonathan Cameron 
Tested-by: Jonathan Cameron  [arm64 + CXL via QEMU]
Acked-by: Dan Williams 
Cc: Alexander Gordeev 
Cc: Andreas Larsson 
Cc: Arnd Bergmann 
Cc: Borislav Petkov 
Cc: Catalin Marinas 
Cc: Christophe Leroy 
Cc: Dave Hansen 
Cc: David Hildenbrand 
Cc: Davidlohr Bueso 
Cc: David S. Miller 
Cc: Greg Kroah-Hartman 
Cc: Heiko Carstens 
Cc: Huacai Chen 
Cc: Ingo Molnar 
Cc: Jiaxun Yang 
Cc: John Paul Adrian Glaubitz 
Cc: Jonathan Corbet 
Cc: Michael Ellerman 
Cc: Palmer Dabbelt 
Cc: Rafael J. Wysocki 
Cc: Rob Herring (Arm) 
Cc: Samuel Holland 
Cc: Thomas Bogendoerfer 
Cc: Thomas Gleixner 
Cc: Vasily Gorbik 
Cc: Will Deacon 
Signed-off-by: Andrew Morton

mm: introduce numa_memblks

2024-09-04T04:15:30+00:00

Move code dealing with numa_memblks from arch/x86 to mm/ and add Kconfig
options to let x86 select it in its Kconfig.

This code will be later reused by arch_numa.

No functional changes.

Link: https://lkml.kernel.org/r/20240807064110.1003856-18-rppt@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) 
Tested-by: Zi Yan  # for x86_64 and arm64
Reviewed-by: Jonathan Cameron 
Tested-by: Jonathan Cameron  [arm64 + CXL via QEMU]
Acked-by: Dan Williams 
Acked-by: David Hildenbrand 
Cc: Alexander Gordeev 
Cc: Andreas Larsson 
Cc: Arnd Bergmann 
Cc: Borislav Petkov 
Cc: Catalin Marinas 
Cc: Christophe Leroy 
Cc: Dave Hansen 
Cc: Davidlohr Bueso 
Cc: David S. Miller 
Cc: Greg Kroah-Hartman 
Cc: Heiko Carstens 
Cc: Huacai Chen 
Cc: Ingo Molnar 
Cc: Jiaxun Yang 
Cc: John Paul Adrian Glaubitz 
Cc: Jonathan Corbet 
Cc: Michael Ellerman 
Cc: Palmer Dabbelt 
Cc: Rafael J. Wysocki 
Cc: Rob Herring (Arm) 
Cc: Samuel Holland 
Cc: Thomas Bogendoerfer 
Cc: Thomas Gleixner 
Cc: Vasily Gorbik 
Cc: Will Deacon 
Signed-off-by: Andrew Morton

mm: move kernel/numa.c to mm/

2024-09-04T04:15:26+00:00

Patch series "mm: introduce numa_memblks", v4.

Following the discussion about handling of CXL fixed memory windows on
arm64 [1] I decided to bite the bullet and move numa_memblks from x86 to
the generic code so they will be available on arm64/riscv and maybe on
loongarch sometime later.

While it could be possible to use memblock to describe CXL memory windows,
it currently lacks notion of unpopulated memory ranges and numa_memblks
does implement this.

Another reason to make numa_memblks generic is that both arch_numa (arm64
and riscv) and loongarch use trimmed copy of x86 code although there is no
fundamental reason why the same code cannot be used on all these
platforms.  Having numa_memblks in mm/ will make it's interaction with
ACPI and FDT more consistent and I believe will reduce maintenance burden.

And with generic numa_memblks it is (almost) straightforward to enable
NUMA emulation on arm64 and riscv.

The first 9 commits in this series are cleanups that are not strictly
related to numa_memblks.
Commits 10-16 slightly reorder code in x86 to allow extracting numa_memblks
and NUMA emulation to the generic code.
Commits 17-19 actually move the code from arch/x86/ to mm/ and commits 20-22
does some aftermath cleanups.
Commit 23 updates of_numa_init() to return error of no NUMA nodes were
found in the device tree.
Commit 24 switches arch_numa to numa_memblks.
Commit 25 enables usage of phys_to_target_node() and
memory_add_physaddr_to_nid() with numa_memblks.
Commit 26 moves the description for numa=fake from x86 to admin-guide.

[1] https://lore.kernel.org/all/20240529171236.32002-1-Jonathan.Cameron@huawei.com/


This patch (of 26):

The stub functions in kernel/numa.c belong to mm/ rather than to kernel/

Link: https://lkml.kernel.org/r/20240807064110.1003856-1-rppt@kernel.org
Link: https://lkml.kernel.org/r/20240807064110.1003856-2-rppt@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) 
Acked-by: David Hildenbrand 
Reviewed-by: Jonathan Cameron 
Tested-by: Zi Yan  # for x86_64 and arm64
Tested-by: Jonathan Cameron  [arm64 + CXL via QEMU]
Acked-by: Dan Williams 
Cc: Alexander Gordeev 
Cc: Andreas Larsson 
Cc: Arnd Bergmann 
Cc: Borislav Petkov 
Cc: Catalin Marinas 
Cc: Christophe Leroy 
Cc: Dave Hansen 
Cc: Davidlohr Bueso 
Cc: David S. Miller 
Cc: Greg Kroah-Hartman 
Cc: Heiko Carstens 
Cc: Huacai Chen 
Cc: Ingo Molnar 
Cc: Jiaxun Yang 
Cc: John Paul Adrian Glaubitz 
Cc: Jonathan Corbet 
Cc: Michael Ellerman 
Cc: Palmer Dabbelt 
Cc: Rafael J. Wysocki 
Cc: Rob Herring (Arm) 
Cc: Samuel Holland 
Cc: Thomas Bogendoerfer 
Cc: Thomas Gleixner 
Cc: Vasily Gorbik 
Cc: Will Deacon 
Signed-off-by: Andrew Morton