linux.git/Documentation/admin-guide/mm, branch vsnprintf

mm: shmem: override mTHP shmem default with a kernel parameter

2024-11-11T21:09:43+00:00

Add the ``thp_shmem=`` kernel command line to allow specifying the default
policy of each supported shmem hugepage size.  The kernel parameter
accepts the following format:

thp_shmem=[KMG],[KMG]:;[KMG]-[KMG]:

For example,

thp_shmem=16K-64K:always;128K,512K:inherit;256K:advise;1M-2M:never;4M-8M:within_size

Some GPUs may benefit from using huge pages.  Since DRM GEM uses shmem to
allocate anonymous pageable memory, it's essential to control the huge
page allocation policy for the internal shmem mount.  This control can be
achieved through the ``transparent_hugepage_shmem=`` parameter.

Beyond just setting the allocation policy, it's crucial to have granular
control over the size of huge pages that can be allocated.  The GPU may
support only specific huge page sizes, and allocating pages larger/smaller
than those sizes would be ineffective.

Link: https://lkml.kernel.org/r/20241101165719.1074234-6-mcanal@igalia.com
Signed-off-by: Maíra Canal 
Reviewed-by: Baolin Wang 
Cc: Barry Song 
Cc: David Hildenbrand 
Cc: Hugh Dickins 
Cc: Jonathan Corbet 
Cc: Lance Yang 
Cc: Ryan Roberts 
Signed-off-by: Andrew Morton

mm: shmem: control THP support through the kernel command line

2024-11-11T21:09:43+00:00

Patch series "mm: add more kernel parameters to control mTHP", v5.

This series introduces four patches related to the kernel parameters
controlling mTHP and a fifth patch replacing `strcpy()` for `strscpy()` in
the file `mm/huge_memory.c`.

The first patch is a straightforward documentation update, correcting the
format of the kernel parameter ``thp_anon=``.

The second, third, and fourth patches focus on controlling THP support for
shmem via the kernel command line.  The second patch introduces a
parameter to control the global default huge page allocation policy for
the internal shmem mount.  The third patch moves a piece of code to a
shared header to ease the implementation of the fourth patch.  Finally,
the fourth patch implements a parameter similar to ``thp_anon=``, but for
shmem.

The goal of these changes is to simplify the configuration of systems that
rely on mTHP support for shmem.  For instance, a platform with a GPU that
benefits from huge pages may want to enable huge pages for shmem.  Having
these kernel parameters streamlines the configuration process and ensures
consistency across setups.


This patch (of 4):

Add a new kernel command line to control the hugepage allocation policy
for the internal shmem mount, ``transparent_hugepage_shmem``. The
parameter is similar to ``transparent_hugepage`` and has the following
format:

transparent_hugepage_shmem=

where ```` is one of the seven valid policies available for
shmem.

Configuring the default huge page allocation policy for the internal
shmem mount can be beneficial for DRM GPU drivers. Just as CPU
architectures, GPUs can also take advantage of huge pages, but this is
possible only if DRM GEM objects are backed by huge pages.

Since GEM uses shmem to allocate anonymous pageable memory, having control
over the default huge page allocation policy allows for the exploration of
huge pages use on GPUs that rely on GEM objects backed by shmem.

Link: https://lkml.kernel.org/r/20241101165719.1074234-2-mcanal@igalia.com
Link: https://lkml.kernel.org/r/20241101165719.1074234-4-mcanal@igalia.com
Signed-off-by: Maíra Canal 
Reviewed-by: Baolin Wang 
Acked-by: David Hildenbrand 
Cc: Barry Song 
Cc: dri-devel@lists.freedesktop.org
Cc: Hugh Dickins 
Cc: Jonathan Corbet 
Cc: kernel-dev@igalia.com
Cc: Lance Yang 
Cc: Ryan Roberts 
Signed-off-by: Andrew Morton

mm: add per-order mTHP swpin counters

2024-11-11T08:26:43+00:00

This helps profile the sizes of folios being swapped in. Currently,
only mTHP swap-out is being counted.
The new interface can be found at:
/sys/kernel/mm/transparent_hugepage/hugepages-/stats
         swpin
For example,
cat /sys/kernel/mm/transparent_hugepage/hugepages-64kB/stats/swpin
12809
cat /sys/kernel/mm/transparent_hugepage/hugepages-32kB/stats/swpin
4763

[v-songbaohua@oppo.com: add a blank line in doc]
  Link: https://lkml.kernel.org/r/20241030233423.80759-1-21cnbao@gmail.com
Link: https://lkml.kernel.org/r/20241026082423.26298-1-21cnbao@gmail.com
Signed-off-by: Barry Song 
Reviewed-by: Baolin Wang 
Acked-by: David Hildenbrand 
Cc: Chris Li 
Cc: Yosry Ahmed 
Cc: "Huang, Ying" 
Cc: Kairui Song 
Cc: Ryan Roberts 
Cc: Kanchana P Sridhar 
Cc: Usama Arif 
Signed-off-by: Andrew Morton

mm: swap: count successful large folio zswap stores in hugepage zswpout stats

2024-11-11T08:26:43+00:00

Added a new MTHP_STAT_ZSWPOUT entry to the sysfs transparent_hugepage
stats so that successful large folio zswap stores can be accounted under
the per-order sysfs "zswpout" stats:

/sys/kernel/mm/transparent_hugepage/hugepages-*kB/stats/zswpout

Other non-zswap swap device swap-out events will be counted under
the existing sysfs "swpout" stats:

/sys/kernel/mm/transparent_hugepage/hugepages-*kB/stats/swpout

Also, added documentation for the newly added sysfs per-order hugepage
"zswpout" stats. The documentation clarifies that only non-zswap swapouts
will be accounted in the existing "swpout" stats.

Link: https://lkml.kernel.org/r/20241001053222.6944-8-kanchana.p.sridhar@intel.com
Signed-off-by: Kanchana P Sridhar 
Reviewed-by: Nhat Pham 
Cc: Chengming Zhou 
Cc: "Huang, Ying" 
Cc: Johannes Weiner 
Cc: Matthew Wilcox 
Cc: Ryan Roberts 
Cc: Shakeel Butt 
Cc: Usama Arif 
Cc: Wajdi Feghali 
Cc: Yosry Ahmed 
Cc: "Zou, Nanhai" 
Cc: Barry Song <21cnbao@gmail.com>
Signed-off-by: Andrew Morton

mm: fix docs for the kernel parameter ``thp_anon=``

2024-11-07T22:14:59+00:00

If we add ``thp_anon=32,64K:always`` to the kernel command line, we
will see the following error:

[    0.000000] huge_memory: thp_anon=32,64K:always: error parsing string, ignoring setting

This happens because the correct format isn't ``thp_anon=,[KMG]:```,
as [KMG] must follow each number to especify its unit. So, the correct
format is ``thp_anon=[KMG],[KMG]:```.

Therefore, adjust the documentation to reflect the correct format of the
parameter ``thp_anon=``.

Link: https://lkml.kernel.org/r/20241101165719.1074234-3-mcanal@igalia.com
Fixes: dd4d30d1cdbe ("mm: override mTHP "enabled" defaults at kernel cmdline")
Signed-off-by: Maíra Canal 
Acked-by: Barry Song 
Acked-by: David Hildenbrand 
Cc: Baolin Wang 
Cc: Hugh Dickins 
Cc: Jonathan Corbet 
Cc: Lance Yang 
Cc: Ryan Roberts 
Signed-off-by: Andrew Morton

Merge tag 'mm-nonmm-stable-2024-09-21-07-52' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

2024-09-21T15:20:50+00:00

Pull non-MM updates from Andrew Morton:
"Many singleton patches - please see the various changelogs for
details.

Quite a lot of nilfs2 work this time around.

Notable patch series in this pull request are:

- "mul_u64_u64_div_u64: new implementation" by Nicolas Pitre, with
assistance from Uwe Kleine-König. Reimplement mul_u64_u64_div_u64()
to provide (much) more accurate results. The current implementation
was causing Uwe some issues in the PWM drivers.

- "xz: Updates to license, filters, and compression options" from
Lasse Collin. Miscellaneous maintenance and kinor feature work to
the xz decompressor.

- "Fix some GDB command error and add some GDB commands" from
Kuan-Ying Lee. Fixes and enhancements to the gdb scripts.

- "treewide: add missing MODULE_DESCRIPTION() macros" from Jeff
Johnson. Adds lots of MODULE_DESCRIPTIONs, thus fixing lots of
warnings about this.

- "nilfs2: add support for some common ioctls" from Ryusuke Konishi.
Adds various commonly-available ioctls to nilfs2.

- "This series fixes a number of formatting issues in kernel doc
comments" from Ryusuke Konishi does that.

- "nilfs2: prevent unexpected ENOENT propagation" from Ryusuke
Konishi. Fix issues where -ENOENT was being unintentionally and
inappropriately returned to userspace.

- "nilfs2: assorted cleanups" from Huang Xiaojia.

- "nilfs2: fix potential issues with empty b-tree nodes" from Ryusuke
Konishi fixes some issues which can occur on corrupted nilfs2
filesystems.

- "scripts/decode_stacktrace.sh: improve error reporting and
usability" from Luca Ceresoli does those things"

* tag 'mm-nonmm-stable-2024-09-21-07-52' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (103 commits)
list: test: increase coverage of list_test_list_replace*()
list: test: fix tests for list_cut_position()
proc: use __auto_type more
treewide: correct the typo 'retun'
ocfs2: cleanup return value and mlog in ocfs2_global_read_info()
nilfs2: remove duplicate 'unlikely()' usage
nilfs2: fix potential oob read in nilfs_btree_check_delete()
nilfs2: determine empty node blocks as corrupted
nilfs2: fix potential null-ptr-deref in nilfs_btree_insert()
user_namespace: use kmemdup_array() instead of kmemdup() for multiple allocation
tools/mm: rm thp_swap_allocator_test when make clean
squashfs: fix percpu address space issues in decompressor_multi_percpu.c
lib: glob.c: added null check for character class
nilfs2: refactor nilfs_segctor_thread()
nilfs2: use kthread_create and kthread_stop for the log writer thread
nilfs2: remove sc_timer_task
nilfs2: do not repair reserved inode bitmap in nilfs_new_inode()
nilfs2: eliminate the shared counter and spinlock for i_generation
nilfs2: separate inode type information from i_state field
nilfs2: use the BITS_PER_LONG macro
...

mm: add sysfs entry to disable splitting underused THPs

2024-09-09T23:39:04+00:00

If disabled, THPs faulted in or collapsed will not be added to
_deferred_list, and therefore won't be considered for splitting under
memory pressure if underused.

Link: https://lkml.kernel.org/r/20240830100438.3623486-7-usamaarif642@gmail.com
Signed-off-by: Usama Arif 
Cc: Alexander Zhu 
Cc: Barry Song 
Cc: David Hildenbrand 
Cc: Domenico Cerasuolo 
Cc: Johannes Weiner 
Cc: Jonathan Corbet 
Cc: Kairui Song 
Cc: Matthew Wilcox 
Cc: Mike Rapoport 
Cc: Nico Pache 
Cc: Rik van Riel 
Cc: Roman Gushchin 
Cc: Ryan Roberts 
Cc: Shakeel Butt 
Cc: Shuang Zhai 
Cc: Shuang Zhai 
Cc: Yu Zhao 
Cc: Hugh Dickins 
Signed-off-by: Andrew Morton

mm: split underused THPs

2024-09-09T23:39:04+00:00

This is an attempt to mitigate the issue of running out of memory when THP
is always enabled.  During runtime whenever a THP is being faulted in
(__do_huge_pmd_anonymous_page) or collapsed by khugepaged
(collapse_huge_page), the THP is added to _deferred_list.  Whenever memory
reclaim happens in linux, the kernel runs the deferred_split shrinker
which goes through the _deferred_list.

If the folio was partially mapped, the shrinker attempts to split it.  If
the folio is not partially mapped, the shrinker checks if the THP was
underused, i.e.  how many of the base 4K pages of the entire THP were
zero-filled.  If this number goes above a certain threshold (decided by
/sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none), the
shrinker will attempt to split that THP.  Then at remap time, the pages
that were zero-filled are mapped to the shared zeropage, hence saving
memory.

Link: https://lkml.kernel.org/r/20240830100438.3623486-6-usamaarif642@gmail.com
Signed-off-by: Usama Arif 
Suggested-by: Rik van Riel 
Co-authored-by: Johannes Weiner 
Cc: Alexander Zhu 
Cc: Barry Song 
Cc: David Hildenbrand 
Cc: Domenico Cerasuolo 
Cc: Jonathan Corbet 
Cc: Kairui Song 
Cc: Matthew Wilcox 
Cc: Mike Rapoport 
Cc: Nico Pache 
Cc: Roman Gushchin 
Cc: Ryan Roberts 
Cc: Shakeel Butt 
Cc: Shuang Zhai 
Cc: Yu Zhao 
Cc: Shuang Zhai 
Cc: Hugh Dickins 
Signed-off-by: Andrew Morton

Docs/damon: use damonitor GitHub organization instead of awslabs

2024-09-09T23:39:00+00:00

Patch series "Docs/damon: update GitHub repo URLs and maintainer-profile".

Replace GitHub URLS on DAMON documents for none-kernel parts DAMON repos
with new ones[1] via the first patch.  With following two patches,
wordsmith maitnainer-profile for better readability, and document the
Google clendsar for bi-weekly meetups, respectively.

[1] https://lore.kernel.org/20240813232158.83903-1-sj@kernel.org


This patch (of 3):

GitHub repos for non-kernel parts of DAMON project including 'damo',
'damon-tests' and 'damoos' will be moved[1] from 'awslabs' org to
'damonitor', by 2024-09-05.  Update related URLs in kernel tree.

[1] https://lore.kernel.org/20240813232158.83903-1-sj@kernel.org

Link: https://lkml.kernel.org/r/20240826015741.80707-1-sj@kernel.org
Link: https://lkml.kernel.org/r/20240826015741.80707-2-sj@kernel.org
Signed-off-by: SeongJae Park 
Cc: Alex Shi 
Cc: Hu Haowen <2023002089@link.tyut.edu.cn>
Cc: Jonathan Corbet 
Cc: Yanteng Si 
Signed-off-by: Andrew Morton

mm: count the number of partially mapped anonymous THPs per size

2024-09-09T23:38:57+00:00

When a THP is added to the deferred_list due to partially mapped, its
partial pages are unused, leading to wasted memory and potentially
increasing memory reclamation pressure.

Detailing the specifics of how unmapping occurs is quite difficult and not
that useful, so we adopt a simple approach: each time a THP enters the
deferred_list, we increment the count by 1; whenever it leaves for any
reason, we decrement the count by 1.

Link: https://lkml.kernel.org/r/20240824010441.21308-3-21cnbao@gmail.com
Signed-off-by: Barry Song 
Acked-by: David Hildenbrand 
Cc: Baolin Wang 
Cc: Chris Li 
Cc: Chuanhua Han 
Cc: Kairui Song 
Cc: Kalesh Singh 
Cc: Lance Yang 
Cc: Ryan Roberts 
Cc: Shuai Yuan 
Cc: Usama Arif 
Cc: Zi Yan 
Signed-off-by: Andrew Morton