linux.git/include/linux/sched.h, branch v7.3-rc2 Linux kernel source tree Merge tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm 2026-08-27T16:17:06+00:00 Linus Torvalds torvalds@linux-foundation.org 2026-08-27T16:17:06+00:00 18fbf5151d2c0bfe433c7428eef03cabf5fdb2fa Pull more MM updates from Andrew Morton: - "mm/rmap: index MAP_PRIVATE file-backed folios by anonymous pgoff" (Lorenzo Stoakes) Index MAP_PRIVATE file-backed folios by their anonymous page offset to resolve confusion around reverse mapping for zeroed and CoW'd file-backed memory. Use this new VMA anonymous page offset tracking to eliminate index conflicts and lay the foundation for scalable CoW performance improvements. - "promote mapped executable folios after first usage for MGLRU" (Baolin Wang) Make MGLRU's protection of mapped executable file folios more reliable. Follow the classical LRU's logic, promoting mapped executable file folios after their first usage to give executable code a better chance to stay in memory and improve workload performance. - "mm: vmscan: fix node reclaim ignoring swappiness parameter" (Ridong Chen) Fix per-node proactive reclaim interface's ignoring the swappiness parameter when CONFIG_MEMCG is disabled by consolidating sc_swappiness() into a single function that checks proactive_swappiness regardless of kernel configuration. - "mm/vmscan: reduce lru_lock contention via vmstat-derived scan-balance cost" (Usama Arif) Reduce lru_lock contention in the reclaim path by deriving scan-balance costs from vmstat counters rather than lock-acquired producer updates. Read and decay these cost signals on the reclaim side under a dedicated per-lruvec lock, reducing total LRU lock wait time by over 60% without impacting scan throughput. - "zram: fix zram issues reported by sashiko" (Sergey Senozhatsky) Fix two low-risk zram bugs which Sashiko spotted in drive-by review. - "Honor XA_FLAGS_ACCOUNT in xas_split_alloc() and charge to folio's memcg" (Zi Yan) Fix xas_split_alloc() by enabling target folio memcg charging during splits and adding the missing __GFP_ACCOUNT flag for proper XArray node memory accounting. - "selftests/mm: use pattern matching in .gitignore" (Pratyush Mallick) Replace hardcoded binary names in selftests/mm/.gitignore with a generic pattern-matching rule to automatically ignore generated test files and avoid manual updates when adding new tests. - "mm/page_ext: remove pgdat_page_ext_init()" (Sang-Heon Jeon) Make the incompatibility between FLATMEM and NUMA explicit in mm/Kconfig and remove the unused pgdat_page_ext_init() function. - "zram: fix zstd error paths and add parameter validation" (Haoqin Huang) Clean up zram compression backends by removing redundant error cleanup, adding parameter and dictionary validation, auto-prefixing algorithm error logs, and resetting parameters prior to reinitialization. - "zram: fix stale scan bounds after reinitialization" (Longlong Xia) Prevent out-of-bounds slot accesses during concurrent zram resets by moving table scan bound calculations under dev_lock in writeback_store() and read_block_state(). - "add anon mTHP collapse test cases" (Baolin Wang) Extend selftests helper functions to support arbitrary page orders and add new test cases and options for mTHP collapse in khugepaged. - "selftests/mm: Handle unsupported and transient test conditions" (Muhammad Usama Anjum) Update MM selftests to report a SKIP status instead of a failure when required kernel or filesystem features are unsupported, while adding retry logic for transient page migration errors. - "mm/zswap: Fixes and improves the zswap shrink" (Hao Jia) Fix the missing zswap global shrinker when CONFIG_MEMCG is disabled and extend shrink_memcg() to support batch writeback for improved writeback efficiency. - "alloc_tag: introduce IOCTL-based filtering for MAP" (Suren Baghdasaryan) Introduce an IOCTL-based binary interface for memory allocation profiling that enables kernel-side filtering before per-CPU counter aggregation. This eliminates the text-parsing overhead of /proc/allocinfo and provides up to a 20x speedup by transferring only filtered allocation data to userspace. - "better block swap batching and a different take on swap_ops v5" (Christoph Hellwig) Refactor block swap I/O to use swap_iocb for batching instead of single-bio requests and rebase the swap_ops interface, achieving faster swap throughput during kernel builds. - "mm: kmemleak: reduce transient false positives by confirming leaks" (Catalin Marinas) Reduce false-positive kmemleak reports by combining two kmemleak enhancements that add a second confirmation scan and a configurable minimum unreferenced scan count module parameter. - "mm: kmemleak: default min_unref_scans to 2 for verbose kernels" (Breno Leitao) Auto-scanning kernels can generate false-positive memory leak reports on single scans, so this patch defaults min_unref_scans to 2 when CONFIG_DEBUG_KMEMLEAK_VERBOSE is enabled to require a second confirming scan. - "swap_ops updates" (Christoph Hellwig) Batching I/O for synchronous swap devices causes performance regressions and filesystem-based swap suffers from double-indirection overhead. This series resolves both issues by reintroducing per-folio writes for synchronous swap and allowing filesystems to directly export their own swap_ops. - "mm/khugepaged: several cleanups" (Nico Pache) khugepaged accumulated redundant state-checking patterns and outdated comments following mTHP integration. Introduce dedicated helpers for PTE validation and event counting while refreshing the internal documentation. - "maple_tree: lock checking and clean ups" (Liam Howlett) Syzbot reports incorrectly blame memory management exit paths for locking bugs, maple tree erase operations risk allocation failures without gfp flags and internal documentation lacks clarity. Improve lock error detection, update docs, fix race and allocation edge cases and optimize erase allocations using a fallback to GFP_KERNEL | GFP_NOFAIL. * tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (172 commits) selftests/proc: make proc-maps-race work with READ_IMPLIES_EXEC memcg: move LRU size accounting on reparenting instead of copying it mm/vmscan: fix comment logic in balance_pgdat maple_tree: add helper mas_make_walkable() maple_tree: avoid extra gap calculation maple_tree: fix argument name in header maple_tree: change two GFP flags in tests maple_tree: document erase and allocations better maple_tree: avoid mas_erase() and mtree_erase() failures maple_tree: document that erase may use GFP_KERNEL for allocations maple_tree: catch race in mas_alloc_cyclic() maple_tree: add bulk parent set helper maple_tree: micro optimisation of mas_wr_store_type() maple_tree: optimise mas_wr_node_store() when not in rcu mode maple_tree: use prefetched value in mas_wr_store_type() maple_tree: clarify comments on mas_nomem() maple_tree: drop MAPLE_ALLOC_SLOTS maple_tree: drop dead code from mas_extend_spanning_null() maple_tree: documentation fix maple_tree: add write lock checking with lockdep sequence numbers ... 
Pull more MM updates from Andrew Morton:

 - "mm/rmap: index MAP_PRIVATE file-backed folios by anonymous pgoff"
   (Lorenzo Stoakes)

   Index MAP_PRIVATE file-backed folios by their anonymous page offset
   to resolve confusion around reverse mapping for zeroed and CoW'd
   file-backed memory.

   Use this new VMA anonymous page offset tracking to eliminate index
   conflicts and lay the foundation for scalable CoW performance
   improvements.

 - "promote mapped executable folios after first usage for MGLRU"
   (Baolin Wang)

   Make MGLRU's protection of mapped executable file folios more
   reliable. Follow the classical LRU's logic, promoting mapped
   executable file folios after their first usage to give executable
   code a better chance to stay in memory and improve workload
   performance.

 - "mm: vmscan: fix node reclaim ignoring swappiness parameter" (Ridong
   Chen)

   Fix per-node proactive reclaim interface's ignoring the swappiness
   parameter when CONFIG_MEMCG is disabled by consolidating
   sc_swappiness() into a single function that checks
   proactive_swappiness regardless of kernel configuration.

 - "mm/vmscan: reduce lru_lock contention via vmstat-derived
   scan-balance cost" (Usama Arif)

   Reduce lru_lock contention in the reclaim path by deriving
   scan-balance costs from vmstat counters rather than lock-acquired
   producer updates.

   Read and decay these cost signals on the reclaim side under a
   dedicated per-lruvec lock, reducing total LRU lock wait time by over
   60% without impacting scan throughput.

 - "zram: fix zram issues reported by sashiko" (Sergey Senozhatsky)

   Fix two low-risk zram bugs which Sashiko spotted in drive-by review.

 - "Honor XA_FLAGS_ACCOUNT in xas_split_alloc() and charge to folio's
   memcg" (Zi Yan)

   Fix xas_split_alloc() by enabling target folio memcg charging during
   splits and adding the missing __GFP_ACCOUNT flag for proper XArray
   node memory accounting.

 - "selftests/mm: use pattern matching in .gitignore" (Pratyush Mallick)

   Replace hardcoded binary names in selftests/mm/.gitignore with a
   generic pattern-matching rule to automatically ignore generated test
   files and avoid manual updates when adding new tests.

 - "mm/page_ext: remove pgdat_page_ext_init()" (Sang-Heon Jeon)

   Make the incompatibility between FLATMEM and NUMA explicit in
   mm/Kconfig and remove the unused pgdat_page_ext_init() function.

 - "zram: fix zstd error paths and add parameter validation" (Haoqin
   Huang)

   Clean up zram compression backends by removing redundant error
   cleanup, adding parameter and dictionary validation, auto-prefixing
   algorithm error logs, and resetting parameters prior to
   reinitialization.

 - "zram: fix stale scan bounds after reinitialization" (Longlong Xia)

   Prevent out-of-bounds slot accesses during concurrent zram resets by
   moving table scan bound calculations under dev_lock in
   writeback_store() and read_block_state().

 - "add anon mTHP collapse test cases" (Baolin Wang)

   Extend selftests helper functions to support arbitrary page orders
   and add new test cases and options for mTHP collapse in khugepaged.

 - "selftests/mm: Handle unsupported and transient test conditions"
   (Muhammad Usama Anjum)

   Update MM selftests to report a SKIP status instead of a failure when
   required kernel or filesystem features are unsupported, while adding
   retry logic for transient page migration errors.

 - "mm/zswap: Fixes and improves the zswap shrink" (Hao Jia)

   Fix the missing zswap global shrinker when CONFIG_MEMCG is disabled
   and extend shrink_memcg() to support batch writeback for improved
   writeback efficiency.

 - "alloc_tag: introduce IOCTL-based filtering for MAP" (Suren
   Baghdasaryan)

   Introduce an IOCTL-based binary interface for memory allocation
   profiling that enables kernel-side filtering before per-CPU counter
   aggregation.

   This eliminates the text-parsing overhead of /proc/allocinfo and
   provides up to a 20x speedup by transferring only filtered allocation
   data to userspace.

 - "better block swap batching and a different take on swap_ops v5"
   (Christoph Hellwig)

   Refactor block swap I/O to use swap_iocb for batching instead of
   single-bio requests and rebase the swap_ops interface, achieving
   faster swap throughput during kernel builds.

 - "mm: kmemleak: reduce transient false positives by confirming leaks"
   (Catalin Marinas)

   Reduce false-positive kmemleak reports by combining two kmemleak
   enhancements that add a second confirmation scan and a configurable
   minimum unreferenced scan count module parameter.

 - "mm: kmemleak: default min_unref_scans to 2 for verbose kernels"
   (Breno Leitao)

   Auto-scanning kernels can generate false-positive memory leak reports
   on single scans, so this patch defaults min_unref_scans to 2 when
   CONFIG_DEBUG_KMEMLEAK_VERBOSE is enabled to require a second
   confirming scan.

 - "swap_ops updates" (Christoph Hellwig)

   Batching I/O for synchronous swap devices causes performance
   regressions and filesystem-based swap suffers from double-indirection
   overhead. This series resolves both issues by reintroducing per-folio
   writes for synchronous swap and allowing filesystems to directly
   export their own swap_ops.

 - "mm/khugepaged: several cleanups" (Nico Pache)

   khugepaged accumulated redundant state-checking patterns and outdated
   comments following mTHP integration. Introduce dedicated helpers for
   PTE validation and event counting while refreshing the internal
   documentation.

 - "maple_tree: lock checking and clean ups" (Liam Howlett)

   Syzbot reports incorrectly blame memory management exit paths for
   locking bugs, maple tree erase operations risk allocation failures
   without gfp flags and internal documentation lacks clarity.

   Improve lock error detection, update docs, fix race and allocation
   edge cases and optimize erase allocations using a fallback to
   GFP_KERNEL | GFP_NOFAIL.

* tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (172 commits)
  selftests/proc: make proc-maps-race work with READ_IMPLIES_EXEC
  memcg: move LRU size accounting on reparenting instead of copying it
  mm/vmscan: fix comment logic in balance_pgdat
  maple_tree: add helper mas_make_walkable()
  maple_tree: avoid extra gap calculation
  maple_tree: fix argument name in header
  maple_tree: change two GFP flags in tests
  maple_tree: document erase and allocations better
  maple_tree: avoid mas_erase() and mtree_erase() failures
  maple_tree: document that erase may use GFP_KERNEL for allocations
  maple_tree: catch race in mas_alloc_cyclic()
  maple_tree: add bulk parent set helper
  maple_tree: micro optimisation of mas_wr_store_type()
  maple_tree: optimise mas_wr_node_store() when not in rcu mode
  maple_tree: use prefetched value in mas_wr_store_type()
  maple_tree: clarify comments on mas_nomem()
  maple_tree: drop MAPLE_ALLOC_SLOTS
  maple_tree: drop dead code from mas_extend_spanning_null()
  maple_tree: documentation fix
  maple_tree: add write lock checking with lockdep sequence numbers
  ...
locking/lockdep: add sequence counter to held_lock 2026-08-25T01:43:28+00:00 Liam R. Howlett (Oracle) liam@infradead.org 2026-08-21T19:26:10+00:00 8f2109843137da8068f19a120d79e58ef3838429 Add an 8 bit small sequence counter to the held_lock struct to detect if the lock as been dropped and reacquired. This is useful when a data structure depends on a constant locking context, but is not able to detect locking and unlocking of the lock through its own API. Since the __lock_unpin_lock() will no longer detect underflow by casting the unsigned int to a signed int, update the casting code to use a temp variable for calculations using a signed int. Link: https://lore.kernel.org/20260821192627.4085470-3-liam@infradead.org Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org> Suggested-by: Peter Zijlstra <peterz@infradead.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Will Deacon <will@kernel.org> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Waiman Long <longman@redhat.com> Link: https://lore.kernel.org/all/h3tpnj5kzcrxms5picmimtkpg4aypcpip5wbd6bt2rpdj5k7eb@nhtzs3lefrkq/ Cc: Breno Leitao <leitao@debian.org> Cc: Chris Mason <clm@meta.com> Cc: Chuck Lever <cel@kernel.org> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Joe Perches <joe@perches.com> Cc: Rik van Riel <riel@surriel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> 
Add an 8 bit small sequence counter to the held_lock struct to detect if
the lock as been dropped and reacquired.  This is useful when a data
structure depends on a constant locking context, but is not able to detect
locking and unlocking of the lock through its own API.

Since the __lock_unpin_lock() will no longer detect underflow by casting
the unsigned int to a signed int, update the casting code to use a temp
variable for calculations using a signed int.

Link: https://lore.kernel.org/20260821192627.4085470-3-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Will Deacon <will@kernel.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Waiman Long <longman@redhat.com>
Link: https://lore.kernel.org/all/h3tpnj5kzcrxms5picmimtkpg4aypcpip5wbd6bt2rpdj5k7eb@nhtzs3lefrkq/
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Rik van Riel <riel@surriel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Merge tag 'mm-nonmm-stable-2026-08-22-16-57' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm 2026-08-23T15:07:11+00:00 Linus Torvalds torvalds@linux-foundation.org 2026-08-23T15:07:11+00:00 e5f92606156a6a823992294d214c285b49cd72e9 Pull non-MM updates from Andrew Morton: - "ocfs2/dlm: bound peer-controlled lengths in the o2dlm" (Bryam Vargas) Validate and bound all input lengths and count fields in the o2dlm migration and recovery receive handlers to prevent memory corruption and kernel panics from malformed cluster messages - "ocfs2: validate xattr entry bounds" (Cen Zhang) Validate OCFS2 extended attribute entry name and value bounds during metadata reads to prevent out-of-range memory accesses during retrieval or listing operations. - "taskstats: fix cgroupstats invalid fd handling and add selftests" (Yiyang Chen) Return -EBADF when cgroupstats receives an invalid file descriptor to prevent caller hangs and misleading success ACKs. Add a kselftest to validate valid cgroup v1 queries and verify proper error handling across different Netlink flag combinations. - "misc lib/raid/ improvements v2" (Christoph Hellwig) Improve benchmark-based algorithm selection for the XOR and RAID6 libraries, add KUnit benchmark tests, and cleanup minor implementation details. - "ocfs2: cluster: o2hb_region_pin() fixes" (Joseph Qi) Fix sleeping-in-atomic, lock order inversion and error-path cleanup bugs in o2hb_region_pin() by releasing o2hb_live_lock across sleeping configfs_depend_item() calls and using unlocked variants from callback context. Ensure failed pin attempts properly decrement user counts and unpin partially initialized heartbeat regions to prevent memory leaks and unprotected states. - "lib/ucs2_string.c: fix out-of-bounds read in ucs2_strnlen()" (Vincent Mailhol) Fix an off-by-one which could cause an out-of-bounds read. - "ocfs2: harden heartbeat teardown races" (Cen Zhang) Fix two OCFS2 heartbeat/o2net teardown races found by KASAN. - "taskstats: tidy up the cpumask command path" *Bradley Morgan) make two small cleanups in kernel/taskstats.c. - "ocfs2: validate active orphan slots during inode read" (ZhengYuan Huang) Validate active ordinary and append-DIO orphan slots read from OCFS2 dinodes at the metadata boundary to prevent corrupted slot indices from causing out-of-bounds array accesses. - "ocfs2: bound-check both readdir re-validation scans" (Zhan Xusheng) Enforce strict boundary checks on directory entry record lengths and offset calculations during OCFS2 directory re-scans to prevent out-of-bounds memory reads and directory position corruption. * tag 'mm-nonmm-stable-2026-08-22-16-57' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (95 commits) mailmap: fix bouncing address for Taniya Das ocfs2: bound-check dir entries in the inline-data re-validation scan ocfs2: bound-check dir entries in the readdir re-validation scan squashfs: avoid thundering-herd cache wakeups prctl: fix PR_SET_MM_AUXV losing the forced AT_NULL terminator mailmap: update email address for Linfeng Sun lib/interval_tree: fix allocation warning messages checkpatch: add NOKPROBE_SYMBOL to the whitelist of lines that can occur immediately after functions Squashfs: check block offset is not negative signal: factor out the kernel reserved si_code check ocfs2: fix readdir position truncation on 32-bit kernels ocfs2: fix cached cluster count after suballocator reclaim ocfs2: fix circular locking dependency in ocfs2_init_acl() ocfs2: validate DIO orphan slot during inode read ocfs2: validate orphan slot during inode read selftests/prctl: fix non-anonymous VMA mapping in set-anon-vma-name test MAINTAINERS: add IRC and patchwork for LTP include/linux/list.h: mark list_add and __list_add as __always_inline tools/mm: prevent page_owner_sort from truncating input hung_task: update DETECT_HUNG_TASK_BLOCKER Kconfig help ... 
Pull non-MM updates from Andrew Morton:

 - "ocfs2/dlm: bound peer-controlled lengths in the o2dlm" (Bryam
   Vargas)

   Validate and bound all input lengths and count fields in the o2dlm
   migration and recovery receive handlers to prevent memory corruption
   and kernel panics from malformed cluster messages

 - "ocfs2: validate xattr entry bounds" (Cen Zhang)

   Validate OCFS2 extended attribute entry name and value bounds during
   metadata reads to prevent out-of-range memory accesses during
   retrieval or listing operations.

 - "taskstats: fix cgroupstats invalid fd handling and add selftests"
   (Yiyang Chen)

   Return -EBADF when cgroupstats receives an invalid file descriptor to
   prevent caller hangs and misleading success ACKs. Add a kselftest to
   validate valid cgroup v1 queries and verify proper error handling
   across different Netlink flag combinations.

 - "misc lib/raid/ improvements v2" (Christoph Hellwig)

   Improve benchmark-based algorithm selection for the XOR and RAID6
   libraries, add KUnit benchmark tests, and cleanup minor
   implementation details.

 - "ocfs2: cluster: o2hb_region_pin() fixes" (Joseph Qi)

   Fix sleeping-in-atomic, lock order inversion and error-path cleanup
   bugs in o2hb_region_pin() by releasing o2hb_live_lock across sleeping
   configfs_depend_item() calls and using unlocked variants from
   callback context. Ensure failed pin attempts properly decrement user
   counts and unpin partially initialized heartbeat regions to prevent
   memory leaks and unprotected states.

 - "lib/ucs2_string.c: fix out-of-bounds read in ucs2_strnlen()"
   (Vincent Mailhol)

   Fix an off-by-one which could cause an out-of-bounds read.

 - "ocfs2: harden heartbeat teardown races" (Cen Zhang)

   Fix two OCFS2 heartbeat/o2net teardown races found by KASAN.

 - "taskstats: tidy up the cpumask command path" *Bradley Morgan)

   make two small cleanups in kernel/taskstats.c.

 - "ocfs2: validate active orphan slots during inode read" (ZhengYuan
   Huang)

   Validate active ordinary and append-DIO orphan slots read from OCFS2
   dinodes at the metadata boundary to prevent corrupted slot indices
   from causing out-of-bounds array accesses.

 - "ocfs2: bound-check both readdir re-validation scans" (Zhan Xusheng)

   Enforce strict boundary checks on directory entry record lengths and
   offset calculations during OCFS2 directory re-scans to prevent
   out-of-bounds memory reads and directory position corruption.

* tag 'mm-nonmm-stable-2026-08-22-16-57' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (95 commits)
  mailmap: fix bouncing address for Taniya Das
  ocfs2: bound-check dir entries in the inline-data re-validation scan
  ocfs2: bound-check dir entries in the readdir re-validation scan
  squashfs: avoid thundering-herd cache wakeups
  prctl: fix PR_SET_MM_AUXV losing the forced AT_NULL terminator
  mailmap: update email address for Linfeng Sun
  lib/interval_tree: fix allocation warning messages
  checkpatch: add NOKPROBE_SYMBOL to the whitelist of lines that can occur immediately after functions
  Squashfs: check block offset is not negative
  signal: factor out the kernel reserved si_code check
  ocfs2: fix readdir position truncation on 32-bit kernels
  ocfs2: fix cached cluster count after suballocator reclaim
  ocfs2: fix circular locking dependency in ocfs2_init_acl()
  ocfs2: validate DIO orphan slot during inode read
  ocfs2: validate orphan slot during inode read
  selftests/prctl: fix non-anonymous VMA mapping in set-anon-vma-name test
  MAINTAINERS: add IRC and patchwork for LTP
  include/linux/list.h: mark list_add and __list_add as __always_inline
  tools/mm: prevent page_owner_sort from truncating input
  hung_task: update DETECT_HUNG_TASK_BLOCKER Kconfig help
  ...
Merge tag 'cgroup-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup 2026-08-20T17:37:42+00:00 Linus Torvalds torvalds@linux-foundation.org 2026-08-20T17:37:42+00:00 40d8c81577db09b71ee5402ba336b642d32d6a82 Pull cgroup updates from Tejun Heo: - Attach path bug fixes: migrations spanning multiple source or destination cpusets were mishandled, most visibly leaving thread affinities stale when the controller is disabled in a threaded subtree. Configuration writes could also race an in-flight attach and apply stale state, and the deadline task count could get corrupted by concurrent updates, skewing SCHED_DEADLINE admission decisions. - Memory binding bug fixes: which node masks get applied differed between the binding update paths, and tasks cloned with CLONE_INTO_CGROUP skipped rebinding entirely. Rebinding also now runs once per process instead of repeating for every thread sharing the mm. - Overhead removals with no behavior change: CPU hotplug iterated tasks of cpusets that just inherit the parent's effective masks, and the slab-spreading task flag was still being maintained although the SLAB allocator that consumed it is long gone. - Data-race annotations for benign races so that KCSAN reports stay meaningful, selftest coverage for the fixes above along with flakiness and portability fixes, and documentation corrections. * tag 'cgroup-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup: (34 commits) selftests/cgroup: Remove redundant chown in test_cgcore_lesser_ns_open selftests/cgroup: Preserve CPU hotplug write errors cgroup/cpuset: Add test for partition root invalidation returning wrong CPUs cgroup/cpuset: Remove obsolete PFA_SPREAD_SLAB task flag docs: cgroup-v2: fix stale "io" controller introduction selftests/cgroup: Avoid awk -e in cpuset tests cgroup/cpuset: Use WRITE_ONCE() for shared prs_err updates selftests/cgroup: add user_usec sanity check in test_cpucg_nice cgroup: drop unneeded semicolon docs: cgroup-v2: mark memory.pressure and io.pressure as read-write selftests/cgroup: Fix minor defects in test_cpuset Docs/admin-guide/cgroup-v2: fix delay_nsec unit in io.latency doc selftests/cgroup: Remove redundant cg_enter_current() call in test_core selftests/cgroup: Add test for cpuset affinity on controller disable cgroup/cpuset: Handle the special case of non-moving tasks in cpuset_can_attach() cgroup/cpuset: Support multiple destination cpusets for cpuset_*attach() selftests/cgroup: fix missing TAP output in test_hugetlb_memcg cgroup/cpuset: Support multiple source cpusets for cpuset_*attach() cgroup/cpuset: Move mpol_rebind_mm/cpuset_migrate_mm() calls inside cpuset_attach_task() cgroup/cpuset: Make attach_ctx.old_cs track task group leader ... 
Pull cgroup updates from Tejun Heo:

 - Attach path bug fixes: migrations spanning multiple source or
   destination cpusets were mishandled, most visibly leaving thread
   affinities stale when the controller is disabled in a threaded
   subtree. Configuration writes could also race an in-flight attach and
   apply stale state, and the deadline task count could get corrupted by
   concurrent updates, skewing SCHED_DEADLINE admission decisions.

 - Memory binding bug fixes: which node masks get applied differed
   between the binding update paths, and tasks cloned with
   CLONE_INTO_CGROUP skipped rebinding entirely. Rebinding also now runs
   once per process instead of repeating for every thread sharing the
   mm.

 - Overhead removals with no behavior change: CPU hotplug iterated tasks
   of cpusets that just inherit the parent's effective masks, and the
   slab-spreading task flag was still being maintained although the SLAB
   allocator that consumed it is long gone.

 - Data-race annotations for benign races so that KCSAN reports stay
   meaningful, selftest coverage for the fixes above along with
   flakiness and portability fixes, and documentation corrections.

* tag 'cgroup-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup: (34 commits)
  selftests/cgroup: Remove redundant chown in test_cgcore_lesser_ns_open
  selftests/cgroup: Preserve CPU hotplug write errors
  cgroup/cpuset: Add test for partition root invalidation returning wrong CPUs
  cgroup/cpuset: Remove obsolete PFA_SPREAD_SLAB task flag
  docs: cgroup-v2: fix stale "io" controller introduction
  selftests/cgroup: Avoid awk -e in cpuset tests
  cgroup/cpuset: Use WRITE_ONCE() for shared prs_err updates
  selftests/cgroup: add user_usec sanity check in test_cpucg_nice
  cgroup: drop unneeded semicolon
  docs: cgroup-v2: mark memory.pressure and io.pressure as read-write
  selftests/cgroup: Fix minor defects in test_cpuset
  Docs/admin-guide/cgroup-v2: fix delay_nsec unit in io.latency doc
  selftests/cgroup: Remove redundant cg_enter_current() call in test_core
  selftests/cgroup: Add test for cpuset affinity on controller disable
  cgroup/cpuset: Handle the special case of non-moving tasks in cpuset_can_attach()
  cgroup/cpuset: Support multiple destination cpusets for cpuset_*attach()
  selftests/cgroup: fix missing TAP output in test_hugetlb_memcg
  cgroup/cpuset: Support multiple source cpusets for cpuset_*attach()
  cgroup/cpuset: Move mpol_rebind_mm/cpuset_migrate_mm() calls inside cpuset_attach_task()
  cgroup/cpuset: Make attach_ctx.old_cs track task group leader
  ...
Merge tag 'smp-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2026-08-18T22:29:53+00:00 Linus Torvalds torvalds@linux-foundation.org 2026-08-18T22:29:53+00:00 0dd1a54f44348d9cf6bae57a2b5cb0b53826a2c7 Pull SMP core updates from Thomas Gleixner: - Reduce the preemption disabled sections in smp_call_function*(). The various smp call functions keep preemption disabled accross the full operation which includes the wait for completion. Especially the latter can take some time when one of the target CPUs is not immediately responding to the IPI, which can result in large latency spikes. To improve this provide a per task CPU mask to track the CPUs to wait for. That makes the information required for the wait task local and therefore allows to reenable preemption before the wait. While this comes with moderate extra memory cost this reduces SMP function call induced latency measured in a fleet for high priority tasks from ~17ms to ~1.5ms (~90%). - Reduce the overhead of the CSD debug code by replacing the heavy memory barriers with smp_store_release()/acquire() - Remove obsolute unused hotplug states * tag 'smp-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: scftorture: Remove preempt_disable() in scftorture_invoke_one() smp: Remove preempt_disable() from on_each_cpu_cond_mask() smp: Remove preempt_disable() from smp_call_function() smp: Enable preemption early in smp_call_function_many_cond() smp: Alloc percpu csd data in smpcfd_prepare_cpu() only once smp: Use task-local IPI cpumask in smp_call_function_many_cond() smp: Refactor remote CPU selection in smp_call_function_any() smp: Enable preemption early in smp_call_function_single() smp: Disable preemption explicitly in __csd_lock_wait() cpu/hotplug: Remove CPUHP_AP_ARM_CORESIGHT_CTI_STARTING smp: Use release stores for csd_lock_record() state 
Pull SMP core updates from Thomas Gleixner:

 - Reduce the preemption disabled sections in smp_call_function*().

   The various smp call functions keep preemption disabled accross the
   full operation which includes the wait for completion. Especially the
   latter can take some time when one of the target CPUs is not
   immediately responding to the IPI, which can result in large latency
   spikes.

   To improve this provide a per task CPU mask to track the CPUs to wait
   for. That makes the information required for the wait task local and
   therefore allows to reenable preemption before the wait.

   While this comes with moderate extra memory cost this reduces SMP
   function call induced latency measured in a fleet for high priority
   tasks from ~17ms to ~1.5ms (~90%).

 - Reduce the overhead of the CSD debug code by replacing the heavy
   memory barriers with smp_store_release()/acquire()

 - Remove obsolute unused hotplug states

* tag 'smp-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  scftorture: Remove preempt_disable() in scftorture_invoke_one()
  smp: Remove preempt_disable() from on_each_cpu_cond_mask()
  smp: Remove preempt_disable() from smp_call_function()
  smp: Enable preemption early in smp_call_function_many_cond()
  smp: Alloc percpu csd data in smpcfd_prepare_cpu() only once
  smp: Use task-local IPI cpumask in smp_call_function_many_cond()
  smp: Refactor remote CPU selection in smp_call_function_any()
  smp: Enable preemption early in smp_call_function_single()
  smp: Disable preemption explicitly in __csd_lock_wait()
  cpu/hotplug: Remove CPUHP_AP_ARM_CORESIGHT_CTI_STARTING
  smp: Use release stores for csd_lock_record() state
Merge tag 'sched-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2026-08-18T20:39:48+00:00 Linus Torvalds torvalds@linux-foundation.org 2026-08-18T20:39:48+00:00 e2457a664ea02c414c6b9828bff3a0df4c300f63 Pull scheduler updates from Ingo Molnar: "Load-balancing updates: - 'flatten the pick': improve cgroup scheduling, which has always been problematic and painful, which has caused various scheduling misbehavior such as the mishandling of reniced tasks et al. Add various cgroup weight distribution methods via cgroup_mode: 'up', 'max', 'concur' and 'tasks' - with the default being 'concur' which is the most precise yet also most expensive version. Finally, change cgroup scheduling to a single runqueue (Peter Zijlstra) - Series to improve the scheduling latency of short slice tasks (Vincent Guittot) - Series to fix cluster scheduling in the presence of asymmetric capacity (Ricardo Neri) - Prefer fully idle cores for NOHZ balancing (Andrea Righi) - Don't trigger active load-balancing if src_rq->curr is not on_rq (Xin Zhao) PSI updates: - Skip irqtime accounting when no new irq time has elapsed (Usama Arif) Scheduler debugging updates: - Remove unused schedstats (Shrikanth Hegde) - Defer freeing of cpumask memblock memory to initcall (Waiman Long) Misc fixes and updates by Yu C Chen, K Prateek Nayak, Peter Zijlstra, Vincent Guittot, Xin Zhao, Yury Norov, Zhan Xusheng" * tag 'sched-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (33 commits) sched/fair: Fix flat hierarchy sched/isolation: Defer freeing of cpumask memblock memory to initcall sched/topology: Restore SD_PREFER_SIBLING in domains with asymmetric capacity sched/fair: Allow load balancing between CPUs of identical capacity sched/fair: Skip misfit load accounting when the destination CPU cannot help sched/fair: Check CPU capacity before comparing group types during load balance sched/fair: Also gate overloaded status update for SD_ASYM_CPUCAPACITY sched/fair: Do not skip CPUs of similar capacity with busy SMT siblings sched/fair: Prefer fully idle cores for NOHZ balancing stop_machine: Make stop_one_cpu_nowait() return void sched/eevdf: Delayed dequeue task can't preempt sched/fair: Fix stale comments referring to removed CFS concepts sched/debug: Remove unused schedstats sched/psi: skip irqtime accounting when no new irq time has elapsed sched/fair: Reflow sched_balance_rq() sched/fair: Simplify balance_interval reset logic in sched_balance_rq() sched/fair: Don't trigger active lb if src_rq->curr is not on_rq sched/eevdf: Speedup short slice task scheduling sched/eevdf: Always update slice protection sched/eevdf: Cancel slice protection if short slice task is eligible ... 
Pull scheduler updates from Ingo Molnar:
 "Load-balancing updates:

   - 'flatten the pick': improve cgroup scheduling, which has always
     been problematic and painful, which has caused various scheduling
     misbehavior such as the mishandling of reniced tasks et al.

     Add various cgroup weight distribution methods via cgroup_mode:
     'up', 'max', 'concur' and 'tasks' - with the default being 'concur'
     which is the most precise yet also most expensive version.

     Finally, change cgroup scheduling to a single runqueue (Peter
     Zijlstra)

   - Series to improve the scheduling latency of short slice tasks
     (Vincent Guittot)

   - Series to fix cluster scheduling in the presence of asymmetric
     capacity (Ricardo Neri)

   - Prefer fully idle cores for NOHZ balancing (Andrea Righi)

   - Don't trigger active load-balancing if src_rq->curr is not on_rq
     (Xin Zhao)

  PSI updates:

   - Skip irqtime accounting when no new irq time has elapsed (Usama
     Arif)

  Scheduler debugging updates:

   - Remove unused schedstats (Shrikanth Hegde)

   - Defer freeing of cpumask memblock memory to initcall (Waiman Long)

  Misc fixes and updates by Yu C Chen, K Prateek Nayak, Peter Zijlstra,
  Vincent Guittot, Xin Zhao, Yury Norov, Zhan Xusheng"

* tag 'sched-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (33 commits)
  sched/fair: Fix flat hierarchy
  sched/isolation: Defer freeing of cpumask memblock memory to initcall
  sched/topology: Restore SD_PREFER_SIBLING in domains with asymmetric capacity
  sched/fair: Allow load balancing between CPUs of identical capacity
  sched/fair: Skip misfit load accounting when the destination CPU cannot help
  sched/fair: Check CPU capacity before comparing group types during load balance
  sched/fair: Also gate overloaded status update for SD_ASYM_CPUCAPACITY
  sched/fair: Do not skip CPUs of similar capacity with busy SMT siblings
  sched/fair: Prefer fully idle cores for NOHZ balancing
  stop_machine: Make stop_one_cpu_nowait() return void
  sched/eevdf: Delayed dequeue task can't preempt
  sched/fair: Fix stale comments referring to removed CFS concepts
  sched/debug: Remove unused schedstats
  sched/psi: skip irqtime accounting when no new irq time has elapsed
  sched/fair: Reflow sched_balance_rq()
  sched/fair: Simplify balance_interval reset logic in sched_balance_rq()
  sched/fair: Don't trigger active lb if src_rq->curr is not on_rq
  sched/eevdf: Speedup short slice task scheduling
  sched/eevdf: Always update slice protection
  sched/eevdf: Cancel slice protection if short slice task is eligible
  ...
cgroup/cpuset: Remove obsolete PFA_SPREAD_SLAB task flag 2026-08-12T01:43:08+00:00 Guopeng Zhang zhangguopeng@kylinos.cn 2026-08-11T08:30:27+00:00 6dd5d93f6c48a461157c73b9e2ce7f6d19d57f5a Commit 16a1d968358a ("mm/slab: remove mm/slab.c and slab_def.h") removed the SLAB allocator, the only allocator that implemented cpuset slab spreading. Commit 61a182ab61a6 ("cgroup/cpuset: Remove cpuset_do_slab_mem_spread()") then removed the last task_spread_slab() caller. Commit 3ab67a9ce82f ("cgroup/cpuset: Mark memory_spread_slab as obsolete") marked the legacy control obsolete. cpuset still updates PFA_SPREAD_SLAB when tasks attach to a legacy cpuset and walks all tasks in a cpuset when memory_spread_slab changes. Remove the unused task flag and its helpers, and make spread task updates depend only on memory_spread_page. Keep the memory_spread_slab control and CS_SPREAD_SLAB state so legacy users retain the existing write, readback and inheritance behavior. Update the comments and documentation to describe only page-cache spreading as functional. Assisted-by: LLM Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Reviewed-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org> 
Commit 16a1d968358a ("mm/slab: remove mm/slab.c and slab_def.h")
removed the SLAB allocator, the only allocator that implemented cpuset
slab spreading. Commit 61a182ab61a6 ("cgroup/cpuset: Remove
cpuset_do_slab_mem_spread()") then removed the last task_spread_slab()
caller. Commit 3ab67a9ce82f ("cgroup/cpuset: Mark memory_spread_slab as
obsolete") marked the legacy control obsolete.

cpuset still updates PFA_SPREAD_SLAB when tasks attach to a legacy
cpuset and walks all tasks in a cpuset when memory_spread_slab changes.
Remove the unused task flag and its helpers, and make spread task
updates depend only on memory_spread_page.

Keep the memory_spread_slab control and CS_SPREAD_SLAB state so legacy
users retain the existing write, readback and inheritance behavior.
Update the comments and documentation to describe only page-cache
spreading as functional.

Assisted-by: LLM
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
Reviewed-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
kcov: fix data corruption and race conditions on PREEMPT_RT 2026-08-05T18:20:13+00:00 Tetsuo Handa penguin-kernel@I-love.SAKURA.ne.jp 2026-07-15T23:01:29+00:00 2eed77fdcb0cc48e8eccb2bcd4b7f2c6d650e84c syzbot is reporting KCOV state corruption on PREEMPT_RT kernels, for the temporary storage used for saving/restoring remote KCOV state is currently allocated as the per-CPU area. On PREEMPT_RT kernels, softirq handlers run as preemptible task threads (e.g., ksoftirqd). If a softirq context preempts a task running a remote KCOV session, it safely saves the task's state into the per-CPU area. However, if that softirq thread is subsequently preempted by a higher- priority softirq thread on the same CPU, the second softirq will overwrite the same per-CPU area, permanently destroying the original task's KCOV state. Fix this data corruption by moving the temporary storage from the per-CPU area to the per-thread area. Since each softirq thread now owns its own task context, nested softirq preemption no longer causes data overwrites. Note that while the temporary storage is now on a per-thread basis, the per-CPU kcov_percpu_data.lock must be retained, for we need to ensure that kcov_remote_start() and kcov_remote_stop() operate atomically without racing against asynchronous interrupts that manipulate the current task's KCOV state. It is likely that GFP_KERNEL allocation by vmalloc_node() in kcov_init() has already called panic() before returning NULL, for there will be no OOM-killable userspace processes when __init function of built-in module runs. But this patch also fixes crashing the kernel when vmalloc_node() in kcov_init() returned NULL, for kcov_init() left per-CPU irq_area == NULL but kcov_remote_start() depends on per-CPU irq_area != NULL, resulting in (1) doing vmalloc() in kcov_remote_start() despite !in_task() context (2) out-of-array-bounds access if (1) succeeded but kcov->remote_size < CONFIG_KCOV_IRQ_AREA_SIZE (3) always leak memory allocated by (1), eventually killing all OOM-killable userspace processes problems. Link: https://lore.kernel.org/43552d09-2ce2-4b19-b0d3-a2d1ab952145@I-love.SAKURA.ne.jp Reported-by: syzbot+3f51ad7ac3ae57a6fdcc@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=3f51ad7ac3ae57a6fdcc Reported-by: syzbot+47cf95ca1f9dcca872c8@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=47cf95ca1f9dcca872c8 Reported-by: syzbot+8a173e13208949931dc7@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=8a173e13208949931dc7 Reported-by: syzbot+90984d3713722683112e@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=90984d3713722683112e Analyzed-by: AI Mode in Google Search (no mail address) Fixes: 5ff3b30ab57d ("kcov: collect coverage from interrupts") Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> Reviewed-by: Alexander Potapenko <glider@google.com> Cc: Alan Stern <stern@rowland.harvard.edu> Cc: Andrey Konovalov <andreyknvl@gmail.com> Cc: Christoph Hellwig <hch@infradead.org> Cc: Clark Williams <williams@redhat.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Marco Elver <elver@google.com> Cc: Mark Brown <broonie@kernel.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> 
syzbot is reporting KCOV state corruption on PREEMPT_RT kernels, for the
temporary storage used for saving/restoring remote KCOV state is currently
allocated as the per-CPU area.

On PREEMPT_RT kernels, softirq handlers run as preemptible task threads
(e.g., ksoftirqd). If a softirq context preempts a task running a remote
KCOV session, it safely saves the task's state into the per-CPU area.
However, if that softirq thread is subsequently preempted by a higher-
priority softirq thread on the same CPU, the second softirq will overwrite
the same per-CPU area, permanently destroying the original task's KCOV
state.

Fix this data corruption by moving the temporary storage from the per-CPU
area to the per-thread area. Since each softirq thread now owns its own
task context, nested softirq preemption no longer causes data overwrites.

Note that while the temporary storage is now on a per-thread basis, the
per-CPU kcov_percpu_data.lock must be retained, for we need to ensure that
kcov_remote_start() and kcov_remote_stop() operate atomically without
racing against asynchronous interrupts that manipulate the current task's
KCOV state.

It is likely that GFP_KERNEL allocation by vmalloc_node() in kcov_init()
has already called panic() before returning NULL, for there will be no
OOM-killable userspace processes when __init function of built-in module
runs. But this patch also fixes crashing the kernel when vmalloc_node()
in kcov_init() returned NULL, for kcov_init() left per-CPU irq_area == NULL
but kcov_remote_start() depends on per-CPU irq_area != NULL, resulting in

  (1) doing vmalloc() in kcov_remote_start() despite !in_task() context

  (2) out-of-array-bounds access if (1) succeeded but
      kcov->remote_size < CONFIG_KCOV_IRQ_AREA_SIZE

  (3) always leak memory allocated by (1), eventually killing all
      OOM-killable userspace processes

problems.

Link: https://lore.kernel.org/43552d09-2ce2-4b19-b0d3-a2d1ab952145@I-love.SAKURA.ne.jp
Reported-by: syzbot+3f51ad7ac3ae57a6fdcc@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=3f51ad7ac3ae57a6fdcc
Reported-by: syzbot+47cf95ca1f9dcca872c8@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=47cf95ca1f9dcca872c8
Reported-by: syzbot+8a173e13208949931dc7@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=8a173e13208949931dc7
Reported-by: syzbot+90984d3713722683112e@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=90984d3713722683112e
Analyzed-by: AI Mode in Google Search (no mail address)
Fixes: 5ff3b30ab57d ("kcov: collect coverage from interrupts")
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Reviewed-by: Alexander Potapenko <glider@google.com>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Clark Williams <williams@redhat.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Marco Elver <elver@google.com>
Cc: Mark Brown <broonie@kernel.org>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
smp: Use task-local IPI cpumask in smp_call_function_many_cond() 2026-07-16T07:24:55+00:00 Chuyi Zhou zhouchuyi@bytedance.com 2026-07-09T12:29:23+00:00 9a560af15fd31b3b93235b1584a2509445a17963 smp_call_function_many_cond() uses the per-CPU cfd->cpumask as the list of remote CPUs to wait for. That is safe while the caller remains pinned to the current CPU for the whole operation, because another task cannot run on the same CPU and reuse the per-CPU mask. The synchronous wait is the long-latency part of the operation. To make that wait preemptible, the mask iterated by csd_lock_wait() must remain stable even if the task is preempted or migrates. If the wait used the per-CPU cfd->cpumask after dropping CPU pinning, another task scheduled on the original CPU could enter smp_call_function_many_cond() and overwrite the mask while the first task is still iterating it. Give each task private IPI cpumask storage and use it as the wait mask in smp_call_function_many_cond(). Other cpumask storage choices do not fit this use case: - Per-CPU storage is the state that becomes unsafe once the wait is made preemptible. After the caller drops CPU pinning, another task scheduled on the original CPU can enter smp_call_function_many_cond() and reuse the same per-CPU mask. - Stack storage is not suitable for large NR_CPUS or CONFIG_CPUMASK_OFFSTACK=y configurations. The wait mask needs to scale with cpumask_size(), and putting that storage on the stack is not acceptable on large systems. - Allocating the mask inside smp_call_function_many_cond() would put an allocation and a failure path in the generic IPI path. A sleeping allocation is not suitable because callers have historically only provided a preempt-disabled context, not a sleepable one. GFP_ATOMIC would avoid sleeping, but a failure fallback would make the latency improvement opportunistic instead of guaranteed. The users are not limited to a small, pre-identifiable class of tasks. On x86, ordinary tasks can reach this path through TLB flushes during exit, unmap and reclaim, so allocating the mask only for a known subset of tasks is not straightforward. The memory cost is explicit: one word is added to task_struct. When cpumask_size() fits in that word, the mask is stored inline and no separate allocation is needed. Larger systems allocate cpumask_size() per task; on x86-64 NR_CPUS=8192 this is 1 KiB per task. For context, x86 already carries several KiB of per-task architecture and FPU state, depending on the enabled features and configuration. That does not make the extra cpumask free, but it puts the large-NR_CPUS case in perspective. Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com> Signed-off-by: Thomas Gleixner <tglx@kernel.org> Tested-by: Paul E. McKenney <paulmck@kernel.org> Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Link: https://patch.msgid.link/20260709122933.4021501-5-zhouchuyi@bytedance.com 
smp_call_function_many_cond() uses the per-CPU cfd->cpumask as the list
of remote CPUs to wait for. That is safe while the caller remains pinned
to the current CPU for the whole operation, because another task cannot
run on the same CPU and reuse the per-CPU mask.

The synchronous wait is the long-latency part of the operation. To make
that wait preemptible, the mask iterated by csd_lock_wait() must remain
stable even if the task is preempted or migrates. If the wait used the
per-CPU cfd->cpumask after dropping CPU pinning, another task scheduled
on the original CPU could enter smp_call_function_many_cond() and
overwrite the mask while the first task is still iterating it.

Give each task private IPI cpumask storage and use it as the wait mask in
smp_call_function_many_cond(). Other cpumask storage choices do not fit
this use case:

 - Per-CPU storage is the state that becomes unsafe once the wait is
   made preemptible. After the caller drops CPU pinning, another task
   scheduled on the original CPU can enter smp_call_function_many_cond()
   and reuse the same per-CPU mask.

 - Stack storage is not suitable for large NR_CPUS or
   CONFIG_CPUMASK_OFFSTACK=y configurations. The wait mask needs to
   scale with cpumask_size(), and putting that storage on the stack is
   not acceptable on large systems.

 - Allocating the mask inside smp_call_function_many_cond() would put an
   allocation and a failure path in the generic IPI path. A sleeping
   allocation is not suitable because callers have historically only
   provided a preempt-disabled context, not a sleepable one. GFP_ATOMIC
   would avoid sleeping, but a failure fallback would make the latency
   improvement opportunistic instead of guaranteed.

The users are not limited to a small, pre-identifiable class of tasks. On
x86, ordinary tasks can reach this path through TLB flushes during exit,
unmap and reclaim, so allocating the mask only for a known subset of
tasks is not straightforward.

The memory cost is explicit: one word is added to task_struct. When
cpumask_size() fits in that word, the mask is stored inline and no
separate allocation is needed. Larger systems allocate cpumask_size() per
task; on x86-64 NR_CPUS=8192 this is 1 KiB per task. For context,
x86 already carries several KiB of per-task architecture and FPU state,
depending on the enabled features and configuration. That does not make
the extra cpumask free, but it puts the large-NR_CPUS case in
perspective.

Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-5-zhouchuyi@bytedance.com
sched/debug: Remove unused schedstats 2026-06-30T08:56:57+00:00 Shrikanth Hegde sshegde@linux.ibm.com 2026-06-25T12:46:25+00:00 b2463ebf2674ddec62f0f0e63061670bc2c75346 nr_migrations_cold, nr_wakeups_passive and nr_wakeups_idle are not being updated anywhere. So remove them. These are per process stats. So updating sched stats version isn't necessary. Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com> Tested-by: K Prateek Nayak <kprateek.nayak@amd.com> Link: https://patch.msgid.link/20260625124648.802832-2-sshegde@linux.ibm.com 
nr_migrations_cold, nr_wakeups_passive and nr_wakeups_idle are not
being updated anywhere. So remove them.

These are per process stats. So updating sched stats version isn't
necessary.

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Link: https://patch.msgid.link/20260625124648.802832-2-sshegde@linux.ibm.com