linux-stable.git/kernel/sched, branch v7.2.6

sched_ext: Drop unlocked scx_rq_clock_invalidate() from scx_root_disable()

2026-09-14T11:40:47+00:00

[ Upstream commit d44093323131a42a85184315bb9b6da4813cc39b ]

scx_root_disable() invalidates each rq's clock before taking the rq lock.
scx_rq_clock_invalidate() is a plain read-modify-write of rq->scx.flags and
every other writer of the word runs under the rq lock, so the unlocked
update can race a concurrent flags update and lose one side's bits.

The invalidation doesn't matter in the first place. The cached clock is read
only by scx_bpf_now() from a loaded scheduler's BPF programs, nothing can
re-validate the clock while sched_ext is disabled as scx_rq_clock_update()
is gated on scx_enabled() too, and the usual rq lock cycles under the next
scheduler refresh or invalidate it before it's practically observable. Drop
the invalidation instead of fixing the locking.

v2: Description and comment updated - the invalidation is unnecessary rather
    than subsumed by the rq lock cycle below.

Fixes: 3a9910b5904d ("sched_ext: Implement scx_bpf_now()")
Signed-off-by: Tejun Heo 
Cc: Changwoo Min 
Signed-off-by: Sasha Levin

sched/isolation: Defer freeing of cpumask memblock memory to initcall

2026-09-14T11:40:44+00:00

[ Upstream commit 2b58c749b8c5244e259a0230bc57b10b010dc545 ]

When testing a linux-next kernel with commit 59bd1d914bb5 ("memblock:
warn when freeing reserved memory before memory map is initialized"),
the following warning was hit when there was a "nohz_full" kernel boot
parameter.

  Cannot free reserved memory because of deferred initialization of the memory map
  WARNING: mm/memblock.c:904 at __free_reserved_area+0xde/0xf0, CPU#0: swapper/0/0
    :
  Call Trace:
   
   memblock_phys_free+0xcb/0x100
   housekeeping_init+0x14c/0x170
   start_kernel+0x207/0x450
   x86_64_start_reservations+0x24/0x30
   x86_64_start_kernel+0xda/0xe0
   common_startup_64+0x13e/0x141
   

IOW, we shouldn't free memblock allocated memory so early
in the boot process when memory map isn't fully initialized in
deferred_init_memmap().

Fix it by saving the housekeeping cpumask memblock memory to be
freed into a llist free list in housekeeping_init() and add a new
housekeeping_late_init() helper to defer the actual freeing of memblock
memory to when initcall's are being processed. The cpumask memblock
memory is treated as a llist_node with the size of a "long" type which
is also smallest cpumask size that can be allocated.

The non-atomic version of the llist APIs are used as there is no
contention.

This commit depends on the presence of commit 7c2eee9c1367 ("memblock:
don't touch memblock arrays when memblock_free() is called late")
to prevent a KASAN UAF bug report [1].

 [1] https://lore.kernel.org/lkml/20260505051821.1107133-1-longman@redhat.com/

Fixes: 27c3a5967f05 ("sched/isolation: Convert housekeeping cpumasks to rcu pointers")
Signed-off-by: Waiman Long 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Frederic Weisbecker 
Reviewed-by: Phil Auld 
Link: https://patch.msgid.link/20260701195810.477326-1-longman@redhat.com
Signed-off-by: Sasha Levin

sched/fair: Check CPU capacity before comparing group types during load balance

2026-09-14T11:40:29+00:00

[ Upstream commit 50b101f6e586b4417d060a976fd831cd87e86e2b ]

update_sd_pick_busiest() may incorrectly select a fully_busy group as the
busiest group when its per-CPU capacity exceeds that of the destination
CPU. This happens because the type of busiest group is initialized to
group_has_spare and allows the fully_busy group to win the type comparison.

update_sd_pick_busiest() should not choose a candidate scheduling group
with at most one runnable task if its per-CPU capacity is greater than that
of the destination CPU. Such a check already exists, but it is done too
late: after the type comparison, preventing a subsequent fully_busy group
of equal per-CPU capacity from being correctly selected.

Move this check to occur before comparing group types.

Fixes: 0b0695f2b34a ("sched/fair: Rework load_balance()")
Signed-off-by: Ricardo Neri 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Christian Loehle 
Reviewed-by: Chen Yu 
Reviewed-by: Tim Chen 
Reviewed-by: Vincent Guittot 
Tested-by: Christian Loehle 
Tested-by: Andrea Righi 
Link: https://patch.msgid.link/20260720-rneri-fix-cas-clusters-v6-3-bb500bf4afd4@linux.intel.com
Signed-off-by: Sasha Levin

sched/fair: Also gate overloaded status update for SD_ASYM_CPUCAPACITY

2026-09-14T11:40:29+00:00

[ Upstream commit 6060d61d13a10da8c90da4eadf4a421825149883 ]

The argument sg_overloaded of update_sg_lb_stats() is only consumed when
balancing at the root domain. It only makes sense to update it in such a
case. Commit 3229adbe7875 ("sched/fair: Do not compute overloaded status
unnecessarily during lb") updated the logic accordingly but missed the case
in which the root domain has the SD_ASYM_CPUCAPACITY flag. Fix this.

Fixes: 3229adbe7875 ("sched/fair: Do not compute overloaded status unnecessarily during lb")
Reported-by: Chen Yu 
Signed-off-by: Ricardo Neri 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Vincent Guittot 
Tested-by: Christian Loehle 
Tested-by: Andrea Righi 
Link: https://patch.msgid.link/20260720-rneri-fix-cas-clusters-v6-2-bb500bf4afd4@linux.intel.com
Signed-off-by: Sasha Levin

sched_ext: Abort directly from the hardlockup handler

2026-09-14T11:39:50+00:00

[ Upstream commit 3c4b38064937a761ebbf85b1649e812db85eb59e ]

scx_hardlockup() defers the abort to an irq_work because exit claiming used
to take scx_sched_lock and couldn't run from NMI. The deferral is now
unnecessary - claiming is NMI-safe and asserting ->aborting is exactly what
breaks the live-locks that hard-lock CPUs. Call handle_lockup() directly and
drop the irq_work. This also makes the self-detected case recoverable: the
perf watchdog fires on the hard-locked CPU itself, where a queued irq_work
never runs with IRQs off.

Also fix the return value: %true used to be returned whenever sched_ext was
loaded, suppressing the kernel's hardlockup report even when the abort was
refused. Return %true only when this call initiated the abort.

Fixes: bd2d76455b65 ("sched_ext: Defer scx_hardlockup() out of NMI")
Signed-off-by: Tejun Heo 
Reviewed-by: Andrea Righi 
Signed-off-by: Sasha Levin

sched_ext: Fix exit_cpu accuracy for lockup paths

2026-09-14T11:39:50+00:00

[ Upstream commit da428d572e07bb9dd2d076297ef5700f6483aafd ]

handle_lockup() uses raw_smp_processor_id() for exit_cpu, which is wrong
for two paths:

- scx_hardlockup_irq_workfn() has the hung CPU in a local variable but
  irq_work may run elsewhere. Pass the local cpu explicitly.
- scx_rcu_cpu_stall() records the detector CPU rather than the stalled
  one. Pass -1 for now. The next patch fixes this properly.

Signed-off-by: Cheng-Yang Chou 
Reviewed-by: Andrea Righi 
Signed-off-by: Tejun Heo 
Stable-dep-of: 3c4b38064937 ("sched_ext: Abort directly from the hardlockup handler")
Signed-off-by: Sasha Levin

cpufreq: schedutil: Fix self-contradictory comment in sugov_iowait_apply()

2026-09-14T11:39:32+00:00

[ Upstream commit db6a017c91b774c15b1b890db45981eacfff540e ]

The kerneldoc of sugov_iowait_apply() says the IO boost value is increased
in sugov_iowait_apply() and, in the same sentence, that it is decreased by
the same function. That is self-contradictory, and the first part is wrong:
sugov_iowait_apply() only decreases the boost.

The boost is actually increased in sugov_iowait_boost(). Fix the comment to
name sugov_iowait_boost() as the place where the boost is increased, so it
matches the code.

No functional change.

Fixes: fd7d5287fd65 ("cpufreq: schedutil: Cleanup and document iowait boost")
Signed-off-by: Zhongqiu Han 
Reviewed-by: Christian Loehle 
Link: https://patch.msgid.link/20260703092433.4080165-1-zhongqiu.han@oss.qualcomm.com
Signed-off-by: Rafael J. Wysocki 
Signed-off-by: Sasha Levin

sched/fair: Fix overflow in update_tg_cfs_runnable()

2026-09-14T11:39:07+00:00

[ Upstream commit 4f166adb5cb0525d9e32d45729fd8f28c80acbee ]

A divide-by-zero crash is observed when running hackbench:

   [14697.488452] CPU: 112 UID: 0 PID: 124791 Comm: hackbench Not tainted 7.1.0-rc2+
   [14697.492627] RIP: 0010:propagate_entity_load_avg+0x35f/0x3e0
   [14697.506799]  
   [14697.507411]  __dequeue_task+0x2b4/0xc70
   [14697.508677]  dequeue_task_fair+0x36/0x370
   [14697.509047]  dequeue_task+0x101/0x2f0
   [14697.509426]  __schedule+0x1b1/0x1a00
   [14697.510868]  anon_pipe_read+0x3da/0x450
   [14697.511400]  vfs_read+0x361/0x390
   [14697.512053]  __x64_sys_read+0x19/0x30

The divide-by-zero happens here:

  if (scale_load_down(gcfs_rq->load.weight)) {
           load_sum = div_u64(gcfs_rq->avg.load_sum,
                              scale_load_down(gcfs_rq->load.weight));
  }

gcfs_rq->load.weight is an insane large value and is truncated
to the lower 32 bits by div_u64, which happen to be 0.

Using AI for investigation, the cause is a u32 overflow in
update_tg_cfs_runnable(), and flat pickup became a victim when using
tg_tasks():

   u32 new_sum, divider;
   ...
   new_sum = se->avg.runnable_avg * divider; <-- boom

The following sequence shows how this triggers the crash:

   propagate_entity_load_avg()
     update_tg_cfs_runnable()     # u32 overflow corrupts runnable_sum

   __update_load_avg_cfs_rq()
     ___update_load_avg()         # computes insane runnable_avg
   update_tg_load_avg()           # propagates to tg->runnable_avg

   update_cfs_group()
     calc_concur_shares()
       tg_tasks()                 # long-to-int truncation, negative nr
     reweight_entity()            # corrupted se->load.weight
       update_load_add()          # corrupted cfs_rq->load.weight

   propagate_entity_load_avg()
     update_tg_cfs_load()
       div_u64()                  # divide-by-zero

Fix by widening new_sum from u32 to u64 (no need to force tg_tasks()
to return unsigned long after this fix)

Fixes: 95246d1ec80b ("sched/pelt: Relax the sync of runnable_sum with runnable_avg")
Assisted-by: Claude:claude-opus-4.6
Signed-off-by: Chen Yu 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://patch.msgid.link/a22eea2b-4c4a-4623-9a44-d7b18c0c91c8@intel.com
Signed-off-by: Sasha Levin

sched/core: Fix inter-class wakeup_preempt()

2026-09-14T11:39:07+00:00

[ Upstream commit fa02b2868420d9f33d64ddcb15ef0f96880b2a6d ]

The way wakeup_preempt() works since commit 704069649b5b ("sched/core: Rework
sched_class::wakeup_preempt() and rq_modified_*()") is that it will call
rq->next_class->wakeup_preempt(rq, p) when p is of an equal or higher class,
and raise ->next_class when higher.

This means that:

	running idle task

	wakeup fair-A
	  (next_class == idle)
	  if (sched_class_above(fair, idle)) {
	     wakeup_preempt_idle(fair-A);
	     resched_curr(rq);
	     next_class = fair;
	  }

	wakeup fair-B
	  (next_class == fair)
	  if (fair == fair)
	    wakeup_preempt_fair(fair-B);
	      (but current is idle)

All wakeup_preempt_$class() methods, except for wakeup_preempt_scx() (for whoem
this was build) ignore cross-class wakeups by testing if @p is of the right
class, but per the above case, it also should check current.

This is mostly harmless in the current form, but will lead to trouble with
later patches.

Fixes: 704069649b5b ("sched/core: Rework sched_class::wakeup_preempt() and rq_modified_*()")
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://patch.msgid.link/20260626074605.GB2568396%40noisy.programming.kicks-ass.net
Signed-off-by: Sasha Levin

futex: Provide rt_mutex_.*_schedule() equivalents for futex scheduling

2026-09-11T09:50:50+00:00

commit 912edebe8501a36c6bedcef03bd238ab90a7e060 upstream.

There is rt_mutex_{pre|post}_schedule() around
rt_mutex_wait_proxy_lock() to ensure that sched_submit_work()/
sched_update_worker() is invoked before we schedule out and block on
rt_mutex while waiting for it become available.

The reason is that blocking on rt_mutex assigns a pi_waiter for the PI
chain and sched_submit_work() will also assign a pi_waiter if it blocks
on lock but a this point we already have a waiter assigned.
We can't skip sched_submit_work() entirely because I/O relies on the
fact that I/O queue is flushed while it blocks on a sleeping lock.
Therefore sched_submit_work() is moved before we block on the lock.

Sleeping lock in this context means mutex or rw_semaphore not spinlock_t
on PREEMPT_RT. Because the mutex abstraction on PREEMPT_RT uses the same
abstraction as the futex proxy lock, the futex code ended up using
rt_mutex_{pre|post}_schedule(), too.
Using it is/ was just to keep the task_struct::sched_rt_mutex assertion
happy. Futex proxy lock is used only in the syscall context of a task.
At this point it never got any I/O that needs to be flushed and it can't
be a workqueue that needs to notify that it will be scheduled out.
Therefore sched_submit_work() does nothing here.

By mistake futex_wait_requeue_pi() -> rt_mutex_wait_proxy_lock() did not
get the rt_mutex_{pre|post}_schedule() annotation. This was not noticed
because in this callchain the lock is (usually) not contended and so
rt_mutex_slowlock_block() does not schedule, triggering the assert.

Adding rt_mutex_pre_schedule() here looks wrong (as noted by PeterZ)
because at this point there is a pi_waiter recorded and invoking
sched_submit_work() with a possible lock contention would be wrong.

Add rt_mutex_futex_{pre|post}_schedule() which toggles the
sched_rt_mutex assert and does not involve sched_submit_work(). Add
asserts here to ensure that sched_submit_work() would do nothing. Use it
only in futex proxy lock case which is rt_mutex_wait_proxy_lock().
Remove it from futex_lock_pi().

Fixes: d14f9e930b90 ("locking/rtmutex: Use rt_mutex specific scheduler helpers")
Reported-by: Yao Kai 
Signed-off-by: Sebastian Andrzej Siewior 
Signed-off-by: Thomas Gleixner 
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260901135453.3121948-2-bigeasy@linutronix.de
Closes: https://lore.kernel.org/all/20260717084922.4153317-2-yaokai34@huawei.com
Signed-off-by: Greg Kroah-Hartman