linux-stable.git/kernel/sched, branch linux-rolling-stable

sched_ext: Preserve rq tracking across local DSQ dispatch

2026-08-03T09:26:04+00:00

[ Upstream commit 18d62044cda7a2b40f59d910659c0b0d6accad37 ]

dispatch_to_local_dsq() can run from scx_bpf_dsq_move_to_local() while
ops.dispatch() has recorded the current rq. Moving a task to a local DSQ
may switch to the source or destination rq before synchronously invoking
ops.dequeue() through the following path:

  SCX_CALL_OP(dispatch, rq)
    ops.dispatch()
      scx_bpf_dsq_move_to_local()
        scx_flush_dispatch_buf()
          finish_dispatch()
            dispatch_to_local_dsq()
              scx_dispatch_enqueue()
                local_dsq_post_enq()
                  call_task_dequeue()
                    SCX_CALL_OP_TASK(dequeue, locked_rq, ...)

The nested callback saves the recorded rq and restores it on return. If
the rq tracking does not follow the lock switch, update_locked_rq() can
trigger the following lockdep assertion while restoring an rq which is
no longer held:

  WARNING: kernel/sched/sched.h:1641 at call_task_dequeue+0x160/0x170
  Call Trace:
    scx_dispatch_enqueue+0x2b0/0x460
    dispatch_to_local_dsq+0x138/0x230
    scx_flush_dispatch_buf+0x1af/0x220
    scx_bpf_dsq_move_to_local___v2+0xe2/0x1c0
    bpf__sched_ext_ops_dispatch+0x4b/0xa7
    do_pick_task_scx+0x3b6/0x910
    __pick_next_task+0x105/0x1f0
    __schedule+0x3e7/0x1980

Introduce switch_rq_lock() to update the tracking state together with
each rq lock handoff. Use it in dispatch_to_local_dsq(),
move_remote_task_to_local_dsq() and the in-balance paths of
scx_dsq_move(), ensuring that scx_locked_rq() consistently refers to the
rq whose lock is actually held throughout the lock dance.

Fixes: 7fb39e4eb4c3 ("sched_ext: Save and restore scx_locked_rq across SCX_CALL_OP")
Cc: stable@vger.kernel.org # 7.1+
Signed-off-by: Andrea Righi 
Signed-off-by: Tejun Heo 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

sched_ext: Move shared helpers from ext.c into internal.h and cid.h

2026-08-03T09:26:04+00:00

[ Upstream commit 4437ad129cf5b37c00a5bc9fa5989d1da4d64d07 ]

idle.c and cid.c are included into build_policy.c together with ext.c and
use helpers that ext.c defines. Because the helpers live in ext.c, the two
files can not parse as standalone units and clangd reports errors in them.

Move the helpers to the headers they belong to. The op-dispatch macros and
helpers plus scx_parent() to internal.h, and scx_cpu_arg()/scx_cpu_ret() to
cid.h. No functional change. idle.c and cid.c now parse clean standalone.

Suggested-by: Peter Zijlstra 
Signed-off-by: Tejun Heo 
Reviewed-by: Andrea Righi 
Stable-dep-of: 18d62044cda7 ("sched_ext: Preserve rq tracking across local DSQ dispatch")
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

sched_ext: Record an error on errno-only sub-enable failure

2026-08-03T09:25:20+00:00

[ Upstream commit db4e9defd2e8620abee04cfe5809c0bcd6ecf06a ]

scx_sub_enable_workfn() has several failure paths that only return an errno
(e.g. -ENOMEM from an allocation) and jump to err_disable without calling
scx_error(). scx_flush_disable_work() runs the disable, and thus ops.exit(),
only when an error has been recorded, so an errno-only failure leaves the
half-initialized sub-scheduler linked.

Record an error at the err_disable sink so every errno-only failure runs the
disable path.

Fixes: ebeca1f930ea ("sched_ext: Introduce cgroup sub-sched support")
Signed-off-by: Tejun Heo 
Reviewed-by: Andrea Righi 
Signed-off-by: Sasha Levin

sched_ext: Enable tick for finite slices on nohz_full

2026-08-03T09:25:20+00:00

[ Upstream commit 4ec10f38ff901dc10503d57cbdcf941248419ac1 ]

set_next_task_scx() updates the tick dependency before __schedule()
updates rq->curr. When switching from a non-EXT task, such as idle, to
an EXT task with a finite slice, sched_update_tick_dependency() checks
the outgoing task and can allow the tick to remain stopped.

The dependency can also be lost without a slice-type transition. After a
finite-slice task leaves the CPU idle, the enqueue path can clear the
dependency against the idle rq->curr. SCX_RQ_CAN_STOP_TICK still records
a finite slice, so another finite task skips the transition block and
can run without the ticks needed to expire its slice.

The reverse mismatch can also happen when the last finite-slice EXT task
is dequeued: sub_nr_running() updates the dependency before rq->curr
changes, so the outgoing task state can keep the dependency set after
the CPU goes idle.

Fix this by unconditionally enabling the scheduler tick whenever a
finite-slice EXT task is selected on a nohz_full CPU. Moreover, when the
last runnable EXT task leaves, ignore the outgoing EXT slice state so
the generic scheduler can correctly re-evaluate and clear the tick
dependency.

Fixes: 22a920209ab6 ("sched_ext: Implement tickless support")
Signed-off-by: Andrea Righi 
Signed-off-by: Tejun Heo 
Signed-off-by: Sasha Levin

sched_ext: Don't warn on core-sched forced idle in put_prev_task_scx()

2026-08-03T09:25:17+00:00

[ Upstream commit b7d9c359e5cf867f7eb23df3bb1c6b9e58af24da ]

put_prev_task_scx() warns when a runnable task drops to a lower sched_class
without SCX_OPS_ENQ_LAST, on the assumption that balance_one() would have
kept it running. Core scheduling breaks that: a forced-idle SMT sibling
reschedules through the core_pick fast path in pick_next_task(), which skips
pick_task_scx() and thus balance_one(), so a runnable task can drop to idle
with ENQ_LAST unset.

Gate the warning on sched_cpu_cookie_match(): a cookie mismatch means core
scheduling forced the idle, while a match (or core scheduling off) still
catches a genuine missing-ENQ_LAST drop.

Fixes: 7c65ae81ea86 ("sched_ext: Don't call put_prev_task_scx() before picking the next task")
Signed-off-by: Tejun Heo 
Reviewed-by: Andrea Righi 
Signed-off-by: Sasha Levin

sched_ext: Annotate ksyncs with __rcu in alloc/free_kick_syncs()

2026-08-03T09:25:16+00:00

[ Upstream commit 115d1ce989747045bd7745c7ab020982660c7e42 ]

scx_kick_syncs is a per-CPU __rcu pointer, so per_cpu_ptr() returns struct
scx_kick_syncs __rcu **. alloc_kick_syncs() and free_kick_syncs() stored it
in a plain struct scx_kick_syncs **ksyncs, which sparse flags as an __rcu
address-space mismatch. Annotate ksyncs to match. Its accesses already go
through rcu_*_pointer().

Fixes: 987e00035c0e ("sched_ext: Rename pnt_seq to kick_sync")
Reported-by: kernel test robot 
Closes: https://lore.kernel.org/oe-kbuild-all/202606122315.pbnDHP0n-lkp@intel.com/
Signed-off-by: Tejun Heo 
Signed-off-by: Sasha Levin

sched_ext: Skip ops.set_weight() for disabled tasks

2026-08-03T09:25:15+00:00

commit 0e2f4ab68a89fad42e0f5a9ff4b740738e7aa1d6 upstream.

When switching a task's sched_class away from sched_ext, we get the
following sequence of events in __sched_setscheduler():

sched_change_begin()
  switched_from_scx()
    scx_disable_task(p)
      ops.disable(p)
__setscheduler_params()
  set_load_weight()
    reweight_task_scx(p)
      ops.set_weight(p)
p->sched_class = next_class;
sched_change_end()
  ...

Notably, ops.set_weight() is called _after_ ops.disable().
This violates the expected semantics of the callbacks, the expectation
being that ops.disable() can only be followed by ops.exit_task() or
ops.enable().

Skipping the weight adjustment for disabled tasks should be harmless
since the weight will be recalculated in scx_enable_task() if the task
ever rejoins SCX.

Fixes: 637b0682821b ("sched: Fold sched_class::switch{ing,ed}_{to,from}() into the change pattern")
Cc: stable@vger.kernel.org # v6.19+
Signed-off-by: Kuba Piecuch 
Signed-off-by: Tejun Heo 
Signed-off-by: Sasha Levin

cpufreq: schedutil: Fix uncleared need_freq_update on the .adjust_perf() path

2026-07-24T14:20:55+00:00

commit 9ef450ca74e43dacf9a2a15db7a851052c78dcf0 upstream.

The need_freq_update flag makes sugov_should_update_freq() return true
regardless of the rate_limit_us throttling, and is cleared in
sugov_update_next_freq(). sugov_update_single_freq() and
sugov_update_shared() go through that helper, so the flag does not
persist there.

However, sugov_update_single_perf(), used by drivers implementing the
.adjust_perf() callback (e.g. intel_pstate or amd-pstate in passive
mode) calls cpufreq_driver_adjust_perf() directly and never goes through
sugov_update_next_freq(), so the need_freq_update flag is not cleared in
that path.

Before commit 75da043d8f88 ("cpufreq/sched: Set need_freq_update in
ignore_dl_rate_limit()"), this was effectively harmless because
sugov_should_update_freq() still honored the rate limit even when
need_freq_update was set. After that change, the flag forces
sugov_should_update_freq() to always return true, so once set, it
stays effective indefinitely on the .adjust_perf() path.

As a result, cpufreq_driver_adjust_perf() gets called on every scheduler
utilization update (with the runqueue lock held) rather than being
throttled by rate_limit_us, even if the driver itself may skip redundant
hardware updates.

Clear need_freq_update at the end of the adjust_perf path as well.

Fixes: 75da043d8f88 ("cpufreq/sched: Set need_freq_update in ignore_dl_rate_limit()")
Signed-off-by: Zhongqiu Han 
Reviewed-by: Hongyan Xia 
Reviewed-by: Christian Loehle 
Cc: All applicable 
[ rjw: Subject and changelog edits ]
Link: https://patch.msgid.link/20260616154733.2405236-1-zhongqiu.han@oss.qualcomm.com
Signed-off-by: Rafael J. Wysocki 
Signed-off-by: Greg Kroah-Hartman

sched/fair: Fix cpu_util runnable_avg arithmetic

2026-07-24T14:19:39+00:00

[ Upstream commit 29922fdfc2a4008d66418bedd0ebf5038fc54efa ]

If we take runnable_avg in max(runnable_avg, util_avg) in cpu_util(), we
should then add or subtract task runnable_avg, but the arithmetic below
is still with task util_avg. This mixes runnable_avg with util_avg which
is incorrect.

Fix by always doing arithmetic with runnable_avg and only take
max(runnable_avg, util_avg) at the last step.

Fixes: 7d0583cf9ec7 ("sched/fair, cpufreq: Introduce 'runnable boosting'")
Signed-off-by: Hongyan Xia 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Vincent Guittot 
Link: https://patch.msgid.link/20260605094318.37931-1-hongyan.xia@transsion.com
Signed-off-by: Sasha Levin

sched: restore timer_slack_ns when resetting RT policy on fork

2026-07-24T14:19:29+00:00

[ Upstream commit 63c1a12bc0e09af7dee919c4fb4a300a719d5125 ]

Commit ed4fb6d7ef68 ("hrtimer: Use and report correct timerslack values
for realtime tasks") sets timer_slack_ns to 0 for RT tasks in
__setscheduler_params(). However, when an RT task with SCHED_RESET_ON_FORK
creates child threads, the children inherit timer_slack_ns=0 from the
parent. sched_fork() resets the child's policy to SCHED_NORMAL but does
not restore timer_slack_ns, leaving the child permanently running with
zero slack.

Fix this by restoring timer_slack_ns from default_timer_slack_ns in
sched_fork() when resetting from RT/DL to NORMAL policy, matching the
existing behavior in __setscheduler_params().

Note: this fix alone requires a correct default_timer_slack_ns to be
effective. See the following patch for that fix.

Fixes: ed4fb6d7ef68 ("hrtimer: Use and report correct timerslack values for realtime tasks")
Reported-by: Qiaoting.Lin 
Signed-off-by: Guanyou.Chen 
Signed-off-by: Chunhui.Li 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://patch.msgid.link/20260522131000.1664983-2-chenguanyou@xiaomi.com
Signed-off-by: Sasha Levin