linux-stable.git/kernel/sched, branch v7.1.9 Linux kernel stable tree sched_ext: Take cgroup_lock() first in scx_cgroup_lock() 2026-08-19T16:20:31+00:00 Tejun Heo tj@kernel.org 2026-07-16T20:44:02+00:00 a054c9ffa9b7a0dffb763837b91ac9d381ec6d10 commit 5f8b69642d18e1f3e11996707842ac530444e959 upstream. scx_cgroup_lock() write-locks scx_cgroup_ops_rwsem and then takes cgroup_lock(), which can deadlock through kernfs: scx enable/disable cgroup rmdir cpu.weight write ------------------ ------------ ---------------- cgroup_lock() percpu_down_write(rwsem) cgroup_lock() kernfs_get_active() percpu_down_read(rwsem) kernfs_drain() The enable path waits for the rmdir to release cgroup_mutex. The rmdir, deactivating the cpu controller's files, waits in kernfs_drain() for the write's active reference. The write, in scx_group_set_weight(), waits for the rwsem behind the pending writer. Take cgroup_lock() first. The set_* paths take no cgroup locks inside the read side, so a pending write-lock then only waits for read sections that always run to completion, and no dependency from the rwsem back to cgroup_mutex remains. Fixes: a5bd6ba30b33 ("sched_ext: Use cgroup_lock/unlock() to synchronize against cgroup operations") Cc: stable@vger.kernel.org # v6.18+ Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit 5f8b69642d18e1f3e11996707842ac530444e959 upstream.

scx_cgroup_lock() write-locks scx_cgroup_ops_rwsem and then takes
cgroup_lock(), which can deadlock through kernfs:

  scx enable/disable         cgroup rmdir           cpu.weight write
  ------------------         ------------           ----------------
                             cgroup_lock()
  percpu_down_write(rwsem)
  cgroup_lock()
                                                    kernfs_get_active()
                                                    percpu_down_read(rwsem)
                             kernfs_drain()

The enable path waits for the rmdir to release cgroup_mutex. The rmdir,
deactivating the cpu controller's files, waits in kernfs_drain() for the
write's active reference. The write, in scx_group_set_weight(), waits for
the rwsem behind the pending writer.

Take cgroup_lock() first. The set_* paths take no cgroup locks inside the
read side, so a pending write-lock then only waits for read sections that
always run to completion, and no dependency from the rwsem back to
cgroup_mutex remains.

Fixes: a5bd6ba30b33 ("sched_ext: Use cgroup_lock/unlock() to synchronize against cgroup operations")
Cc: stable@vger.kernel.org # v6.18+
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
sched/psi: Create the psimon kthread outside of cgroup_mutex 2026-08-19T16:20:31+00:00 Tejun Heo tj@kernel.org 2026-07-12T17:23:55+00:00 d217d851f0a26d42fbc19e44c789e40f25db3498 commit fadeedd7cfc5d73d33fa3d7ac54b9b27aabd09d2 upstream. a5b98009f16d ("sched/psi: fix race between file release and pressure write") made pressure_write() hold cgroup_mutex across psi_trigger_create(), which forks the psimon kthread for the first rtpoll trigger. As kthread creation depends on the whole fork path, the commit inadvertently created a lot of unwanted locking dependencies from cgroup_mutex. sched_ext got hit by one: its enable path blocks forks and then grabs cgroup_mutex, so a pressure write racing a scheduler enable deadlocks, with every other fork piling up behind. Fix it by splitting trigger creation so that the worker is forked with cgroup_mutex dropped and the kernfs active reference left broken. The latter matters because rmdir and cgroup.pressure writes drain active references under cgroup_mutex. Publishing the trigger last keeps error reporting synchronous and preserves the of->priv lifetime rules. The trigger registered in the first stage pins the group's rtpoll machinery across the unlocked window, leaving only creation races to resolve. The catch-up poll on installation covers scheduling attempts dropped while there was no worker. v2: Retagged sched/psi (was cgroup). Fixes: a5b98009f16d ("sched/psi: fix race between file release and pressure write") Cc: stable@vger.kernel.org Cc: Edward Adam Davis <eadavis@qq.com> Cc: Chen Ridong <chenridong@huaweicloud.com> Reported-by: Matt Fleming <mfleming@cloudflare.com> Closes: https://lore.kernel.org/all/20260710100441.2653477-1-matt@readmodwrite.com/ Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Tested-by: Matt Fleming <mfleming@cloudflare.com> Acked-by: Suren Baghdasaryan <surenb@google.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit fadeedd7cfc5d73d33fa3d7ac54b9b27aabd09d2 upstream.

a5b98009f16d ("sched/psi: fix race between file release and pressure write")
made pressure_write() hold cgroup_mutex across psi_trigger_create(), which
forks the psimon kthread for the first rtpoll trigger. As kthread creation
depends on the whole fork path, the commit inadvertently created a lot of
unwanted locking dependencies from cgroup_mutex.

sched_ext got hit by one: its enable path blocks forks and then grabs
cgroup_mutex, so a pressure write racing a scheduler enable deadlocks, with
every other fork piling up behind.

Fix it by splitting trigger creation so that the worker is forked with
cgroup_mutex dropped and the kernfs active reference left broken. The latter
matters because rmdir and cgroup.pressure writes drain active references
under cgroup_mutex. Publishing the trigger last keeps error reporting
synchronous and preserves the of->priv lifetime rules.

The trigger registered in the first stage pins the group's rtpoll machinery
across the unlocked window, leaving only creation races to resolve. The
catch-up poll on installation covers scheduling attempts dropped while there
was no worker.

v2: Retagged sched/psi (was cgroup).

Fixes: a5b98009f16d ("sched/psi: fix race between file release and pressure write")
Cc: stable@vger.kernel.org
Cc: Edward Adam Davis <eadavis@qq.com>
Cc: Chen Ridong <chenridong@huaweicloud.com>
Reported-by: Matt Fleming <mfleming@cloudflare.com>
Closes: https://lore.kernel.org/all/20260710100441.2653477-1-matt@readmodwrite.com/
Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Tested-by: Matt Fleming <mfleming@cloudflare.com>
Acked-by: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
sched/psi: Shut down rtpoll_timer in psi_cgroup_free() 2026-08-19T16:20:31+00:00 Tejun Heo tj@kernel.org 2026-07-12T17:23:55+00:00 611e7821c4f83a671455658797336faecc3a5196 commit 5457025fa8ca3c0d2732109513de839e3e797190 upstream. psi_schedule_rtpoll_work() is called locklessly from the scheduler hotpath and can race psi_trigger_destroy() taking down the last rtpoll trigger under rtpoll_trigger_lock: psi_schedule_rtpoll_work() psi_trigger_destroy() rcu_read_lock(); task = rcu_dereference(rtpoll_task); rcu_assign_pointer(rtpoll_task, NULL); timer_delete(&rtpoll_timer); mod_timer(&rtpoll_timer, ...); rcu_read_unlock(); synchronize_rcu(); kthread_stop(task_to_destroy); The group can then be freed with the re-armed timer still pending, and poll_timer_fn() runs on freed memory. 461daba06bdc ("psi: eliminate kthread_worker from psi trigger scheduling mechanism") deleted the timer synchronously after the synchronize_rcu(), which prevented this but raced trigger creation instead: the deletion could cancel the timer that a new trigger set armed during the grace period and, as creation also reinitialized the timer at the time, corrupt it. 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy") moved the initialization into group_init() and the deletion into the locked section, trading the creation races for the window above. Neither placement in the destruction path works. A pending timer firing while the group is alive is harmless though. poll_timer_fn() just wakes the rtpoll waitqueue and doesn't re-arm itself. Bind the timer to the group's lifetime instead and shut it down in psi_cgroup_free(). Nothing can arm it by then. timer_shutdown_sync() because the timer is never armed again. Fixes: 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy") Cc: stable@vger.kernel.org # v5.10+ Reported-by: Sashiko AI <sashiko-bot@kernel.org> Closes: https://lore.kernel.org/all/20260711000434.36C4A1F000E9@smtp.kernel.org/ Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Tested-by: Matt Fleming <mfleming@cloudflare.com> Acked-by: Suren Baghdasaryan <surenb@google.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit 5457025fa8ca3c0d2732109513de839e3e797190 upstream.

psi_schedule_rtpoll_work() is called locklessly from the scheduler hotpath
and can race psi_trigger_destroy() taking down the last rtpoll trigger under
rtpoll_trigger_lock:

  psi_schedule_rtpoll_work()        psi_trigger_destroy()

  rcu_read_lock();
  task = rcu_dereference(rtpoll_task);
                                    rcu_assign_pointer(rtpoll_task, NULL);
                                    timer_delete(&rtpoll_timer);
  mod_timer(&rtpoll_timer, ...);
  rcu_read_unlock();
                                    synchronize_rcu();
                                    kthread_stop(task_to_destroy);

The group can then be freed with the re-armed timer still pending, and
poll_timer_fn() runs on freed memory.

461daba06bdc ("psi: eliminate kthread_worker from psi trigger scheduling
mechanism") deleted the timer synchronously after the synchronize_rcu(),
which prevented this but raced trigger creation instead: the deletion could
cancel the timer that a new trigger set armed during the grace period and,
as creation also reinitialized the timer at the time, corrupt it.
8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy") moved the
initialization into group_init() and the deletion into the locked section,
trading the creation races for the window above.

Neither placement in the destruction path works. A pending timer firing
while the group is alive is harmless though. poll_timer_fn() just wakes the
rtpoll waitqueue and doesn't re-arm itself. Bind the timer to the group's
lifetime instead and shut it down in psi_cgroup_free(). Nothing can arm it
by then. timer_shutdown_sync() because the timer is never armed again.

Fixes: 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy")
Cc: stable@vger.kernel.org # v5.10+
Reported-by: Sashiko AI <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260711000434.36C4A1F000E9@smtp.kernel.org/
Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Tested-by: Matt Fleming <mfleming@cloudflare.com>
Acked-by: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
sched_ext: Don't enable non-ext tasks in the sub-sched task loops 2026-08-19T16:20:13+00:00 Tejun Heo tj@kernel.org 2026-07-16T20:46:04+00:00 517b1e4fe28bcbab2113c00bbfa5475dda86fc8a [ Upstream commit 5cdc928598095b2c7d5f265e5f21eadd1634bfbe ] Root enable and scx_post_fork() enable a task only if it's on the ext class. Tasks on other classes, possible under an SCX_OPS_SWITCH_PARTIAL root, are left READY and enabled by switching_to_scx() when they switch over. The sub enable-commit pass and the sub-disable re-home loop enable unconditionally, so a fair-class READY task in the subtree becomes ENABLED while not on sched_ext. A later switch to SCHED_EXT then trips the task state validation WARN (ENABLED with the previous state not READY) and calls ops.enable() a second time. Gate scx_enable_task() on the task's class in both loops. Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling") Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 5cdc928598095b2c7d5f265e5f21eadd1634bfbe ]

Root enable and scx_post_fork() enable a task only if it's on the ext class.
Tasks on other classes, possible under an SCX_OPS_SWITCH_PARTIAL root, are
left READY and enabled by switching_to_scx() when they switch over. The sub
enable-commit pass and the sub-disable re-home loop enable unconditionally,
so a fair-class READY task in the subtree becomes ENABLED while not on
sched_ext. A later switch to SCHED_EXT then trips the task state validation
WARN (ENABLED with the previous state not READY) and calls ops.enable() a
second time.

Gate scx_enable_task() on the task's class in both loops.

Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
sched_ext: Skip sub-disable teardown for never-linked sub-schedulers 2026-08-19T16:20:13+00:00 Tejun Heo tj@kernel.org 2026-07-16T20:45:23+00:00 6428093a4a986c38c9089b5eb32b56d914ef437a [ Upstream commit 8c13364db9c9a43ed286f3a8d0fb9477b1adc43c ] A sub-scheduler enable can fail before scx_link_sched() links the sched into the hierarchy, e.g. when the parent is already being disabled, and cleanup still runs the full scx_sub_disable(). That is racy against root disable: drain_descendants() is the only ordering between a sub's disable-time task walk and root disable's all-task teardown, and an unlinked sub is invisible to it. Root's teardown can thus run between the never-linked sub's drain and its walk, exiting every task to no scheduler. The walk then trips the membership WARN and re-homes the exited tasks onto the dying hierarchy, a use-after-free. Skip the cgroup ownership reset and the task walk if @sch was never linked, indicated by the empty ->sibling as unlinking only happens later in the same function. The membership WARN remains valid: a linked sub is always waited on by an ancestor's drain. Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling") Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 8c13364db9c9a43ed286f3a8d0fb9477b1adc43c ]

A sub-scheduler enable can fail before scx_link_sched() links the sched into
the hierarchy, e.g. when the parent is already being disabled, and cleanup
still runs the full scx_sub_disable().

That is racy against root disable: drain_descendants() is the only ordering
between a sub's disable-time task walk and root disable's all-task teardown,
and an unlinked sub is invisible to it. Root's teardown can thus run between
the never-linked sub's drain and its walk, exiting every task to no
scheduler.

The walk then trips the membership WARN and re-homes the exited tasks onto
the dying hierarchy, a use-after-free.

Skip the cgroup ownership reset and the task walk if @sch was never linked,
indicated by the empty ->sibling as unlinking only happens later in the same
function. The membership WARN remains valid: a linked sub is always waited
on by an ancestor's drain.

Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
sched_ext: Reject setting disallow from init_task outside the enable path 2026-08-19T16:20:13+00:00 Tejun Heo tj@kernel.org 2026-07-16T20:43:25+00:00 592bc5000d6b5f083c79f74e869e87011c3eb458 [ Upstream commit 477869bfafea65492d23de62c1b5208147c09dd2 ] The p->scx.disallow revert assumes the root enable path, where the switching loop reads the reverted policy right afterwards and leaves the task off SCX. The sub-scheduler disable path also reaches it when re-initializing the returned tasks on a root parent. Nothing reads the policy there: the task is enabled on root anyway and keeps running on the ext class with a silently rewritten policy. Kill the sched instead, matching the fork and non-root branches, and update the disallow documentation, which equated !fork with the load path and pointed at a stale debugfs path for nr_rejected. Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling") Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 477869bfafea65492d23de62c1b5208147c09dd2 ]

The p->scx.disallow revert assumes the root enable path, where the switching
loop reads the reverted policy right afterwards and leaves the task off SCX.
The sub-scheduler disable path also reaches it when re-initializing the
returned tasks on a root parent. Nothing reads the policy there: the task is
enabled on root anyway and keeps running on the ext class with a silently
rewritten policy.

Kill the sched instead, matching the fork and non-root branches, and update
the disallow documentation, which equated !fork with the load path and
pointed at a stale debugfs path for nr_rejected.

Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
cpufreq: schedutil: Publish util hooks only after all sg_cpu are initialized 2026-08-09T18:26:49+00:00 Zhongqiu Han zhongqiu.han@oss.qualcomm.com 2026-07-16T11:51:58+00:00 b6814d55ccd4254c0f28850874776ef3d5fd8346 commit f0a3f042293a8c5a2152346b3637ea60866c503a upstream. Commit 16a03c71bba0 ("cpufreq: schedutil: Merge initialization code of sg_cpu in single loop") merged the per-CPU initialization and the utilization-hook registration into a single loop in sugov_start(). For a shared cpufreq policy this re-introduces the race originally fixed by commit ab2f7cf141aa ("cpufreq: schedutil: Fix sugov_start() versus sugov_update_shared() race"). The scheduler's util path reaches the hook under RCU-sched and never takes policy->rwsem, so the rwsem held across sugov_start() cannot serialize the two. Once the first CPU's hook is published, sugov_update_shared() may run and, via sugov_next_freq_shared(), read/write each sibling sugov_cpu (iowait_boost, util, bw_min, ...) concurrently with the memset() still initializing them, with no lock common to both sides: the update side holds sg_policy->update_lock while the init side holds only policy->rwsem, which the scheduler's util path never takes. The walk only accesses scalar members, never a pointer like ->sg_policy, so it does not crash today; it merely uses stale (or zero on first start) values that skew the frequency selection and tracepoints. It is still a genuine data race, and a latent crash once any pointer member is dereferenced there. Restore the two-phase approach: initialize all per-CPU structures first, and only then publish the per-CPU utilization update hooks. Fixes: 16a03c71bba0 ("cpufreq: schedutil: Merge initialization code of sg_cpu in single loop") Cc: stable@vger.kernel.org Signed-off-by: Zhongqiu Han <zhongqiu.han@oss.qualcomm.com> Reviewed-by: Christian Loehle <christian.loehle@arm.com> Link: https://patch.msgid.link/20260716115159.848403-1-zhongqiu.han@oss.qualcomm.com Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit f0a3f042293a8c5a2152346b3637ea60866c503a upstream.

Commit 16a03c71bba0 ("cpufreq: schedutil: Merge initialization code of
sg_cpu in single loop") merged the per-CPU initialization and the
utilization-hook registration into a single loop in sugov_start().

For a shared cpufreq policy this re-introduces the race originally fixed
by commit ab2f7cf141aa ("cpufreq: schedutil: Fix sugov_start() versus
sugov_update_shared() race").

The scheduler's util path reaches the hook under RCU-sched and never takes
policy->rwsem, so the rwsem held across sugov_start() cannot serialize the
two. Once the first CPU's hook is published, sugov_update_shared() may run
and, via sugov_next_freq_shared(), read/write each sibling sugov_cpu
(iowait_boost, util, bw_min, ...) concurrently with the memset() still
initializing them, with no lock common to both sides: the update side holds
sg_policy->update_lock while the init side holds only policy->rwsem, which
the scheduler's util path never takes.

The walk only accesses scalar members, never a pointer like ->sg_policy,
so it does not crash today; it merely uses stale (or zero on first start)
values that skew the frequency selection and tracepoints. It is still a
genuine data race, and a latent crash once any pointer member is
dereferenced there.

Restore the two-phase approach: initialize all per-CPU structures first,
and only then publish the per-CPU utilization update hooks.

Fixes: 16a03c71bba0 ("cpufreq: schedutil: Merge initialization code of sg_cpu in single loop")
Cc: stable@vger.kernel.org
Signed-off-by: Zhongqiu Han <zhongqiu.han@oss.qualcomm.com>
Reviewed-by: Christian Loehle <christian.loehle@arm.com>
Link: https://patch.msgid.link/20260716115159.848403-1-zhongqiu.han@oss.qualcomm.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
sched/deadline: Use revised wakeup rule only for running dl_server 2026-08-09T18:26:37+00:00 Gabriele Monaco gmonaco@redhat.com 2026-05-22T12:58:33+00:00 3ab00c9fd4208be6af5b207512491f63674ae9e4 [ Upstream commit 1842bf97af109f5ebf830175c9725bf81ebb78b1 ] Commit 14a857056466 ("sched/deadline: Use revised wakeup rule for dl_server") applies the revised wakeup rule to any server, as a result servers that are not running (dl_defer_running == 0) and start with a deadline overflow get enqueued and can boost tasks as if they were running, invalidating the defer rule and the documented state model. Apply the revised wakeup rule only for deferrable servers that are marked as running. Fixes: 14a857056466 ("sched/deadline: Use revised wakeup rule for dl_server") Signed-off-by: Gabriele Monaco <gmonaco@redhat.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Juri Lelli <juri.lelli@redhat.com> Tested-by: Andrea Righi <arighi@nvidia.com> Link: https://patch.msgid.link/20260522125833.264145-1-gmonaco@redhat.com Signed-off-by: Sasha Levin <sashal@kernel.org> 
[ Upstream commit 1842bf97af109f5ebf830175c9725bf81ebb78b1 ]

Commit 14a857056466 ("sched/deadline: Use revised wakeup rule for
dl_server") applies the revised wakeup rule to any server, as a result
servers that are not running (dl_defer_running == 0) and start with a
deadline overflow get enqueued and can boost tasks as if they were
running, invalidating the defer rule and the documented state model.

Apply the revised wakeup rule only for deferrable servers that are
marked as running.

Fixes: 14a857056466 ("sched/deadline: Use revised wakeup rule for dl_server")
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Juri Lelli <juri.lelli@redhat.com>
Tested-by: Andrea Righi <arighi@nvidia.com>
Link: https://patch.msgid.link/20260522125833.264145-1-gmonaco@redhat.com
Signed-off-by: Sasha Levin <sashal@kernel.org>
sched_ext: Preserve rq tracking across local DSQ dispatch 2026-08-03T09:26:04+00:00 Andrea Righi arighi@nvidia.com 2026-07-29T16:24:33+00:00 97c09c9f5739b8757ee29dabb0af30069137e286 [ Upstream commit 18d62044cda7a2b40f59d910659c0b0d6accad37 ] dispatch_to_local_dsq() can run from scx_bpf_dsq_move_to_local() while ops.dispatch() has recorded the current rq. Moving a task to a local DSQ may switch to the source or destination rq before synchronously invoking ops.dequeue() through the following path: SCX_CALL_OP(dispatch, rq) ops.dispatch() scx_bpf_dsq_move_to_local() scx_flush_dispatch_buf() finish_dispatch() dispatch_to_local_dsq() scx_dispatch_enqueue() local_dsq_post_enq() call_task_dequeue() SCX_CALL_OP_TASK(dequeue, locked_rq, ...) The nested callback saves the recorded rq and restores it on return. If the rq tracking does not follow the lock switch, update_locked_rq() can trigger the following lockdep assertion while restoring an rq which is no longer held: WARNING: kernel/sched/sched.h:1641 at call_task_dequeue+0x160/0x170 Call Trace: scx_dispatch_enqueue+0x2b0/0x460 dispatch_to_local_dsq+0x138/0x230 scx_flush_dispatch_buf+0x1af/0x220 scx_bpf_dsq_move_to_local___v2+0xe2/0x1c0 bpf__sched_ext_ops_dispatch+0x4b/0xa7 do_pick_task_scx+0x3b6/0x910 __pick_next_task+0x105/0x1f0 __schedule+0x3e7/0x1980 Introduce switch_rq_lock() to update the tracking state together with each rq lock handoff. Use it in dispatch_to_local_dsq(), move_remote_task_to_local_dsq() and the in-balance paths of scx_dsq_move(), ensuring that scx_locked_rq() consistently refers to the rq whose lock is actually held throughout the lock dance. Fixes: 7fb39e4eb4c3 ("sched_ext: Save and restore scx_locked_rq across SCX_CALL_OP") Cc: stable@vger.kernel.org # 7.1+ Signed-off-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Tejun Heo <tj@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
[ Upstream commit 18d62044cda7a2b40f59d910659c0b0d6accad37 ]

dispatch_to_local_dsq() can run from scx_bpf_dsq_move_to_local() while
ops.dispatch() has recorded the current rq. Moving a task to a local DSQ
may switch to the source or destination rq before synchronously invoking
ops.dequeue() through the following path:

  SCX_CALL_OP(dispatch, rq)
    ops.dispatch()
      scx_bpf_dsq_move_to_local()
        scx_flush_dispatch_buf()
          finish_dispatch()
            dispatch_to_local_dsq()
              scx_dispatch_enqueue()
                local_dsq_post_enq()
                  call_task_dequeue()
                    SCX_CALL_OP_TASK(dequeue, locked_rq, ...)

The nested callback saves the recorded rq and restores it on return. If
the rq tracking does not follow the lock switch, update_locked_rq() can
trigger the following lockdep assertion while restoring an rq which is
no longer held:

  WARNING: kernel/sched/sched.h:1641 at call_task_dequeue+0x160/0x170
  Call Trace:
    scx_dispatch_enqueue+0x2b0/0x460
    dispatch_to_local_dsq+0x138/0x230
    scx_flush_dispatch_buf+0x1af/0x220
    scx_bpf_dsq_move_to_local___v2+0xe2/0x1c0
    bpf__sched_ext_ops_dispatch+0x4b/0xa7
    do_pick_task_scx+0x3b6/0x910
    __pick_next_task+0x105/0x1f0
    __schedule+0x3e7/0x1980

Introduce switch_rq_lock() to update the tracking state together with
each rq lock handoff. Use it in dispatch_to_local_dsq(),
move_remote_task_to_local_dsq() and the in-balance paths of
scx_dsq_move(), ensuring that scx_locked_rq() consistently refers to the
rq whose lock is actually held throughout the lock dance.

Fixes: 7fb39e4eb4c3 ("sched_ext: Save and restore scx_locked_rq across SCX_CALL_OP")
Cc: stable@vger.kernel.org # 7.1+
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
sched_ext: Move shared helpers from ext.c into internal.h and cid.h 2026-08-03T09:26:04+00:00 Tejun Heo tj@kernel.org 2026-07-29T16:24:32+00:00 046899bcfc2dfa368e12a40655aa85621046b95f [ Upstream commit 4437ad129cf5b37c00a5bc9fa5989d1da4d64d07 ] idle.c and cid.c are included into build_policy.c together with ext.c and use helpers that ext.c defines. Because the helpers live in ext.c, the two files can not parse as standalone units and clangd reports errors in them. Move the helpers to the headers they belong to. The op-dispatch macros and helpers plus scx_parent() to internal.h, and scx_cpu_arg()/scx_cpu_ret() to cid.h. No functional change. idle.c and cid.c now parse clean standalone. Suggested-by: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com> Stable-dep-of: 18d62044cda7 ("sched_ext: Preserve rq tracking across local DSQ dispatch") Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
[ Upstream commit 4437ad129cf5b37c00a5bc9fa5989d1da4d64d07 ]

idle.c and cid.c are included into build_policy.c together with ext.c and
use helpers that ext.c defines. Because the helpers live in ext.c, the two
files can not parse as standalone units and clangd reports errors in them.

Move the helpers to the headers they belong to. The op-dispatch macros and
helpers plus scx_parent() to internal.h, and scx_cpu_arg()/scx_cpu_ret() to
cid.h. No functional change. idle.c and cid.c now parse clean standalone.

Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Stable-dep-of: 18d62044cda7 ("sched_ext: Preserve rq tracking across local DSQ dispatch")
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>