The ddsp_vtimelocal_fail and ddsp_bogus_dsq_fail tests skip calling
scx_bpf_dsq_insert_vtime() if scx_bpf_pick_idle_cpu() fails to find an idle
CPU (returns a negative error code). On loaded systems, this results in the
tests skipping the very assertions they are meant to verify.

Eliminate this flakiness by falling back to prev_cpu if no idle CPU is
found, ensuring the illegal dispatch operations are unconditionally
attempted and tested.

Fixes: a5db7817af78 ("sched_ext: Add selftests")
Signed-off-by: Michal Blaszczyk <michalblk@google.com>
Signed-off-by: Tejun Heo <tj@kernel.org>

A CPU returned by scx_bpf_pick_idle_cpu_node() can be re-advertised as
idle by an idle-to-idle re-pick before the BPF program validates the
selection, and the scx_bpf_pick_any_cpu_node() fallback doesn't claim
the CPU at all. Asserting that the picked CPU is absent from the node's
idle cpumask is therefore inherently racy.

Follow the same approach as commit 12da4723b679 ("selftests/sched_ext:
Make allowed_cpus idle validation race-free") and validate a stable
local invariant instead: a CPU executing ops.select_cpu() in a non-idle
scheduling context must not be advertised as idle in its node's idle
cpumask. Keep the node-membership validation of the picked CPU, which
is stable.

Signed-off-by: Cheng-Yang Chou <yphbchou0911@gmail.com>
Signed-off-by: Tejun Heo <tj@kernel.org>

Pull to receive:

 c10b216a072f ("sched/core: Handle pick_task() releasing the rq lock")
 f3629c63a4af ("sched/core: Make core-sched flips wait for in-flight selections")
 ffaab58d2175 ("sched_ext: Replace SCX_RQ_BAL_KEEP with a dispatch verdict return")
 3dd52416e44a ("sched_ext: Fix this_rq() assumptions in dispatch kfuncs")
 f2da9587118d ("sched_ext: Count rq lock releases in rq->scx.lock_drop_seq")
 d954004205c1 ("sched_ext: Fix rq->core_pick corruption under core scheduling")

for the pending core scheduling follow-ups and to resolve the conflicts
with the code reorganization and cap gate work on for-7.3.

ffaab58d2175 converts scx_dispatch_sched() to a dispatch verdict return
which for-7.3 moved from ext.c into inlines.h. Resolved by applying the
conversion to the relocated copy and combining balance_one()'s verdict
returns with the scx_task_can_stay_on_cpu() gate from the cap work.

ffaab58d2175 and 3dd52416e44a update scx_bpf_sub_dispatch() which
for-7.3 moved into sub.c. Resolved by applying the scx_locked_rq()
switch and the verdict test to the sub.c copy.

f2da9587118d instruments the open-coded lock releases in
consume_remote_task() which for-7.3 folded into switch_rq_lock().
Resolved by keeping the accounting in switch_rq_lock() which covers all
its callers.

d954004205c1 widens the put_prev_task_scx() WARN suppression to all
core-sched rqs on the same condition that for-7.3 gated with
scx_task_can_stay_on_cpu(). Resolved by combining both.

Signed-off-by: Tejun Heo <tj@kernel.org>

exit.c does not check the return value of exit__open(). If it returns
NULL, which happens when the skeleton wrapper fails to allocate the
object or open the BPF ELF, the very next line dereferences it via
SCX_ENUM_INIT(). That macro expands to SCX_ENUM_SET(), which accesses
skel->rodata, causing a NULL pointer dereference.

Every other selftest guards this with SCX_FAIL_IF(!skel, ...); exit.c
is the only one missing it.

Fixes: a5db7817af78 ("sched_ext: Add selftests")
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>

A remotely selected CPU can be re-advertised as idle by an idle-to-idle
re-pick before the BPF program validates the selection. Checking that
the selected CPU remains absent from the idle mask is therefore
inherently racy.

Validate a stable local invariant instead: a CPU executing
ops.select_cpu() or ops.enqueue() in a non-idle scheduling context must
not be advertised as idle. Read the idle mask without modifying it and
also validate selected CPUs against the requested domain and task
affinity.

Suggested-by: Kuba Piecuch <jpiecuch@google.com>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Reviewed-by: Kuba Piecuch <jpiecuch@google.com>
Signed-off-by: Tejun Heo <tj@kernel.org>

When a sleeping task's affinity is changed, task_cpu(p) can be outside
of p->cpus_ptr until after select_task_rq() selects a new runqueue for
the task during wakeup.

Thus, the task's NUMA node determined by numa_select_cpu() can be
completely outside of the task's cpumask, leading to
scx_pick_{idle,any}_cpu_node() failing to find an eligible CPU and
returning -EBUSY. This leads to the numa.bpf.c scheduler abnormally
exiting with the following message in dmesg:

sched_ext: numa: invalid CPU -16
   scx_bpf_cpu_node+0x120/0x190
   bpf_prog_0a34b8e0f515771f_numa_select_cpu+0x108/0x14e
   bpf__sched_ext_ops_select_cpu+0x4f/0xb4
   select_task_rq_scx+0xb0/0x210
   select_task_rq+0xa0/0xd0
   __try_to_wake_up+0x196/0x650
   complete_all+0x76/0x100
   migration_cpu_stop+0x22b/0x300
   cpu_stopper_thread+0xc1/0x180
   smpboot_thread_fn+0x16b/0x230
   kthread+0x2d7/0x350
   ret_from_fork+0x1c2/0x350
   ret_from_fork_asm+0x1a/0x30

Make numa_select_cpu() robust against this case by returning @prev_cpu
if no CPU could be found in the selected NUMA node _and_ we have reason
to believe that the task's affinity was changed while it was sleeping.

Fixes: 5ae5161820e5 ("selftests/sched_ext: Add NUMA-aware scheduler test")
Signed-off-by: Kuba Piecuch <jpiecuch@google.com>
Signed-off-by: Tejun Heo <tj@kernel.org>

Pull to receive:

 db4e9defd2e8 ("sched_ext: Record an error on errno-only sub-enable failure")
 49b3378a750c ("sched_ext: Fix premature ops->priv publication in scx_alloc_and_add_sched()")
 e6979d05c6a6 ("tools/sched_ext: scx - Fix cmask_subset(), cmask_equal() and cmask_weight()")

for further sub-sched changes and to resolve the conflicts with the
sub-sched updates on for-7.3.

db4e9defd2e8 adds scx_error() to the sub-enable err_disable sink which
for-7.3 moved from ext.c into sub.c. Resolved by applying the fix to
scx_sub_enable_workfn() in sub.c.

49b3378a750c drops RCU_INIT_POINTER() from an scx_alloc_and_add_sched()
unwind label whose body changed with for-7.3's stall_cpus addition.
Resolved by dropping the line from the updated unwind.

Signed-off-by: Tejun Heo <tj@kernel.org>

In prog_run's run(), the bpf_link is attached early but only destroyed
on the success path. The three SCX_EQ assertions between attach and
destroy expand to a direct 'return SCX_TEST_FAIL', so if any of them
triggers, bpf_link__destroy() is never reached and the BPF scheduler
stays loaded. All subsequent tests then fail to attach because SCX is
not in the DISABLED state.

Convert those assertions to explicit checks that jump to a unified
'out' label which always runs the cleanup, matching the pattern used
in cyclic_kick_wait.c.

Fixes: a5db7817af78 ("sched_ext: Add selftests")
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>

Finite-slice EXT tasks need the periodic scheduler tick to expire their
slices even when nohz_full is enabled.

Add a regression test that selects a nohz_full CPU and exercises both
infinite-to-finite and finite-to-finite slice transitions across an idle
interval. For each finite task, verify that its ops.tick() callback is
invoked.

Skip the test when an allowed nohz_full CPU and a separate housekeeping
CPU are not available.

Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>

Pull sched_ext updates from Tejun Heo:
 "Most of this continues the in-development sub-scheduler support, which
  lets a root BPF scheduler delegate to nested sub-schedulers. The
  dispatch-path building blocks landed in 7.1. A follow-up patchset in
  development will complete enqueue-path support for hierarchical
  scheduling. This cycle adds most of that infrastructure:

   - Topological CPU IDs (cids): a dense, topology-ordered CPU numbering
     where the CPUs of a core, LLC, or NUMA node form contiguous ranges,
     so a topology unit becomes a (start, length) slice. Raw CPU numbers
     are sparse and don't track topological closeness, which makes them
     clumsy for sharding work across sub-schedulers and awkward in BPF.

   - cmask: bitmaps windowed over a slice of cid space, so a
     sub-scheduler can track, for example, the idle cids of its shard
     without a full NR_CPUS cpumask.

   - A struct_ops variant that cid-form sub-schedulers register with,
     along with the cid-form kfuncs they call.

   - BPF arena integration, which sub-scheduler support is built on. The
     bpf-next additions let the kernel read and write the BPF
     scheduler's arena directly, turning it into a real kernel/BPF
     shared-memory channel. Shared state like the per-CPU cmask now
     lives there.

   - scx_qmap is reworked to exercise the new arena and cid interfaces.

  Additionally:

   - Exit-dump improvements: dump the faulting CPU first, expose the
     exit CPU to BPF and userspace, and normalize the dump header.

   - Misc kfuncs and cleanups: a task-ID lookup kfunc, __printf checking
     on the error and dump formatters, header reorganization, and
     assorted fixes"

* tag 'sched_ext-for-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext: (59 commits)
  sched_ext: Add scx_arena_to_kaddr() / scx_kaddr_to_arena()
  sched_ext: Make scx_bpf_kick_cid() return s32
  sched_ext: Add scx_cmask_test() and scx_cmask_for_each_cid()
  tools/sched_ext: Order single-cid cmask helpers as (cid, mask)
  sched_ext: Order single-cid cmask helpers as (cid, mask)
  selftests/sched_ext: Fix dsq_move_to_local check
  sched_ext: Guard BPF arena helper calls to fix 32-bit build
  sched_ext: idle: Fix errno loss in scx_idle_init()
  sched_ext: Convert ops.set_cmask() to arena-resident cmask
  sched_ext: Sub-allocator over kernel-claimed BPF arena pages
  sched_ext: Require an arena for cid-form schedulers
  sched_ext: Add cmask mask ops
  sched_ext: Track bits[] storage size in struct scx_cmask
  sched_ext: Rename scx_cmask.nr_bits to nr_cids
  tools/sched_ext: scx_qmap: Fix qa arena placement
  sched_ext: Mark !CONFIG_EXT_SUB_SCHED dummy stubs static inline
  sched_ext: Replace tryget_task_struct() with get_task_struct()
  sched_ext: Add scx_task_iter_relock() and use it in scx_root_enable_workfn()
  sched_ext: Fix ops_cid layout assert
  sched_ext: Use offsetofend on both sides of the ops_cid layout assert
  ...