| Age | Commit message (Collapse) | Author |
|---|
|
Root enable and scx_post_fork() enable a task only if it's on the ext class.
Tasks on other classes, possible under an SCX_OPS_SWITCH_PARTIAL root, are
left READY and enabled by switching_to_scx() when they switch over. The sub
enable-commit pass and the sub-disable re-home loop enable unconditionally,
so a fair-class READY task in the subtree becomes ENABLED while not on
sched_ext. A later switch to SCHED_EXT then trips the task state validation
WARN (ENABLED with the previous state not READY) and calls ops.enable() a
second time.
Gate scx_enable_task() on the task's class in both loops.
Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A sub-scheduler enable can fail before scx_link_sched() links the sched into
the hierarchy, e.g. when the parent is already being disabled, and cleanup
still runs the full scx_sub_disable().
That is racy against root disable: drain_descendants() is the only ordering
between a sub's disable-time task walk and root disable's all-task teardown,
and an unlinked sub is invisible to it. Root's teardown can thus run between
the never-linked sub's drain and its walk, exiting every task to no
scheduler.
The walk then trips the membership WARN and re-homes the exited tasks onto
the dying hierarchy, a use-after-free.
Skip the cgroup ownership reset and the task walk if @sch was never linked,
indicated by the empty ->sibling as unlinking only happens later in the same
function. The membership WARN remains valid: a linked sub is always waited
on by an ancestor's drain.
Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_cgroup_lock() write-locks scx_cgroup_ops_rwsem and then takes
cgroup_lock(), which can deadlock through kernfs:
scx enable/disable cgroup rmdir cpu.weight write
------------------ ------------ ----------------
cgroup_lock()
percpu_down_write(rwsem)
cgroup_lock()
kernfs_get_active()
percpu_down_read(rwsem)
kernfs_drain()
The enable path waits for the rmdir to release cgroup_mutex. The rmdir,
deactivating the cpu controller's files, waits in kernfs_drain() for the
write's active reference. The write, in scx_group_set_weight(), waits for
the rwsem behind the pending writer.
Take cgroup_lock() first. The set_* paths take no cgroup locks inside the
read side, so a pending write-lock then only waits for read sections that
always run to completion, and no dependency from the rwsem back to
cgroup_mutex remains.
Fixes: a5bd6ba30b33 ("sched_ext: Use cgroup_lock/unlock() to synchronize against cgroup operations")
Cc: stable@vger.kernel.org # v6.18+
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The p->scx.disallow revert assumes the root enable path, where the switching
loop reads the reverted policy right afterwards and leaves the task off SCX.
The sub-scheduler disable path also reaches it when re-initializing the
returned tasks on a root parent. Nothing reads the policy there: the task is
enabled on root anyway and keeps running on the ext class with a silently
rewritten policy.
Kill the sched instead, matching the fork and non-root branches, and update
the disallow documentation, which equated !fork with the load path and
pointed at a stale debugfs path for nr_rejected.
Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Cross-merge BPF and other fixes after downstream PR.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
check_kfunc_call() reads meta.func_name when bpf_fetch_kfunc_arg_meta()
returns -EACCES, but that error can come from fetch_kfunc_meta() (e.g.
fd_array_get_btf() rejecting BTF binding for a signed program) before
meta is memset(), leaving it uninitialized and risking a garbage deref
in verbose().
Move the memset() to the start of bpf_fetch_kfunc_arg_meta() so meta is
zeroed on every error return. The intended "not allowed" -EACCES path
still sets func_name first, so its message is unchanged.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260715172127.2416388-3-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
For a bloom filter, the value argument of bpf_map_peek_elem() is always
an input. Therefore, the verifier should not allow passing uninitialized
stack memory to it to avoid information leak.
bpf_map_peek_elem() tags its value argument ARG_PTR_TO_MAP_VALUE |
MEM_UNINIT, telling the verifier the callee fills the buffer. This holds
for queue/stack maps, but not for a bloom filter, which reads the buffer
as an input to test set membership and never writes it.
As a result, a program can pass an uninitialized stack buffer to
bpf_map_peek_elem() on a bloom filter. The verifier accepts it and marks
the buffer initialized on return, letting the program read back leftover
kernel stack memory. Bloom maps require CAP_BPF to create, so this is a
CAP_BPF-gated stack infoleak that bypasses the boundary CAP_BPF is meant
to enforce (arbitrary kernel reads are gated behind CAP_PERFMON).
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260715172127.2416388-2-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Pull bpf fixes from Kumar Kartikeya Dwivedi:
- Fix a UAF in socket clone early bailout paths (Matt Bobrowski)
- Reject unhashed UDP sockets on sockmap update to prevent refcount
leaks (Michal Luczaj)
- Account for receive queue data in FIONREAD on sockmap sockets without
a verdict program (Mattia Meleleo)
- Reject negative constant offsets for verifier buffer pointers (Sun
Jian)
- Fix for tracing of kfuncs with implicit arguments (Ihor Solodrai)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
selftests/bpf: Cover tracing implicit kfunc args
bpf: Fix tracing of kfuncs with implicit args
selftests/bpf: Cover negative buffer pointer offsets
bpf: Reject negative const offsets for buffer pointers
selftests/bpf: Test FIONREAD on a sockmap socket without a verdict program
bpf, sockmap: Account for receive queue in FIONREAD without a verdict program
selftests/bpf: Fail unbound UDP on sockmap update
selftests/bpf: Adapt sockmap update error handling
bpf, sockmap: Reject unhashed UDP sockets on sockmap update
selftests/bpf: Ensure UDP sockets are bound
bpf: Fix UAF in sock clone early bailouts
|
|
The function try_enable_preferred_console() currently has the
non-obvious side effect of returning success for consoles that are
already pre-enabled. This obscures the logic flow during console
registration.
Move the check for pre-enabled consoles directly into the top-level
register_console(). This change makes the handling of pre-enabled
consoles explicit and easier to follow.
Furthermore, this separation lays the groundwork for future cleanups
where try_enable_preferred_console() can be restricted to cases where
an entry actually exists in the console_cmdline[] array.
Also it fixes a possible out-of-bound access when the console_cmdline[]
array is full and no console matched. In fact, the check of
c->user_specified did not make much sense. The new console either
matched and was handled in the for-cycle. Or it did not match
and then *c pointed to an unused entry.
Possible behavior change:
try_enable_preferred_console() will newly be called also with
@user_specified parameter set to "false" when it failed with the "true"
variant. But it looks like the right way to do. It will allow to call
newcon->setup() when the console was preferred by some platform
specific code.
Reported-by: Naveen Kumar Chaudhary <naveen.osdev@gmail.com> # out-of-bound access
Closes: https://lore.kernel.org/r/7sq4tr2nmlz32tvkf6vpsghv6exvqfghsrlvywjcqihzsqqbf7@bspclmti5xg4
Reviewed-by: John Ogness <john.ogness@linutronix.de>
Link: https://patch.msgid.link/20260604101459.393162-2-pmladek@suse.com
Signed-off-by: Petr Mladek <pmladek@suse.com>
|
|
The SWIOTLB bounce buffer pool size is hardcoded at 64 MB via
IO_TLB_DEFAULT_SIZE with no compile-time knob to adjust it. On
memory-constrained embedded or mobile platforms equipped with a
hardware IOMMU (e.g., ARM SMMU) covering most DMA-capable devices,
reserving 64 MB at boot is unnecessarily wasteful — the SWIOTLB is
only exercised for devices that bypass the IOMMU or have restricted
DMA address ranges.
Introduce CONFIG_SWIOTLB_DEFAULT_SIZE_MB, an integer Kconfig option
(range 1–64 MB, default 64) that allows platforms to set a smaller
compile-time default. IO_TLB_DEFAULT_SIZE is updated to derive from
this value when CONFIG_SWIOTLB is enabled, preserving the existing
64 MB default when the option is not configured.
The runtime "swiotlb=<nslabs>" kernel parameter override remains
fully supported and takes precedence over the compile-time default.
Signed-off-by: Jagadeesh Pagadala <jpagadal@qti.qualcomm.com>
Signed-off-by: Bibek Kumar Patro <bibek.patro@oss.qualcomm.com>
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Link: https://lore.kernel.org/r/20260702-swiotlb-v2-1-9205f3ba5408@oss.qualcomm.com
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
Some JIT compilers, such as x86_64, rely on a register to pass the TCC.
When subprograms of synchronous callback invoke tailcall, C helpers
invoking bpf callback clobber this register, and the corrupted TCC may
bypass the TCC limit, leading to infinite tailcall.
Fix this by rejecting tailcall inside all subprogs of sync callback.
This also cleanly consolidates the existing async and exception callback
checks into a single unified `is_cb` check.
Reported-by: Sashiko <sashiko-bot@kernel.org>
Reported-by: Björn Töpel <bjorn@kernel.org>
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260716120157.835937-3-pulehui@huaweicloud.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Currently in check_max_stack_depth_subprog, when the verifier enters a
new callee branch, the local tail_call_reachable is not properly
synchronized with the callee's state.
Consider a main prog branching into multiple subprogs:
subprog0 -> tailcall
main <
subprog1 -> subprog2
When the verifier finishes checking subprog0 and backtracks to main
prog, the local tail_call_reachable state is left as true. As it
proceeds to subprog1, this uncleared state leaks into the new branch,
falsely marking subprog1 and subprog2 as tailcall reachable.
Fix this by explicitly syncing tail_call_reachable with the callee's
has_tail_call state on entry. The caller's state is safely preserved and
restored via the existing backtracking logic.
Fixes: ebf7d1f508a7 ("bpf, x64: rework pro/epilogue and tailcall handling in JIT")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Link: https://patch.msgid.link/20260716120157.835937-2-pulehui@huaweicloud.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
A kfunc marked with KF_IMPLICIT_ARGS flag takes implicit arguments
(such as bpf_prog_aux) that the verifier injects at load time.
resolve_btfids strips those from the kfunc's BTF-visible prototype and
keeps the real kernel ABI in a counterpart _impl prototype [1].
fentry/fexit/fmod_ret/fsession programs may attach to the BPF kernel
functions, including those with implicit args. However
bpf_check_attach_target() and bpf_check_attach_btf_id_multi() extract
the struct btf_func_model from the wrong BTF prototype of the
kfunc. The btf_func_model is later read to construct the trampoline,
which then causes the injected implicit argument to be clobbered and
the kfunc dereferencing garbage.
Add btf_attach_func_proto() to resolve the real ABI prototype of the
kfunc the way the call site does: by looking up the _impl prototype
for a KF_IMPLICIT_ARGS kfunc. Use it at both attach-target model
construction sites.
To enable this, make two supporting changes:
* pass bpf_verifier_log instead of bpf_verifier_env to
find_kfunc_impl_proto(), so it can be reused from the attach path
* add btf_kfunc_check_flag() to test a flag across all of a kfunc's
hook sets, because a program attaching to a kfunc is not in the
kfunc's call-set
KF_IMPLICIT_ARGS must be consistent across the sets, so
btf_kfunc_check_flag() returns -EINVAL on inconsistency.
btf_kfunc_check_flag() reads the kfunc's flags from the target's
kfunc_set_tab. For a module BTF that table is stable only after the
module is live, so take a module reference around the read, mirroring
how the kfunc call path gates the same lookup with btf_try_get_module().
The remaining call sites of btf_distill_func_proto() are safe as
is. The BPF_TRACE_ITER case distills a registered iterator's
prototype, and bpf_struct_ops_desc_init() distills the
function-pointer members of a struct_ops type. Neither is a kfunc, and
so can't have implicit arguments.
[1] https://lore.kernel.org/all/20260120222638.3976562-1-ihor.solodrai@linux.dev/
Fixes: 64e1360524b9 ("bpf: Verifier support for KF_IMPLICIT_ARGS")
Reported-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://github.com/sched-ext/scx/issues/3687#issuecomment-4906694106
Link: https://patch.msgid.link/20260713235223.1639022-2-ihor.solodrai@linux.dev
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
cpuset_can_attach()
With cgroup v2 migration of a multithreaded process having threads
in different cgroups of a threaded subtree, it is possible that
cpuset_can_attach() can be called with tasks that are not migrating with
respect to cpuset if cpuset controller is not enabled in some of the
subtree cgroups. IOW, the old cpuset can be the same as the new one. This
can cause problem when we need to track the set of old cpusets and the
new cpusets in singly linked lists as a cpuset cannot be in both lists.
As reported by Tejun, the following is an example threaded subtree with
partial cpuset delegation that can cause this issue to show up.
P (+cpuset)
|- R (cpuset) <- destination
| `- C (no cpuset) -> effective cpuset == R
`- W (cpuset)
Group leader in R, thread_a in C, thread_b in W; migrate the whole
process into R (echo $PID > R/cgroup.procs). thread_a moves C->R:
its cgroup changes so compare_css_sets() keeps it in the taskset, but
its cpuset css is unchanged (C inherits R's), so task_cs() == cs ==
R. cpuset is in ss_mask because thread_b (W->R) changed. can_attach()
then tags R as a source (thread_a) and the destination (thread_b):
Handle this special case by skipping tasks that are not migrating in
cpuset_can_attach() and avoid calling cpuset_can_attach_check() in this
case. By doing so, the destination cpuset will not be put into source
cpuset linked list.
As the source cpuset cannot be easily determined in cpuset_attach(),
unnecessary work can be performed if a task is not actually
migrating. However, no harm will be done except wasting some
CPU cycles. If it happens that none of the tasks is migrating,
attach_ctx.old_cs will be NULL and task iteration won't be needed.
Reported-by: Tejun Heo <tj@kernel.org>
Closes: https://lore.kernel.org/lkml/e254af713b5345aec3d086771ecf1e71@kernel.org
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The only case where the cgroup_taskset structure requires task migration
to multiple cpusets is when enabling a cpuset controller in cgroup v2
where the newly created child cpusets inherits the same effective CPUs
and memory nodes from the parent. In that case, task migration can happen
directly with no update to tasks' CPU and memory nodes assignment and no
further work needed from the cpuset side except updating nr_deadline_tasks
when DL tasks are involved and setting old_mems_allowed in the child
cpusets.
Do that by tracking all the destination cpusets with a new dst_cs_head
singly linked list. The reset_migrate_dl_data() function is integrated
into clear_attach_data() so that it can be used for both source and
destination cpusets.
A warning will be printed if there are multiple destination cpusets but
it is not on default hierarchy or when the CPUs or memory nodes change.
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The smp_call*() functions handle their required preemption and CPU
pinning internally. The explicit preempt_disable() in
scftorture_invoke_one() is therefore no longer required for correctness.
Keeping the outer preempt_disable() would also prevent scftorture from
exercising the narrowed internal preemption-disabled regions during IPI
dispatch.
Removing the outer preemption protection can expose a CPU hotplug race in
the test validation when use_cpus_read_lock is false. For multicast
operations, SCF_PRIM_MANY or SCF_PRIM_ALL, if only one CPU is online,
smp_call_function_many() correctly skips sending IPIs and leaves scfc_out
false. Without preemption disabled, a CPU hotplug thread can preempt the
test thread, bring a second CPU online and increment num_online_cpus().
When the test thread resumes, the validation check can observe
num_online_cpus() > 1 and falsely trigger the memory-ordering warning,
leaking the scfcp structure.
Remove the preempt_disable() and preempt_enable() pairs around the
smp_call*() invocations in scftorture_invoke_one(). Restrict the
num_online_cpus() > 1 validation to the use_cpus_read_lock=true case,
where the CPU count is stable during the evaluation.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-10-zhouchuyi@bytedance.com
|
|
smp_call_function_many_cond() handles the preemption and CPU pinning
requirements internally. on_each_cpu_cond_mask() only builds the call
flags and forwards the request to that helper.
Remove the outer preempt_disable() and preempt_enable() pair from
on_each_cpu_cond_mask().
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-9-zhouchuyi@bytedance.com
|
|
smp_call_function_many_cond() handles the preemption and CPU pinning
requirements internally. smp_call_function() only forwards the request to
that helper for cpu_online_mask and does not access CPU-local state on
its own.
Remove the outer preempt_disable() and preempt_enable() pair from
smp_call_function().
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-8-zhouchuyi@bytedance.com
|
|
smp_call_function_many_cond() still has to keep the caller pinned to the
current CPU while the remote IPI request is built and dispatched. This
protects the queueing state and CPU-hotplug boundary that are required
before the synchronous wait starts:
- It protects the current CPU's per-CPU scratch cpumask,
cfd->cpumask_ipi. Another task running on the same CPU could otherwise
enter smp_call_function_many_cond() and reuse that scratch cpumask
before the current caller has finished building and sending the IPI
request.
- It provides the CPU-hotplug exclusion required by the CSD queueing
side. New CSDs must not be queued after smpcfd_dying_cpu() has flushed
the outgoing CPU's callback queue. Keeping preemption disabled until
all required CSDs have been queued and the corresponding IPIs have
been sent prevents CPU offline from crossing that boundary in the
middle of the queueing operation.
The CSD acquisition side also relies on that caller-side CPU pinning.
csd_lock() waits for CSD_FLAG_LOCK to clear and then marks the CSD busy
with a regular store, so another task on the same CPU must not be
allowed to acquire and reinitialize the same per-CPU CSD concurrently.
After the callbacks have been queued and the IPIs have been sent, the
caller only performs the final csd_lock_wait() completion wait. If it is
preempted there, another task running on the original CPU may enter
smp_call_function_many_cond(), but any attempt to reuse the same per-CPU
CSD will block in csd_lock() until the previous callback clears
CSD_FLAG_LOCK. The final csd_lock_wait() does not acquire or reinitialize
the CSD, so it does not need the same caller-side preemption-disabled
protection.
The wait mask is task-local, so it cannot be overwritten by another task
on the original CPU. The per-CPU CSD storage also remains allocated
across CPU offline, so csd_lock_wait() can safely dereference it even if
the target CPU is offlined after the caller is unpinned.
With those requirements satisfied, enable preemption before the
synchronous csd_lock_wait() loop. This makes the potentially long wait
preemptible and migratable while keeping the CPU-pinned section around
the remote CPU selection and IPI dispatch.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-7-zhouchuyi@bytedance.com
|
|
smp_call_function_many_cond() uses per-CPU CSD objects when queueing
callbacks to remote CPUs, and the wait path later dereferences those CSDs
from csd_lock_wait().
Making the wait path preemptible allows the initiating task to be
preempted or migrated before it waits for completion. A target CPU can be
offlined in that window. If smpcfd_dead_cpu() frees the target CPU's
per-CPU CSD storage, csd_lock_wait() can later dereference freed memory.
One way to protect the CSD storage is to free it via RCU or after a
synchronization step in the CPU offline path, but that would add
unnecessary complexity and can delay CPU shutdown.
Allocate the per-CPU CSD storage the first time a CPU comes up and keep
it allocated when the CPU is offlined. This allows csd_lock_wait() to
access the CSD even when the target CPU is offlined after preemption is
re-enabled and before the wait is invoked.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Acked-by: Muchun Song <muchun.song@linux.dev>
Link: https://patch.msgid.link/20260709122933.4021501-6-zhouchuyi@bytedance.com
|
|
smp_call_function_many_cond() uses the per-CPU cfd->cpumask as the list
of remote CPUs to wait for. That is safe while the caller remains pinned
to the current CPU for the whole operation, because another task cannot
run on the same CPU and reuse the per-CPU mask.
The synchronous wait is the long-latency part of the operation. To make
that wait preemptible, the mask iterated by csd_lock_wait() must remain
stable even if the task is preempted or migrates. If the wait used the
per-CPU cfd->cpumask after dropping CPU pinning, another task scheduled
on the original CPU could enter smp_call_function_many_cond() and
overwrite the mask while the first task is still iterating it.
Give each task private IPI cpumask storage and use it as the wait mask in
smp_call_function_many_cond(). Other cpumask storage choices do not fit
this use case:
- Per-CPU storage is the state that becomes unsafe once the wait is
made preemptible. After the caller drops CPU pinning, another task
scheduled on the original CPU can enter smp_call_function_many_cond()
and reuse the same per-CPU mask.
- Stack storage is not suitable for large NR_CPUS or
CONFIG_CPUMASK_OFFSTACK=y configurations. The wait mask needs to
scale with cpumask_size(), and putting that storage on the stack is
not acceptable on large systems.
- Allocating the mask inside smp_call_function_many_cond() would put an
allocation and a failure path in the generic IPI path. A sleeping
allocation is not suitable because callers have historically only
provided a preempt-disabled context, not a sleepable one. GFP_ATOMIC
would avoid sleeping, but a failure fallback would make the latency
improvement opportunistic instead of guaranteed.
The users are not limited to a small, pre-identifiable class of tasks. On
x86, ordinary tasks can reach this path through TLB flushes during exit,
unmap and reclaim, so allocating the mask only for a known subset of
tasks is not straightforward.
The memory cost is explicit: one word is added to task_struct. When
cpumask_size() fits in that word, the mask is stored inline and no
separate allocation is needed. Larger systems allocate cpumask_size() per
task; on x86-64 NR_CPUS=8192 this is 1 KiB per task. For context,
x86 already carries several KiB of per-task architecture and FPU state,
depending on the enabled features and configuration. That does not make
the extra cpumask free, but it puts the large-NR_CPUS case in
perspective.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-5-zhouchuyi@bytedance.com
|
|
smp_call_function_any() disables preemption across the entire operation:
selecting a target CPU, enqueueing the IPI, and synchronously waiting for
the remote CPU. smp_call_function_single() already re-enables preemption
before the synchronous csd_lock_wait(), so callers of
smp_call_function_any() should benefit from the same shorter
preemption-disabled section.
Simply removing get_cpu() and put_cpu() from smp_call_function_any()
would leave the preemption disablement entirely to
smp_call_function_single(). That opens a preemption window between
selecting the remote CPU, for example via sched_numa_find_nth_cpu(), and
dispatching the IPI in smp_call_function_single(). If the selected CPU is
fully offlined in that window, smp_call_function_single() fails its
cpu_online() check and returns -ENXIO to the caller, violating the
guarantee that smp_call_function_any() executes on any online CPU in the
mask.
Move the remote CPU selection into a common
__smp_call_function_single() helper. Keep the target CPU selection and
IPI dispatch within the same preemption-disabled region, while still
allowing the wait path to use the shorter preemption-disabled section
provided by smp_call_function_single().
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-4-zhouchuyi@bytedance.com
|
|
smp_call_function_single() disables preemption while it validates the
target CPU, prepares the call single data, queues the callback and sends
the IPI.
For the !wait case, preemption protects the per-CPU csd_data from
concurrent modification by another task on the same CPU. For the wait
case, the CSD is stack allocated and no other task can reuse it. CPU
pinning is still required until the callback has been queued and the IPI
has been sent, to ensure that the target CPU cannot be offlined after the
online check but before dispatch.
After generic_exec_single() has queued the callback, the synchronous
csd_lock_wait() invocation at the end of the execution does not require
the caller to remain pinned to the current CPU.
Enable preemption before csd_lock_wait() to shorten the
preemption-disabled section.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-3-zhouchuyi@bytedance.com
|
|
The CSD lock wait debugging code in __csd_lock_wait() must run with
preemption disabled. The smp function call mechanisms which invoke it
currently keep preemption disabled across the wait, so the debugging code
inherits that guarantee from its callers.
Keeping preemption disabled across the whole smp function call operation
can induce large scheduling latencies. Shortening the caller-side
preemption-disabled region would invoke __csd_lock_wait() with preemption
enabled.
Prepare for that by disabling preemption explicitly around the CSD lock
wait debugging code in __csd_lock_wait().
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Acked-by: Muchun Song <muchun.song@linux.dev>
Link: https://patch.msgid.link/20260709122933.4021501-2-zhouchuyi@bytedance.com
|
|
LIVEUPDATE_SESSION_GET_NAME was developed in the liveupdate/next branch
while the session type validation change was carried in liveupdate-fixes.
When the conflict between the two branches was resolved, the GET_NAME
operation descriptor picked up the structure and last member from
RETRIEVE_FD.
This makes both its known size and minimum size 16 bytes rather than 72.
A zero-initialized request still succeeds because luo_session_get_name()
writes the full name before luo_ucmd_respond() copies the full GET_NAME
response to userspace. However, copy_struct_from_user() treats the
output-only name field as unknown trailing data and rejects the request
with -E2BIG if any byte in that field is nonzero.
Use the GET_NAME structure and its name field in the descriptor.
Link: https://lore.kernel.org/all/ahWlYXNjGUbkKoHy@sirena.org.uk/
Assisted-by: Codex:gpt-5.6-sol
Reviewed-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Signed-off-by: Jackie Liu <liuyun01@kylinos.cn>
Link: https://patch.msgid.link/20260716012607.22020-1-liu.yun@linux.dev
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
With CONFIG_EXT_SUB_SCHED=y but no sub-scheduler attached - the common case
- hot paths still pay for sub-sched bookkeeping. Gate it behind
__scx_has_subs, a static key counting live sub-schedulers, so that a
root-only system stops paying.
Most conversions are simple skip-if-no-sub tests. scx_idle_notify() is
special - it's a hierarchy walk, so give it a fast path which notifies the
root directly using the same tests as the walk. A pending
SCX_RQ_SUB_IDLE_RENOTIFY can be ignored as no sub can be owed one and the
caller clears the flag either way.
Suggested-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
rq->scx.sub_dispatch_prev is sub-sched-only but was left unconditional. Move
it into the CONFIG_EXT_SUB_SCHED block next to ecaps_to_sync and gate its
updates.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_dispatch_sched() is common dispatch machinery and looks out of place in
sub.h, but it needs scx_cpu_arg() from cid.h and can't move into internal.h
without creating a circular include. Add inlines.h on top of internal.h and
cid.h, and move the function there. The function was sub.h's only cid.h
user, so drop that include. Pure code move, no functional change.
v2: Host the function in a new inlines.h instead of at internal.h's tail,
which formed a circular include with cid.h. Drop sub.h's now-unused
cid.h include. (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_discard_ecaps_to_sync() waited for balance_one() to consume a dying
sched's queued ecaps sync, polling with resched_cpu() + msleep(). The wait
is unbounded - the ext dl_server forces picks through sustained fair or RT
load only while ext tasks are queued, so an ext-idle cpu monopolized by a
higher class can stall the teardown indefinitely.
Remove the node directly instead: take all queued nodes, drop the dying
sched's and resplice the rest. Consumption runs under the rq lock and batch
nodes read as on-list throughout, so the producer-side dedup stays correct.
A node that an in-flight scx_process_sync_ecaps() batch holds across a
dispatch-induced rq unlock still needs a wait, but one bounded by that batch
completing rather than by a future balance.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
This is needed by rust/helpers/srcu.c which now adds
rust_helper_srcu_readers_active() as a wrapper around the SRCU helper
for Rust callers.
To achive this:
1- Move the srcu_readers_active() implementation from
"kernel/rcu/srcutree.c" to "include/linux/srcutree.h".
2- Implement a matching srcu_readers_active() in
"include/linux/srcutiny.h" and use it on the existing open-coded
WARN_ON() check in cleanup_srcu_struct().
Signed-off-by: Onur Özkan <work@onurozkan.dev>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Restructure the SRCU initialization functions so it always follows
one direction:
init_srcu_struct() -> __init_srcu_struct() -> lockdep or generic
This uses the same wrapper style as mutex. It avoids the old confusing
style where init_srcu_struct() and __init_srcu_struct() called each
other in different configs. It also helps Rust side to have simpler
helper for SRCU initialization.
Signed-off-by: Onur Özkan <work@onurozkan.dev>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
This commit abstracts the open-coded dumping of reader segments in the
rcu_torture_cleanup() function into a new rcu_torture_dump_read_segs()
function. This abstraction will allow reader segments to be dumped for
other purposes.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Now that Tasks Trace RCU is implemented in terms of SRCU, it no longer
has any particular need for the IRQ_WORK Kconfig option. This commit
therefore removes the "select IRQ_WORK" from the TASKS_TRACE_RCU Kconfig
option.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
The kerneldoc of sugov_iowait_apply() says the IO boost value is increased
in sugov_iowait_apply() and, in the same sentence, that it is decreased by
the same function. That is self-contradictory, and the first part is wrong:
sugov_iowait_apply() only decreases the boost.
The boost is actually increased in sugov_iowait_boost(). Fix the comment to
name sugov_iowait_boost() as the place where the boost is increased, so it
matches the code.
No functional change.
Fixes: fd7d5287fd65 ("cpufreq: schedutil: Cleanup and document iowait boost")
Signed-off-by: Zhongqiu Han <zhongqiu.han@oss.qualcomm.com>
Reviewed-by: Christian Loehle <christian.loehle@arm.com>
Link: https://patch.msgid.link/20260703092433.4080165-1-zhongqiu.han@oss.qualcomm.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
|
|
In some cases, the processor may not actually stick to the "desired"
performance level programmed through the driver's .adjust_perf()
callback and may go above it, which may not be desirable (for instance,
there may be a UCLAMP_MAX limit set for the task currently running on
the given CPU which should be respected).
Address that by adjusting the .adjust_perf() callback to take an
additional argument, max_perf, representing the maximum allowed
performance level of the CPU and update the intel_pstate driver to
take that argument into account as appropriate.
Accordingly, adjust cpufreq_driver_adjust_perf() and the other existing
user of .adjust_perf(), which is the amd-pstate driver (but the behavior
of that driver is not changed).
While at it, also update the cpufreq_driver_adjust_perf()
documentation to reflect this change and some previous code
changes that have not been taken into account in it.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Zhongqiu Han <zhongqiu.han@oss.qualcomm.com>
Reviewed-by: Mario Limonciello (AMD) <superm1@kernel.org>
Link: https://patch.msgid.link/6277654.lOV4Wx5bFT@rafael.j.wysocki
[ rjw: Adjusted Rust function formatting ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
|
|
psi_schedule_rtpoll_work() is called locklessly from the scheduler hotpath
and can race psi_trigger_destroy() taking down the last rtpoll trigger under
rtpoll_trigger_lock:
psi_schedule_rtpoll_work() psi_trigger_destroy()
rcu_read_lock();
task = rcu_dereference(rtpoll_task);
rcu_assign_pointer(rtpoll_task, NULL);
timer_delete(&rtpoll_timer);
mod_timer(&rtpoll_timer, ...);
rcu_read_unlock();
synchronize_rcu();
kthread_stop(task_to_destroy);
The group can then be freed with the re-armed timer still pending, and
poll_timer_fn() runs on freed memory.
461daba06bdc ("psi: eliminate kthread_worker from psi trigger scheduling
mechanism") deleted the timer synchronously after the synchronize_rcu(),
which prevented this but raced trigger creation instead: the deletion could
cancel the timer that a new trigger set armed during the grace period and,
as creation also reinitialized the timer at the time, corrupt it.
8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy") moved the
initialization into group_init() and the deletion into the locked section,
trading the creation races for the window above.
Neither placement in the destruction path works. A pending timer firing
while the group is alive is harmless though. poll_timer_fn() just wakes the
rtpoll waitqueue and doesn't re-arm itself. Bind the timer to the group's
lifetime instead and shut it down in psi_cgroup_free(). Nothing can arm it
by then. timer_shutdown_sync() because the timer is never armed again.
Fixes: 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy")
Cc: stable@vger.kernel.org # v5.10+
Reported-by: Sashiko AI <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260711000434.36C4A1F000E9@smtp.kernel.org/
Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Tested-by: Matt Fleming <mfleming@cloudflare.com>
Acked-by: Suren Baghdasaryan <surenb@google.com>
|
|
a5b98009f16d ("sched/psi: fix race between file release and pressure write")
made pressure_write() hold cgroup_mutex across psi_trigger_create(), which
forks the psimon kthread for the first rtpoll trigger. As kthread creation
depends on the whole fork path, the commit inadvertently created a lot of
unwanted locking dependencies from cgroup_mutex.
sched_ext got hit by one: its enable path blocks forks and then grabs
cgroup_mutex, so a pressure write racing a scheduler enable deadlocks, with
every other fork piling up behind.
Fix it by splitting trigger creation so that the worker is forked with
cgroup_mutex dropped and the kernfs active reference left broken. The latter
matters because rmdir and cgroup.pressure writes drain active references
under cgroup_mutex. Publishing the trigger last keeps error reporting
synchronous and preserves the of->priv lifetime rules.
The trigger registered in the first stage pins the group's rtpoll machinery
across the unlocked window, leaving only creation races to resolve. The
catch-up poll on installation covers scheduling attempts dropped while there
was no worker.
v2: Retagged sched/psi (was cgroup).
Fixes: a5b98009f16d ("sched/psi: fix race between file release and pressure write")
Cc: stable@vger.kernel.org
Cc: Edward Adam Davis <eadavis@qq.com>
Cc: Chen Ridong <chenridong@huaweicloud.com>
Reported-by: Matt Fleming <mfleming@cloudflare.com>
Closes: https://lore.kernel.org/all/20260710100441.2653477-1-matt@readmodwrite.com/
Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Tested-by: Matt Fleming <mfleming@cloudflare.com>
Acked-by: Suren Baghdasaryan <surenb@google.com>
|
|
The verifier rejects variable offsets for PTR_TO_TP_BUFFER and PTR_TO_BUF
accesses, but it currently accepts a constant negative offset produced by
pointer arithmetic.
Commit 022ac0750883 ("bpf: use reg->var_off instead of reg->off for
pointers") moved constant pointer offsets from reg->off to reg->var_off.
However, __check_buffer_access() continued to check only the instruction
offset. An access with reg->var_off equal to -8 and an instruction offset
of zero therefore passes verification.
For writable raw tracepoints, the access end is also calculated from the
unsigned reg->var_off.value. An eight-byte access starting at -8 wraps
the calculated end to zero, allowing the program to load and attach
without increasing max_tp_access.
After ensuring that reg->var_off is constant, calculate the effective
access start using signed arithmetic and reject it when it is negative.
Use the validated start to calculate the access end for both
PTR_TO_TP_BUFFER and PTR_TO_BUF.
Fixes: 022ac0750883 ("bpf: use reg->var_off instead of reg->off for pointers")
Signed-off-by: Sun Jian <sun.jian.kdev@gmail.com>
Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Cc: stable@vger.kernel.org # 5.2.0
Link: https://patch.msgid.link/20260714093846.18159-2-sun.jian.kdev@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Helper and kfunc argument checking carried two separate meta structs: the
verifier-local struct bpf_call_arg_meta and bpf_kfunc_call_arg_meta.
Merge them into a single struct bpf_call_arg_meta. This is groundwork for
sharing argument checking between helpers and kfuncs.
While merging, drop the btf_id field from the helper meta since it is
never used.
No functional change.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-8-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
meta->pkt_access is only ever a copy of fn->pkt_access, assigned once in
check_helper_call() and read back in may_access_direct_pkt_data(). Have
may_access_direct_pkt_data() take the bpf_func_proto and read
fn->pkt_access directly, and drop the meta field along with its
assignment.
The only non-NULL caller, check_func_arg(), already has fn in scope; the
remaining callers pass NULL and are unaffected.
No functional change intended.
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-7-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
The constant "size of the PTR_TO_MEM returned in R0" argument is handled by
both helpers (ARG_CONST_ALLOC_SIZE_OR_ZERO) and kfuncs (__rdonly_buf_size /
__rdwr_buf_size), each with its own meta field (meta->mem_size,
meta->r0_size) and duplicated validation. Add struct arg_alloc_mem_desc
and a shared process_const_alloc_mem_size(), and replace both fields with
meta->arg_alloc_mem.
The desc records presence with a 'found' flag instead of using a non-zero
size as the sentinel. This also fixes a pre-existing bug on the kfunc
return path: "no size argument" was tested as r0_size == 0, so an explicit
__rdonly_buf_size/__rdwr_buf_size of 0 was treated as absent and fell
through to btf_resolve_size(), giving R0 the size of the pointed-to return
type instead of 0. With 'found', an explicit zero size is honored and
btf_resolve_size() is used only when no size argument was passed.
The size is stored in a u32, matching regs[R0].mem_size. The U32_MAX
check now apply to both helper and kfunc through
process_const_alloc_mem_size().
Fold bpf_session_cookie return size assignment into current kfunc return
size resolution path.
Note that verifier saves kfunc return size through r0_size instead of
mem_size. The later has no active readers so remove it.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-5-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
check_func_arg() only ever handles register arguments (the caller
loops over the first MAX_BPF_FUNC_REG_ARGS arguments), so a single
argno_t built from the register number identifies the argument for
every callee. Remove the duplicated argno_from_reg() calls to
simplify check_func_arg().
'regno' is still kept for the few places that need the raw register
number directly (register reads, verbose R%d messages) and for
referring to the neighbouring size/memory argument in the
ARG_CONST_SIZE{,_OR_ZERO} cases.
No functional change intended.
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-4-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
To prepare for unifying the helper and kfunc call_arg_meta, group the
scattered MEM_UNINIT ("raw") memory argument fields (raw_mode, regno and
access_size) into a new struct arg_raw_mem_desc. The intention is to
make it clear about when these are set and used instead of fields with
overly generic names.
Identify the raw argument once, up front, in check_raw_mode_ok() (like
check_proto_release_reg() does for release_regno), recording its regno.
check_stack_range_initialized() now recognizes the raw buffer by matching
that regno, so the separate raw_mode flag is no longer needed, and the
per-argument "meta->raw_mode = arg_type & MEM_UNINIT" assignments in
check_func_arg() go away with it.
A raw memory argument can be tagged either ARG_PTR_TO_MEM | MEM_UNINIT or
ARG_PTR_TO_MAP_VALUE | MEM_UNINIT (the output buffer of bpf_map_pop_elem()
and bpf_map_peek_elem()). Either may be passed as a PTR_TO_STACK, which
reaches check_stack_range_initialized() through check_helper_mem_access(),
so both must be treated as raw. Extend arg_type_is_raw_mem() to match the
map value case as well; otherwise check_raw_mode_ok() would not record the
regno for it and an uninitialized stack buffer passed to those helpers
would be wrongly rejected for programs without CAP_PERFMON.
No functional change intended. This patch does not enable raw_mode
memory access for kfunc (i.e., uninit stack will not be allowed to be
passed to kfunc for unprivileged programs). Existing kfuncs with arguments
tagged with __uninit are either priviledged or dynptr kfuncs, which take
another path to make sure the access is checked by check_mem_access().
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-3-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
For a MEM_UNINIT ("raw mode") helper argument,
check_stack_range_initialized() open-coded a scan that rejected any
STACK_DYNPTR slot in the range with "potential write to dynptr". This
duplicated, and was stricter than, the handling that runs when the
buffer is actually marked initialized. check_helper_call() later replays
the write byte by byte via check_mem_access(), which goes through
destroy_if_dynptr_stack_slot(), which rejects overwritting a referenced
dynptr. Therefore drop the redundant scan and rely on check_mem_access().
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-2-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
luo_session_alloc() calls INIT_LIST_HEAD(&session->file_set.files_list)
followed immediately by luo_file_set_init(), which also performs
INIT_LIST_HEAD() on the same list head. Remove the duplicate call.
Signed-off-by: Hongfu Li <lihongfu@kylinos.cn>
Reviewed-by: Pratyush Yadav <pratyush@kernel.org>
Link: https://patch.msgid.link/20260714074056.95335-1-hongfu.li@linux.dev
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
To avoid potential stack overflows on limited kernel stacks, convert
parse_probe_arg() from a recursive function into a loop-based
implementation with a simple local state stack.
Since recursion is eliminated using a loop with a fixed-size
stack in the context, this restricts the dereference nesting
depth. The maximum nesting depth of dereferences is now restricted
to the same limit as typecasts (TRACEPROBE_MAX_NESTED_LEVEL, which
is 8) and reports the same TOO_MANY_NESTED error. Update ftrace
selftests to reflect this restriction and simplify the checks.
Note that this change slightly alters the behavior of nested
dereferencing in fetcharg. Previously, dereferencing without BTF
allowed for up to 14 levels of nesting, whereas dereferencing
with BTF was limited to 3 levels. With this change, the nesting
depth is now limited to 8 levels in both cases.
Link: https://lore.kernel.org/all/178399141396.27810.5390060618628718661.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
To support BTF argument parsing (such as accessing fields within nested
structures via typecasting), the maximum argument string length needs
to be extended. Extend MAX_ARGSTR_LEN from 63 to 255.
Since MAX_ARGSTR_LEN was previously reused to format command heads in
trace_*probe_match_command_head() functions, introduce a dedicated
MAX_COMMON_HEAD_LEN (63) macro for matching command heads and switch
these functions to use the new macro.
Link: https://lore.kernel.org/all/178399140457.27810.11387684872148824707.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
Sort the C-macro ERRORS list alphabetically in trace_probe.h to make
it easier to find and maintain error entries.
Link: https://lore.kernel.org/all/178399139516.27810.14930549550788208142.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
Decompose parse_probe_arg() by extracting register, memory/symbol,
dereference, immediate, and default BTF/CPU parsing handlers into
dedicated static helper functions (parse_probe_arg_register,
parse_probe_arg_mem_symbol, parse_probe_arg_deref, parse_probe_arg_imm,
and parse_probe_arg_default). This modularizes the recursive argument
parser and improves readability.
Link: https://lore.kernel.org/all/178399138581.27810.9471730417467607229.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
Decompose parse_probe_vars() by extracting retval, stack, current task
struct, and function argument parsing logic into dedicated static
helper functions (parse_probe_var_retval, parse_probe_var_stack,
parse_probe_var_current, and parse_probe_var_arg). This simplifies
parse_probe_vars() and improves its readability.
Link: https://lore.kernel.org/all/178399137612.27810.14869178899276413372.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
