| Age | Commit message (Collapse) | Author |
|---|
|
scx_cpumask_to_cmask() has no callers.
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
select_task_rq_scx() falls into the default path when the scheduler has no
ops.select_cpu or is bypassing. There it calls scx_select_cpu_dfl() and
direct-dispatches to the picked CPU's local DSQ.
While bypassing, neither does anything: the enqueue path routes the task to
a bypass DSQ before consulting the direct-dispatch target, so the direct
dispatch never happens, and the CPU pick at most shifts which CPU's bypass
DSQ receives the task. Worse, when the scheduler does its own idle tracking,
the built-in idle cpumasks the pick consults are not even updated, so it
doesn't work anyway.
Return prev_cpu without the default selection while bypassing and let the
bypass enqueue place the task.
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
check_rq_for_timeouts() blames a runnable stall on the task's owner. Under
a sub-scheduler hierarchy the stalled task can be sitting on a DSQ that a
different scheduler has to drain, e.g. an ancestor's bypass DSQ while the
owner is bypassing. The drainer then escapes blame while the owner is
exited, and when the owner's exit is already claimed, nothing actionable is
reported at all.
Blame the DSQ's owning scheduler instead. The local DSQ is consumed by the
cpu itself and keeps blame on the owner. Detection keeps the owner's timeout
and single-scheduler behavior is unchanged.
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
BPF_REFCOUNT is not marked as a unique field, while it should be. Fix
this oversight.
Fixes: d54730b50bae ("bpf: Introduce opaque bpf_refcount struct and add btf_record plumbing")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260719153634.2908692-4-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
btf_find_struct_field() initializes a fresh seen mask for every recursive
descent. Unique special fields in different levels of the same aggregate
therefore do not see one another. The duplicate fields can reach
btf_parse_fields(), where they trigger an invariant WARN_ON_ONCE(). A
crafted user BTF can consequently trigger the warning before map creation
checks capabilities.
Initialize the seen mask once in btf_find_field() and pass the same pointer
through struct, datasec, and nested-struct walks. This gives the entire field
traversal one shared uniqueness state.
Fixes: 64e8ee814819 ("bpf: look into the types of the fields of a struct type recursively.")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260719153634.2908692-3-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Sashiko pointed out correctly that the case statement for
BPF_RES_SPIN_LOCK incorrectly checks offset for BPF_SPIN_LOCK.
Fix it by checking res_spin_lock_off instead.
Fixes: 0de2046137f9 ("bpf: Implement verifier support for rqspinlock")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260719153634.2908692-2-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Queue and stack pop/peek helpers accept an uninitialized output buffer
because the verifier expects the helper to initialize it. The empty-map
error path clears the buffer, but a failed lock acquisition returns
-EBUSY without writing it.
Clear the output before returning -EBUSY so BPF programs cannot observe
uninitialized stack contents after a failed helper call.
Fixes: a34a9f1a19af ("bpf: Avoid deadlock when using queue and stack maps from NMI")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260719125419.1782196-1-memxor@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Accept only one "region value" pair entry for the dmem.max, dmem.min,
dmem.low files.
This changes the UAPI that otherwise accepted multiple lines for setting
multiple entries in one write. No existing user is known to rely on
writing multiple regions in a single write.
Processing multiple regions in dmemcg_limit_write() could quietly change
first limits before failing on a later one and returning an error to the
writer, with no indication some changes occurred.
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Eric Chanudet <echanude@redhat.com>
Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Link: https://patch.msgid.link/20260608-cgroup-dmem-write-single-region-v2-1-b0cd6c4ccf1b@redhat.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
|
|
Clean up extra space after '=' in fetch_store_strlen_user().
Link: https://lore.kernel.org/all/178454430235.296567.9486503882045148585.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
kprobe_perf_func() calls memset(&entry[1], 0, dsize) prior to calling
store_trace_args(). store_trace_args() populates the entry buffer and
handles dynamic data fields.
Remove this redundant memset call to align with kretprobe_perf_func() and
other probe perf functions.
Link: https://lore.kernel.org/all/178454429309.296567.17379087988750549247.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
fentry_perf_func() calls memset(&entry[1], 0, dsize) prior to calling
store_trace_args(). store_trace_args() populates the entry buffer and
handles dynamic data fields.
Furthermore, passing dsize (the dynamic data byte length) to memset at
&entry[1] (the start of fixed trace arguments) is inaccurate as it zeroes
from the fixed args area rather than the dynamic data region.
Remove this redundant memset call to align with fexit_perf_func() and other
probe perf functions.
Link: https://lore.kernel.org/all/178454428366.296567.16465331441301815980.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
trace_fprobe_match_command_head() copies trace_fprobe_symbol(tf) into a
local buffer 'buf' of size MAX_COMMON_HEAD_LEN + 1 using snprintf before
comparing with argv[0].
Since trace_fprobe_symbol(tf) already returns a null-terminated string,
comparing it directly with argv[0] via strcmp() avoids stack buffer usage
and potential symbol truncation at MAX_COMMON_HEAD_LEN.
Link: https://lore.kernel.org/all/178454427449.296567.12336315661120939938.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
Use a ternary operator for checking IS_ENABLED(CONFIG_64BIT) in the
BTF_KIND_PTR case of fetch_type_from_btf_type() to simplify the code.
Link: https://lore.kernel.org/all/178454426554.296567.17700307233923830044.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
In store_trace_entry_data(), edata is cast to unsigned long for pointer
offset arithmetic before being cast back to unsigned long *. Cast edata
to u8 * instead.
Link: https://lore.kernel.org/all/178454425622.296567.16952341966130280432.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
The parameter 'arg' in parse_probe_var_retval() is unused. Remove it
and update its caller accordingly.
Link: https://lore.kernel.org/all/178454424711.296567.15352344258939682112.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
In trace_probe_compare_arg_type(), prior to entering the comparison loop,
a->nr_args and b->nr_args are checked for equality. Since the loop
condition is i < a->nr_args, i is guaranteed to be less than b->nr_args
inside the loop.
Remove the redundant (b->nr_args <= i) check.
Link: https://lore.kernel.org/all/178454423769.296567.6694636865644203423.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
trace_probe_has_single_file()
list_is_singular() returns a boolean value, so the double negation (!!)
in trace_probe_has_single_file() is redundant. Remove it.
Link: https://lore.kernel.org/all/178454422826.296567.4382363406595169533.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
MAX_ARRAY_LEN is defined twice in trace_probe.h. Remove the redundant
definition.
Link: https://lore.kernel.org/all/178454421926.296567.6649983666070787570.stgit@devnote2/
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
eprobe_dyn_event_match() checks if the target event system in argv[0]
matches ep->event_system using strncmp(ep->event_system, argv[0], len).
However, if ep->event_system is longer than len (e.g. "eprobes" vs
"ep/event"), strncmp() still returns 0 because the first len characters
match.
Check that ep->event_system[len] is '\0' to ensure exact system name
matching.
Link: https://lore.kernel.org/all/178454235856.290363.14872590900774231133.stgit@devnote2/
Fixes: 7d5fda1c841f ("tracing: Fix event probe removal from dynamic events")
Cc: stable@vger.kernel.org
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
In __set_print_fmt(), LEN_OR_ZERO is defined as (len ? len - pos : 0).
If len is non-zero but smaller than pos, len - pos evaluates to a negative
integer. When passed as a size argument to snprintf(), this negative value
is cast to a large unsigned size_t, bypassing buffer size limits.
Ensure len > pos before subtracting to avoid integer underflow.
Link: https://lore.kernel.org/all/178454234934.290363.15247317871499514139.stgit@devnote2/
Fixes: 5bf652aaf46c ("tracing/probes: Integrate duplicate set_print_fmt()")
Cc: stable@vger.kernel.org
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
If trace_probe_log.argc is 0 in __trace_probe_log_err(), the loop
constructing the command string will not execute and p will remain equal to
command. Writing to *(p - 1) will cause an out-of-bounds access before
command. This should not happen, but better to be treated.
Reject if trace_probe_log.argc is 0.
Link: https://lore.kernel.org/all/178454233992.290363.18323091580600697731.stgit@devnote2/
Fixes: ab105a4fb894 ("tracing: Use tracing error_log with probe events")
Cc: stable@vger.kernel.org
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
trace_probe_match_command_args()
In trace_probe_match_command_args(), a stack buffer buf[MAX_ARGSTR_LEN + 1]
(256 bytes) is used to format "<name>=<comm>". However, since name can
be up to 32 bytes (MAX_ARG_NAME_LEN) and comm up to 255 bytes
(MAX_ARGSTR_LEN), the formatted string can exceed 256 bytes and get
truncated by snprintf(), causing spurious argument matching failures.
Instead of formatting into a temporary buffer on stack, compare the
argument name, the '=' delimiter, and the comm expression directly.
Link: https://lore.kernel.org/all/178454233010.290363.10428767141343428804.stgit@devnote2/
Fixes: eb5bf81330a7 ("tracing/kprobe: Add per-probe delete from event")
Cc: stable@vger.kernel.org
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
Reread the syscall number from pt_regs and stop returning the eventually
modified syscall number.
That moves the reread to the end of syscall_trace_enter() and prepares for
moving it to the call site.
No functional change.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Michal Suchánek <msuchanek@suse.de>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Link: https://patch.msgid.link/20260712141346.639115923@kernel.org
|
|
Move it out of line and let it reread the syscall number on it's own. That
makes the low level entry code denser and allows to move the reread to the
call site of syscall_trace_enter() once the tracer is fixed up.
To prevent the compiler from putting audit_context() out of line and
thereby breaking dead code elimination, mark audit_context()
__always_inline.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Michal Suchánek <msuchanek@suse.de>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Link: https://patch.msgid.link/20260712141346.576865340@kernel.org
|
|
If a watcher deliberately disables interrupts (either by user choice, or
because we're dealing with a scoped reordered access) to avoid detecting
any data races in interrupts, NMIs are still able to fire.
When we set up a watchpoint on a scoped reordered access, we disabled
interrupts because the same CPU cannot observe reordering of its own
accesses. To ensure we observe no false positives from NMIs, disable
access checking for interrupt contexts as well.
Fixes: 69562e4983d9 ("kcsan: Add core support for a subset of weak memory modeling")
Signed-off-by: Marco Elver <elver@google.com>
|
|
Running kernel-doc on timer_migration.h reports:
Warning: kernel/time/timer_migration.h:105 bad line:
The empty line separating the @available paragraph from the @idle
member documentation in the struct tmigr_cpu kernel-doc block lacks
the " *" line prefix that kernel-doc requires on every line inside a
block. The header is not scanned by the build-time kernel-doc checks,
so the warning only shows up when kernel-doc is invoked on the file
directly.
Add the missing prefix. The empty line was introduced when the
@available documentation was expanded by commit 45a13ba52c82
("timers/migration: Update stale @online doc to @available").
No functional change.
Signed-off-by: Babanpreet Singh <bbnpreetsingh@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Claude:claude-sonnet-5
Link: https://patch.msgid.link/20260717035947.7-1-bbnpreetsingh@gmail.com
|
|
Hybrid automata monitors's clock variables have two different
representations:
- The invariant representation, which is the timestamp when the invariant
expires
- The guard representation, which is the timestamp when the clock is last
reset
This dual representation makes the logic quite difficult to follow (well,
at least for me). It also complicates the monitors and the generation tool,
as it requires conversion back and forth between the representation.
Simplify by using the clock variables for a single purpose: storing the
time stamp since the clock is last reset.
This also allows simplifying rvgen, which will be done in a follow-up
commit.
Reviewed-by: Gabriele Monaco <gmonaco@redhat.com>
Signed-off-by: Nam Cao <namcao@linutronix.de>
Link: https://lore.kernel.org/r/c0f600dcbf3d8b487c944406851a39146f4d91fa.1781847583.git.namcao@linutronix.de
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
|
|
EXPORT_SYMBOL_GPL(msi_domain_free_irqs_all) was mistakenly placed after
msi_domain_free_irqs_range() instead of after its intended function
msi_domain_free_irqs_all().
Signed-off-by: Li RongQing <lirongqing@baidu.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Radu Rendec <radu@rendec.net>
Link: https://patch.msgid.link/20260717143157.1718-1-lirongqing@baidu.com
|
|
Two unrelated things go by "cgroup" in the cid form. Sub-schedulers attach
to cgroups, and the cgroup_*() ops deliver cpu controller events. While the
ops names suggest cgroup2 hierarchy, they actually operate on the cpu
controller.
Rename them to cpuctl_* in struct sched_ext_ops_cid, which has no users
outside scx_qmap yet. The cpu form is deployed ABI and keeps the old names.
The layout is unchanged and the kernel keeps calling through the cpu-form
union view.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Sub-schedulers don't get cgroups yet: every task_group is inited on the root
sched and the routing added by the previous patches always resolves to it.
Add the handover: an enabling sub-scheduler takes over the cgroups in its
subtree and a disabling one returns them to its parent.
scx_cgroup_claim_subtree() runs while the sub enables, after the subtree's
cgrp->scx_sched's are set and before any task is claimed. It inits each
subtree task_group on the sub, exits it from the parent and updates
tg->scx.sched. A failed ops.cgroup_init() unwinds the sub-side inits and
aborts the enable with the parent untouched.
Disabling reverses it with scx_cgroup_return_subtree(): exit each cgroup
from the sub, then re-init it on the parent with the current tg->scx.*
values, resyncing weight and bandwidth changes made while the sub had it.
When a re-init fails, the parent is failed and the remaining task_groups
still transfer uninited and get no cgroup ops - the same punting done for
tasks. The dying parent's own disable moves them onward.
The handover walks include dying but not yet offlined task_groups, the same
as root's bulk walks: a removed cgroup keeps hosting scheduling events until
its dying tasks finish their final context switches, and its
ops.cgroup_exit() must follow the last of them. tg on/offlining is excluded
through cgroup_lock(), so either ordering against an rmdir of a subtree
cgroup delivers balanced init/exit pairs.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
With sub-schedulers claiming cgroup subtrees, cgroup ops must be delivered
to each task_group's sched rather than always to root. Add tg->scx.sched to
track which sched initialized the task_group. It is set and cleared together
with SCX_TG_INITED.
Deliver the ops accordingly:
- ops.cgroup_exit() goes to the sched whose ops.cgroup_init() it pairs with.
- ops.cgroup_prep_move/move/cancel_move() go to the task's sched, and only
for moves that don't re-home the task. A re-homing move is reported
through the ops.exit_task/init_task() pair instead. The cgroups passed to
the move ops can be outside the sched's inited set as the cpu controller
can be coarser than the sub-scheduler topology.
- Knobs of a cgroup belong to the parent, so ops.set_weight/idle/bandwidth()
go to the parent task_group's sched.
All task_groups currently resolve to the root sched, so no behavior changes
until sub-schedulers start claiming cgroups.
While at it, scx_cgroup_init() is restructured so both paths share the
recording.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A task's sched (p->scx.sched) must match its cgroup's owner
(cgrp->scx_sched). cgroup migration breaks the invariant:
scx_cgroup_move_task() only fires root's ops.cgroup_move() and never
re-homes the task, leading to wrong-sched scheduling and, once the stale
sched is freed, a use-after-free.
Hook into the new cgroup task migration events and re-home each task whose
destination cgroup is owned by a different sched. The events map naturally
to the transfer: MIGRATING runs the fallible init for the destination sched,
letting it reject the migration the same way ops.cgroup_prep_move() can,
MIGRATED does the re-home, which can't fail, and CANCELED undoes the init
when the migration falls through.
Pre-commit, the task's task_group still reflects the source, so
__scx_init_task() grows an explicit cgroup argument for the migration path
to hand ops.init_task() the destination cgroup.
Signed-off-by: Tejun Heo <tj@kernel.org>
Closes: https://lore.kernel.org/r/alnxrsexEe_nQwqL@gpd4
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_cgroup_enabled is in the CONFIG_EXT_GROUP_SCHED block. The upcoming
cgroup migration re-homing needs the gate outside the block. Move the
definition and flag flips outside CONFIG_EXT_GROUP_SCHED. No functional
changes.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Factor out scx_rehome_task() and scx_punt_task() from the sub-disable
re-home loop and scx_fail_parent(). The upcoming cgroup migration re-homing
also needs scx_rehome_task(). No functional changes.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A subsystem can attach to the cgroup hierarchy itself, independent of which
controllers are enabled where - BPF hooks already behave this way and
sched_ext sub-schedulers do too. Controller callbacks can't track task
migrations for them: sched_ext must re-home a task whose migration crosses a
sub-scheduler boundary, but the cpu controller's attach callbacks fire only
when the task_group changes and miss moves whenever the controller topology
is coarser than the sub-scheduler topology.
Add cgroup_task_notifier with per-task migration events mirroring the
can_attach/attach/cancel_attach phases so that a consumer which prepares
per-task state can also veto a migration: CGROUP_TASK_MIGRATING fires
pre-commit, CGROUP_TASK_MIGRATED post-commit and
CGROUP_TASK_MIGRATE_CANCELED unwinds a failed migration. Only migrations
that change a task's dfl cgroup are reported.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The BPF_MOV64_PERCPU_REG insn requires JIT to emit native code to for
'dst_reg = src_reg + <percpu_base_off>'.
However, the interpreter ignores the 'off' at its ALU64_MOV_X label.
The 'off' indicates the insn is BPF_MOV64_PERCPU_REG insn. Then, when
the interpreter loads memory from the register, it will hit a page
fault.
[ 2.545572] BUG: unable to handle page fault for address: ffffffffacaaf034
[ 2.546485] #PF: supervisor read access in kernel mode
[ 2.547167] #PF: error_code(0x0000) - not-present page
[ 2.547850] PGD 134e63067 P4D 134e63067 PUD 134e64063 PMD 10021c063 PTE 800ffffeca550062
[ 2.548912] Oops: Oops: 0000 [#1] SMP PTI
Set jit_required as true in order to disallow interpreter fallback in
core.c::__bpf_prog_select_runtime(), if any BPF_ADDR_PERCPU insn is
patched to the prog.
BTW, rename the helper bpf_map_supports_cpu_flags() to
bpf_map_is_percpu_map().
Fixes: 7bdbf7446305 ("bpf: add special internal-only MOV instruction to resolve per-CPU addrs")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260715141122.15783-4-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
The interpreter does not recognize the BPF_JMP|BPF_JA|BPF_X insn, which
is used for insn_array map. Thereafter, it would hit the BUG_ON() in
___bpf_prog_run() at run time.
[ 2.563726] BPF interpreter: unknown opcode 0d (imm: 0x0)
[ 2.564557] ------------[ cut here ]------------
[ 2.565206] kernel BUG at kernel/bpf/core.c:2349!
[ 2.565882] Oops: invalid opcode: 0000 [#1] SMP PTI
Set jit_required as true when insn_array map is used in the prog in
order to disallow interpreter fallback for gotox insn in
core.c::__bpf_prog_select_runtime().
Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260715141122.15783-3-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Since the interpreter does not support the arena-related insns,
interpreter fallback should not be allowed for these insns in
core.c::__bpf_prog_select_runtime().
Currently, when the interpreter executes the arena ST/LDX/STX insns,
it would hit the BUG_ON() in ___bpf_prog_run() at run time.
[ 2.579196] BPF interpreter: unknown opcode a2 (imm: 0x0)
[ 2.579998] ------------[ cut here ]------------
[ 2.580652] kernel BUG at kernel/bpf/core.c:2349!
[ 2.581314] Oops: invalid opcode: 0000 [#1] SMP PTI
Set jit_required as true when arena map is used in the prog to disallow
interpreter fallback for arena-related insns.
Fixes: 6082b6c328b5 ("bpf: Recognize addr_space_cast instruction in the verifier.")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260715141122.15783-2-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
bpf_arena_free_pages() accepts scalar arena addresses. The runtime
masks the address to the low 32 bits and reconstructs a full user
address from the arena base before returning the range to the arena
free tree.
When the scalar value is below the low 32 bits of the arena base,
full_uaddr falls below user_vm_start. The existing upper-end clipping
then turns this into an out-of-range free-tree offset. A later
allocation can reuse that offset and return an address below the arena
mapping.
Reject such frees before computing the clipped range.
Fixes: 317460317a02a ("bpf: Introduce bpf_arena.")
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260717-c10-031-public-bpf-next-v2-b4-v2-1-54b555443a7c@mails.tsinghua.edu.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
arena is the only map type whose map_mem_usage() still returns 0, so
"bpftool map show" and fdinfo always showed 0 memlock for an arena no
matter how many pages it had.
Count the pages that are actually mapped into the arena: bump a counter in
apply_range_set_cb() when a page goes in and drop it in
apply_range_clear_cb() when a page goes out, both under the arena spinlock.
map_mem_usage() then just returns nr_pages << PAGE_SHIFT.
Only real data pages are counted, not the scratch page.
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260717114117.350851-3-jiayuan.chen@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Replace the scratch_page field in the pte-callback data with the arena
pointer; later patches use other arena fields from these callbacks. No
functional change.
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260717114117.350851-2-jiayuan.chen@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Pull to receive:
477869bfafea ("sched_ext: Reject setting disallow from init_task outside the enable path")
5f8b69642d18 ("sched_ext: Take cgroup_lock() first in scx_cgroup_lock()")
8c13364db9c9 ("sched_ext: Skip sub-disable teardown for never-linked sub-schedulers")
5cdc92859809 ("sched_ext: Don't enable non-ext tasks in the sub-sched task loops")
as dependencies for the upcoming cgroup migration patchset and to
resolve the conflicts with the ext.c/sub.c split on for-7.3.
5f8b69642d18 comments scx_cgroup_lock() which for-7.3 exported for
sub.c. Resolved by keeping the exported version with the comment.
8c13364db9c9 and 5cdc92859809 patch the pre-split sub-sched enable and
disable paths in ext.c which for-7.3 moved to sub.c. Resolved by
applying the never-linked teardown skip and the class gates to sub.c.
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Root enable and scx_post_fork() enable a task only if it's on the ext class.
Tasks on other classes, possible under an SCX_OPS_SWITCH_PARTIAL root, are
left READY and enabled by switching_to_scx() when they switch over. The sub
enable-commit pass and the sub-disable re-home loop enable unconditionally,
so a fair-class READY task in the subtree becomes ENABLED while not on
sched_ext. A later switch to SCHED_EXT then trips the task state validation
WARN (ENABLED with the previous state not READY) and calls ops.enable() a
second time.
Gate scx_enable_task() on the task's class in both loops.
Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A sub-scheduler enable can fail before scx_link_sched() links the sched into
the hierarchy, e.g. when the parent is already being disabled, and cleanup
still runs the full scx_sub_disable().
That is racy against root disable: drain_descendants() is the only ordering
between a sub's disable-time task walk and root disable's all-task teardown,
and an unlinked sub is invisible to it. Root's teardown can thus run between
the never-linked sub's drain and its walk, exiting every task to no
scheduler.
The walk then trips the membership WARN and re-homes the exited tasks onto
the dying hierarchy, a use-after-free.
Skip the cgroup ownership reset and the task walk if @sch was never linked,
indicated by the empty ->sibling as unlinking only happens later in the same
function. The membership WARN remains valid: a linked sub is always waited
on by an ancestor's drain.
Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_cgroup_lock() write-locks scx_cgroup_ops_rwsem and then takes
cgroup_lock(), which can deadlock through kernfs:
scx enable/disable cgroup rmdir cpu.weight write
------------------ ------------ ----------------
cgroup_lock()
percpu_down_write(rwsem)
cgroup_lock()
kernfs_get_active()
percpu_down_read(rwsem)
kernfs_drain()
The enable path waits for the rmdir to release cgroup_mutex. The rmdir,
deactivating the cpu controller's files, waits in kernfs_drain() for the
write's active reference. The write, in scx_group_set_weight(), waits for
the rwsem behind the pending writer.
Take cgroup_lock() first. The set_* paths take no cgroup locks inside the
read side, so a pending write-lock then only waits for read sections that
always run to completion, and no dependency from the rwsem back to
cgroup_mutex remains.
Fixes: a5bd6ba30b33 ("sched_ext: Use cgroup_lock/unlock() to synchronize against cgroup operations")
Cc: stable@vger.kernel.org # v6.18+
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The p->scx.disallow revert assumes the root enable path, where the switching
loop reads the reverted policy right afterwards and leaves the task off SCX.
The sub-scheduler disable path also reaches it when re-initializing the
returned tasks on a root parent. Nothing reads the policy there: the task is
enabled on root anyway and keeps running on the ext class with a silently
rewritten policy.
Kill the sched instead, matching the fork and non-root branches, and update
the disallow documentation, which equated !fork with the load path and
pointed at a stale debugfs path for nr_rejected.
Fixes: 337ec00b1d9c ("sched_ext: Implement cgroup sub-sched enabling and disabling")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Cross-merge BPF and other fixes after downstream PR.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
check_kfunc_call() reads meta.func_name when bpf_fetch_kfunc_arg_meta()
returns -EACCES, but that error can come from fetch_kfunc_meta() (e.g.
fd_array_get_btf() rejecting BTF binding for a signed program) before
meta is memset(), leaving it uninitialized and risking a garbage deref
in verbose().
Move the memset() to the start of bpf_fetch_kfunc_arg_meta() so meta is
zeroed on every error return. The intended "not allowed" -EACCES path
still sets func_name first, so its message is unchanged.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260715172127.2416388-3-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
For a bloom filter, the value argument of bpf_map_peek_elem() is always
an input. Therefore, the verifier should not allow passing uninitialized
stack memory to it to avoid information leak.
bpf_map_peek_elem() tags its value argument ARG_PTR_TO_MAP_VALUE |
MEM_UNINIT, telling the verifier the callee fills the buffer. This holds
for queue/stack maps, but not for a bloom filter, which reads the buffer
as an input to test set membership and never writes it.
As a result, a program can pass an uninitialized stack buffer to
bpf_map_peek_elem() on a bloom filter. The verifier accepts it and marks
the buffer initialized on return, letting the program read back leftover
kernel stack memory. Bloom maps require CAP_BPF to create, so this is a
CAP_BPF-gated stack infoleak that bypasses the boundary CAP_BPF is meant
to enforce (arbitrary kernel reads are gated behind CAP_PERFMON).
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260715172127.2416388-2-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Pull bpf fixes from Kumar Kartikeya Dwivedi:
- Fix a UAF in socket clone early bailout paths (Matt Bobrowski)
- Reject unhashed UDP sockets on sockmap update to prevent refcount
leaks (Michal Luczaj)
- Account for receive queue data in FIONREAD on sockmap sockets without
a verdict program (Mattia Meleleo)
- Reject negative constant offsets for verifier buffer pointers (Sun
Jian)
- Fix for tracing of kfuncs with implicit arguments (Ihor Solodrai)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
selftests/bpf: Cover tracing implicit kfunc args
bpf: Fix tracing of kfuncs with implicit args
selftests/bpf: Cover negative buffer pointer offsets
bpf: Reject negative const offsets for buffer pointers
selftests/bpf: Test FIONREAD on a sockmap socket without a verdict program
bpf, sockmap: Account for receive queue in FIONREAD without a verdict program
selftests/bpf: Fail unbound UDP on sockmap update
selftests/bpf: Adapt sockmap update error handling
bpf, sockmap: Reject unhashed UDP sockets on sockmap update
selftests/bpf: Ensure UDP sockets are bound
bpf: Fix UAF in sock clone early bailouts
|
