linux.git/kernel/bpf, branch v7.2-rc7

rqspinlock: Reset tail when preserving queue on deadlock

2026-08-06T23:32:42+00:00

Currently, the destruction of the waiter queue is suppressed for
rqspinlock in cases where a deadlock is detected. Deadlock checks happen
relatively frequently (on entry for AA, within 1ms for ABBA), and waiter
threads may not be involved in locking scenarios involving deadlocks.
Thus, it is useful to not flush the queue and let other waiters take a
stab at acquiring the lock after we detect a deadlock and exit.

However, we need to follow the same logic as what we did previously for
the waitq_timeout label: reset the tail, and if we cannot, signal the
next waiter appropriately. In case of deadlocks, this signal would just
mark the MCS node as unlocked, and in case of timeouts, it would signal
RES_TIMEOUT_VAL. The difference thus is in the value propagated, which
decides whether the queue remains active or gets flushed.

Not doing the tail reset, and waiting for the next waiter can lead to
cases where we are the final waiter, and thus no next waiter arrives,
leading to intermittent stalls in this path. Once the next waiter does
join, we will be unblocked. In the theoretical case when the next waiter
never joins, we risk stalling indefinitely.

This can only happen for ABBA deadlocks, since entry into the wait queue
is guarded with AA checks. A precise sequence of executions leading up
to this scenario can be:

CPU 0 holds lock A.
CPU 1 holds lock B.
CPU 2 attempts lock B, becomes the pending waiter for B.
CPU 0 attempts lock B. B has locked+pending bits set, thus CPU 0 queues.
CPU 1 attempts lock A.
CPU 0 detects an ABBA deadlock.

Once deadlock detection happens for CPU 0, it will sit waiting for the
next waiter in the queue to populate node->next, which will experience
delays until such a waiter arrives.

Fix this by adjusting the logic for the check for deadlocks preceding
the waitq_timeout label. It would make sense to consolidate code for
both cases and use 'ret' to distinguish the value being propagated, but
that is left as an exercise for a future refactoring task to avoid diff
noise in this patch.

Fixes: 7bd6e5ce5be6 ("rqspinlock: Disable queue destruction for deadlocks")
Signed-off-by: Kumar Kartikeya Dwivedi 
Link: https://patch.msgid.link/20260802021759.1139457-1-memxor@gmail.com
Signed-off-by: Eduard Zingerman

bpf: Propagate untrusted pointer state in commuted arithmetic

2026-07-31T19:45:28+00:00

The untrusted PTR_TO_MEM early return skips pointer offset tracking
because accesses go through probe-read handling. Moving it after full
pointer-state propagation ensures scalar += untrusted_pointer leaves the
destination as PTR_TO_MEM instead of an unrelated scalar.

Fixes: f2362a57aeff ("bpf: allow void* cast using bpf_rdonly_cast()")
Signed-off-by: Yiyang Chen 
Tested-by: Daniel Wade 
Link: https://patch.msgid.link/20260729-c3-035-public-bpf-v4-v4-3-8ee297e2346b@mails.tsinghua.edu.cn
Signed-off-by: Eduard Zingerman

bpf: Preserve pointer state for commuted arithmetic

2026-07-31T19:45:21+00:00

When scalar += pointer is handled in adjust_ptr_min_max_vals(), the
destination register inherits the pointer state from the source pointer.
Copying only selected fields is fragile because pointer provenance is
tracked by several bpf_reg_state fields.

Use the caller's temporary offset register to preserve the scalar operand
while replacing the destination with the full pointer state. This preserves
the frame number for PTR_TO_STACK registers and keeps parent identity
fields consistent.

Fixes: f4d7e40a5b71 ("bpf: introduce function calls (verification)")
Signed-off-by: Yiyang Chen 
Tested-by: Daniel Wade 
Acked-by: Shung-Hsi Yu 
Link: https://patch.msgid.link/20260729-c3-035-public-bpf-v4-v4-2-8ee297e2346b@mails.tsinghua.edu.cn
Signed-off-by: Eduard Zingerman

bpf: Simplify sanitize_err() signature

2026-07-31T19:44:06+00:00

The sanitize_err() function is called when:
- ptr += scalar
- scalar += ptr
- scalar += scalar
ALU operations are processed.

This commit drops offset and pointer registers parameters from its
signature to simplify the follow-up changes for 'scalar += ptr' case.
regs[src].type is safe to access, as it is not mutated by the callers.

Signed-off-by: Yiyang Chen 
Acked-by: Shung-Hsi Yu 
Link: https://patch.msgid.link/20260729-c3-035-public-bpf-v4-v4-1-8ee297e2346b@mails.tsinghua.edu.cn
Signed-off-by: Eduard Zingerman

bpf: Reject passing scalar NULL to nonnull arg of a global subprog

2026-07-23T23:09:07+00:00

A global subprogram argument tagged __arg_nonnull is set up as a
non-nullable PTR_TO_MEM. However the verifier does not check against a
scalar NULL, leading to real NULL pointer dereference. Reject it as
well.

Fixes: 94e1c70a3452 ("bpf: support 'arg:xxx' btf_decl_tag-based hints for global subprog args")
Signed-off-by: Amery Hung 
Acked-by: Eduard Zingerman 
Link: https://patch.msgid.link/20260723221815.367797-1-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman

bpf: Fix tracing of kfuncs with implicit args

2026-07-16T23:11:29+00:00

A kfunc marked with KF_IMPLICIT_ARGS flag takes implicit arguments
(such as bpf_prog_aux) that the verifier injects at load time.
resolve_btfids strips those from the kfunc's BTF-visible prototype and
keeps the real kernel ABI in a counterpart _impl prototype [1].

fentry/fexit/fmod_ret/fsession programs may attach to the BPF kernel
functions, including those with implicit args. However
bpf_check_attach_target() and bpf_check_attach_btf_id_multi() extract
the struct btf_func_model from the wrong BTF prototype of the
kfunc. The btf_func_model is later read to construct the trampoline,
which then causes the injected implicit argument to be clobbered and
the kfunc dereferencing garbage.

Add btf_attach_func_proto() to resolve the real ABI prototype of the
kfunc the way the call site does: by looking up the _impl prototype
for a KF_IMPLICIT_ARGS kfunc. Use it at both attach-target model
construction sites.

To enable this, make two supporting changes:
  * pass bpf_verifier_log instead of bpf_verifier_env to
    find_kfunc_impl_proto(), so it can be reused from the attach path
  * add btf_kfunc_check_flag() to test a flag across all of a kfunc's
    hook sets, because a program attaching to a kfunc is not in the
    kfunc's call-set

KF_IMPLICIT_ARGS must be consistent across the sets, so
btf_kfunc_check_flag() returns -EINVAL on inconsistency.

btf_kfunc_check_flag() reads the kfunc's flags from the target's
kfunc_set_tab. For a module BTF that table is stable only after the
module is live, so take a module reference around the read, mirroring
how the kfunc call path gates the same lookup with btf_try_get_module().

The remaining call sites of btf_distill_func_proto() are safe as
is. The BPF_TRACE_ITER case distills a registered iterator's
prototype, and bpf_struct_ops_desc_init() distills the
function-pointer members of a struct_ops type. Neither is a kfunc, and
so can't have implicit arguments.

[1] https://lore.kernel.org/all/20260120222638.3976562-1-ihor.solodrai@linux.dev/

Fixes: 64e1360524b9 ("bpf: Verifier support for KF_IMPLICIT_ARGS")
Reported-by: Tejun Heo 
Signed-off-by: Ihor Solodrai 
Link: https://github.com/sched-ext/scx/issues/3687#issuecomment-4906694106
Link: https://patch.msgid.link/20260713235223.1639022-2-ihor.solodrai@linux.dev
Signed-off-by: Eduard Zingerman

bpf: Reject negative const offsets for buffer pointers

2026-07-15T09:32:42+00:00

The verifier rejects variable offsets for PTR_TO_TP_BUFFER and PTR_TO_BUF
accesses, but it currently accepts a constant negative offset produced by
pointer arithmetic.

Commit 022ac0750883 ("bpf: use reg->var_off instead of reg->off for
pointers") moved constant pointer offsets from reg->off to reg->var_off.
However, __check_buffer_access() continued to check only the instruction
offset. An access with reg->var_off equal to -8 and an instruction offset
of zero therefore passes verification.

For writable raw tracepoints, the access end is also calculated from the
unsigned reg->var_off.value. An eight-byte access starting at -8 wraps
the calculated end to zero, allowing the program to load and attach
without increasing max_tp_access.

After ensuring that reg->var_off is constant, calculate the effective
access start using signed arithmetic and reject it when it is negative.
Use the validated start to calculate the access end for both
PTR_TO_TP_BUFFER and PTR_TO_BUF.

Fixes: 022ac0750883 ("bpf: use reg->var_off instead of reg->off for pointers")
Signed-off-by: Sun Jian 
Acked-by: Shung-Hsi Yu 
Cc: stable@vger.kernel.org # 5.2.0
Link: https://patch.msgid.link/20260714093846.18159-2-sun.jian.kdev@gmail.com
Signed-off-by: Eduard Zingerman

bpf: Prefer dirty packs for eBPF allocations

2026-07-01T08:34:41+00:00

The pack allocator only flushes predictors when reusing a dirty pack for
cBPF, eBPF allocations never trigger a flush. Currently, eBPF picks the
first free pack, which could be a clean pack. As an optimization, leaving
a clean pack for cBPF can avoid flushes.

Prefer dirty packs for eBPF and keep clean packs free for cBPF. This
mirrors the existing cBPF preference for clean packs: each program kind
prefers the pack that avoids an extra flush, and falls back to the other
kind only when no preferred pack has room. eBPF reuse of a dirty pack is
harmless since eBPF being privileged does not flush.

Signed-off-by: Pawan Gupta 
Acked-by: Daniel Borkmann 
Signed-off-by: Daniel Borkmann

bpf: Prefer packs that won't trigger an IBPB flush on allocation

2026-07-01T08:33:45+00:00

Currently BPF pack allocator picks the chunks from the first available
pack. While this is okay, it naturally leads to more frequent flushes
when there are multiple packs in the system that weren't used since the
last flush.

As an optimization prefer allocating the new programs from packs that
are unused since last flush. When all packs are dirty, allocation forces
a flush and marks all packs clean.

Below are some future optimizations ideas:

  1. Currently, the "dirty" tracking is only done at the pack-level.
     Flush frequency can further be reduced with chunk-level tracking.
     This requires a new bitmap per-pack to track the dirty state.
  2. IBPB flush is done on all CPUs, even if only a single CPU ran the
     BPF program. On a system with hundreds of CPUs this could be a
     major bottleneck forcing hundreds of IPIs to deliver the flush.
     The solution is to track the CPUs where a BPF program ran, and
     issue IBPB only on those CPUs.
  3. Avoid IBPB when flush is already done at other sources (e.g.
     context switch).

Signed-off-by: Pawan Gupta 
Acked-by: Daniel Borkmann 
Signed-off-by: Daniel Borkmann

bpf: Skip redundant IBPB in pack allocator

2026-07-01T08:33:40+00:00

bpf_prog_pack_alloc() issues IBPB on all CPUs on every cBPF allocation,
even when reusing chunks from an existing pack where no new memory was
touched since the last IBPB.

Since IBPB on all CPUs is heavy, Dave Hansen suggested to track allocation
since last IBPB, and only issue IBPB at reuse for the chunks that have not
seen an IBPB since they were last freed.

Track per-pack whether an IBPB is needed via arch_flush_needed. Set it when
allocating a chunk, reset on IBPB flush. On reuse, conditionally issue the
flush. Since IBPB invalidates all BTB entries, clear the flag on all packs
after flushing.

Signed-off-by: Pawan Gupta 
Acked-by: Daniel Borkmann 
Signed-off-by: Daniel Borkmann