linux-stable.git/kernel/bpf/stackmap.c, branch v7.3-rc2

bpf: Mark faultable stack helpers as sleepable

2026-09-04T02:22:52+00:00

The faultable variants of bpf_get_stack() and bpf_get_task_stack() pass
may_fault=true into the common stack collection code. Resolving user-space
build IDs may then call build_id_parse_file() and block on filesystem
reads.

Neither helper prototype sets might_sleep. Since prototype selection uses
the sleepability of the whole program, the verifier can still allow these
helpers from a non-sleepable region within that program, such as an
explicit RCU or preemption-disabled region. The task-stack helper can also
be called from a non-sleepable timer callback of a sleepable program.

Mark both faultable prototypes as sleepable. The existing helper context
check then rejects these calls while continuing to allow them in genuinely
sleepable contexts.

Fixes: d4dd9775ec24 ("bpf: wire up sleepable bpf_get_stack() and bpf_get_task_stack() helpers")
Signed-off-by: Kumar Kartikeya Dwivedi 
Acked-by: Eduard Zingerman 
Link: https://lore.kernel.org/r/20260903214758.2727663-6-memxor@gmail.com
Signed-off-by: Alexei Starovoitov

bpf: Fix mmap_lock leak in irq_work path

2026-08-08T08:25:36+00:00

stack_map_get_build_id_offset() introduced a per-CPU irq_work to defer
mmap_read_unlock() from NMI context, and bpf_find_vma() later reused the
same mmap_unlock_work. Both callers only check whether the work is busy
before taking mmap_lock, so a nested caller can reuse the slot before the
first caller queues it. Two read locks may then be acquired while only one
deferred unlock runs, leaking a read lock and blocking exit_mmap().

Reserve the per-CPU slot before mmap_read_trylock(). Use the same wrapper
in stackmap and bpf_find_vma() so both callers release the reservation on
trylock failure. Keep rejecting the slot while the irq_work remains busy.
Release it after the irq_work callback unlocks the mm.

Fixes: eac9153f2b58 ("bpf/stackmap: Fix deadlock with rq_lock in bpf_get_stack()")
Reported-by: syzbot+cdd6c0925e12b0af60cc@syzkaller.appspotmail.com
Reported-by: sashiko-bot@kernel.org
Signed-off-by: Sanghyun Park 
Signed-off-by: Andrii Nakryiko 
Signed-off-by: Daniel Borkmann 
Closes: https://syzkaller.appspot.com/bug?extid=cdd6c0925e12b0af60cc
Closes: https://lore.kernel.org/r/20260630033745.B80201F000E9@smtp.kernel.org
Link: https://lore.kernel.org/bpf/20260805031425.2157475-2-sanghyun.park.cnu@gmail.com

bpf: Avoid changing callchain in bpf_get_stackid_pe

2026-08-05T18:34:46+00:00

There's no need to modify the trace object bpf_get_stackid_pe, we just
need to pass the needed callchain length in separate argument.

This way we can have callchain pointers const and remove the trace->nr
modification and restoration.

Assisted-by: Codex:GPT-5.5

Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20260803210149.296496-13-jolsa@kernel.org

bpf: Avoid changing callchain in bpf_get_stack_pe

2026-08-05T18:34:45+00:00

There's no need to modify the trace object bpf_get_stack_pe, we just
need to pass the needed callchain length in separate argument.

This way we can have callchain pointers const and remove the trace->nr
modification and restoration.

Assisted-by: Codex:GPT-5.5

Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20260803210149.296496-12-jolsa@kernel.org

bpf: Disable preemption in __bpf_get_stack

2026-08-05T18:34:44+00:00

get_perf_callchain() returns a per-CPU perf_callchain_entry buffer and
releases its recursion slot via put_callchain_entry() before returning,
so nothing keeps the entry reserved while __bpf_get_stack() consumes
it below.

A preemptible BPF program (e.g. a non-sleepable raw tracepoint program
on a PREEMPT kernel, which runs under migrate_disable() but not
preempt_disable()) can be scheduled out between obtaining the entry
and the copy. Another task scheduled on the same CPU then reuses the
same per-CPU buffer and overwrites trace->nr with a larger value.
copy_len is then computed from the inflated trace->nr and can exceed
the caller's buffer, causing an out-of-bounds write in the memcpy()
and in the build_id path.

The rcu_read_lock() taken here alone does not prevent this. It is
only taken on the may_fault path, and under CONFIG_PREEMPT_RCU it does
not disable preemption; it merely keeps perf's callchain buffer array
alive (freed via call_rcu()) and does nothing to stop another task
from reusing the entry.

Disable preemption around obtaining the callchain entry and copying
it into the caller's buffer, so the entry cannot be reused underneath
us and trace->nr stays bounded by max_depth. Build ID resolution may
fault and is therefore deferred until after preemption is re-enabled;
by then the instruction pointers have already been copied into buf,
so it operates only on that private copy. Note, preempt_disable() also
subsumes the buffer-lifetime guarantee the rcu_read_lock() provided,
since a preempt-disabled section is an RCU read-side critical section
for the callchain buffers' call_rcu() reclaim.

Fixes: c195651e565a ("bpf: add bpf_get_stack helper")
Reported-by: Tao Chen 
Reported-by: STAR Labs SG 
Signed-off-by: Daniel Borkmann 
Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/20260803210149.296496-11-jolsa@kernel.org

Closes: https://lore.kernel.org/bpf/20260206090653.1336687-1-chen.dylane@linux.dev/
[ changed Fixes: commit ]

bpf: Clear buf on error in __bpf_get_task_stack

2026-08-05T18:32:29+00:00

Both bpf_get_task_stack and bpf_get_task_stack_sleepable helpers that
use __bpf_get_task_stack have buf defined as ARG_PTR_TO_UNINIT_MEM
argument and we should initialize the buf on every return path.

Adding missing buf memset for __bpf_get_task_stack fail paths. This
provides deterministic buffer contents, which is useful when the buffer
is used directly as a map key.

Fixes: 06ab134ce8ec ("bpf: Refcount task stack in bpf_get_task_stack")
Fixes: b992f01e6615 ("bpf: Guard against accessing NULL pt_regs in bpf_get_task_stack()")
Reported-by: Sashiko 
Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20260803210149.296496-10-jolsa@kernel.org

bpf: Remove trace_in argument from __bpf_get_stack

2026-08-05T18:32:29+00:00

Now with the new callchain_* helper functions we can process trace_in
case directly in bpf_get_stack_pe function and remove it from
__bpf_get_stack which makes things easier for preemption fix in
following change.

Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20260803210149.296496-9-jolsa@kernel.org

bpf: Factor callchain_finalize function from __bpf_get_stack

2026-08-05T18:32:29+00:00

The new callchain_finalize function calls the build-id retrieval
(if needed) and zeroes the buffer. This makes things easier for
preemption fix in following change.

Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20260803210149.296496-8-jolsa@kernel.org

bpf: Factor callchain_store function from __bpf_get_stack

2026-08-05T18:32:29+00:00

The new callchain_store function stores trace entries buffer into
user supplied buffer. It covers both just-ip and buildid data.

Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20260803210149.296496-7-jolsa@kernel.org

bpf: Disable preemption in bpf_get_stackid

2026-08-05T18:32:29+00:00

The get_perf_callchain call needs disabled preemption plus we need
it disabled as long as we access its returned trace entries buffer.

Note the bpf_get_stackid_pe function is executed already with
preemption disabled.

Fixes: d5a3b1f69186 ("bpf: introduce BPF_MAP_TYPE_STACK_TRACE")
Reported-by: Tao Chen 
Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/20260803210149.296496-6-jolsa@kernel.org

Closes: https://lore.kernel.org/bpf/20260206090653.1336687-2-chen.dylane@linux.dev/