linux-stable.git/kernel/bpf, branch v7.2.5

bpf: Disable preemption in bpf_get_stackid

2026-09-11T09:51:25+00:00

[ Upstream commit 15f1bd8574662f1b7b26aaa2e23ebf4066f0117d ]

The get_perf_callchain call needs disabled preemption plus we need
it disabled as long as we access its returned trace entries buffer.

Note the bpf_get_stackid_pe function is executed already with
preemption disabled.

Fixes: d5a3b1f69186 ("bpf: introduce BPF_MAP_TYPE_STACK_TRACE")
Reported-by: Tao Chen 
Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/20260803210149.296496-6-jolsa@kernel.org

Closes: https://lore.kernel.org/bpf/20260206090653.1336687-2-chen.dylane@linux.dev/
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

bpf: Use stack id functions instead of __bpf_get_stackid

2026-09-11T09:51:25+00:00

[ Upstream commit 09b3fd6caa0b57f8a39254ee5db3af30bdd53c18 ]

Replacing __bpf_get_stackid calls with sequence of following functions:

  stackid_fastpath
  stackid_new_bucket
  stackid_install

This makes code more structured and allows us to easily disable
preemption only in bpf_get_stackid in following changes.

Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20260803210149.296496-5-jolsa@kernel.org
Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid")
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

bpf: Factor stackid_new_bucket from __bpf_get_stackid

2026-09-11T09:51:25+00:00

[ Upstream commit bb4e6f4e1b68fe60c04ca04c564c6624e837dbf4 ]

The new stackid_new_bucket allocates the new bucket and initializes it
with the trace data.

Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20260803210149.296496-4-jolsa@kernel.org
Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid")
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

bpf: Factor stackid_fastpath function from __bpf_get_stackid

2026-09-11T09:51:24+00:00

[ Upstream commit 0ca56befcffec3a6c9d1842eae06c74e1cf41f11 ]

The new stackid_fastpath does the fast stack hash and trace check, that
does not need new bucket allocation. It covers both just-ip and buildid
code paths.

Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20260803210149.296496-3-jolsa@kernel.org
Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid")
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

bpf: Factor stackid_init function from __bpf_get_stackid

2026-09-11T09:51:24+00:00

[ Upstream commit 15b837759a97237d647962f9943afe0d55af615a ]

The new stackid_init function stores all the necessary bits for stackid
trace and it will be used by other functions in following changes.

Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20260803210149.296496-2-jolsa@kernel.org
Stable-dep-of: 15f1bd857466 ("bpf: Disable preemption in bpf_get_stackid")
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

bpf: Fix infinite loop in pcpu_freelist push with one possible CPU

2026-09-11T09:50:52+00:00

commit efebf6496685c93150df5bb0794363ae70c5f58a upstream.

__pcpu_freelist_push() can loop forever when only one CPU is possible
and an NMI re-enters pcpu_freelist_push() while the interrupted context
holds that CPU's freelist lock.

After the current-CPU fast path fails, the fallback loop walks
cpu_possible_mask while skipping the current CPU. With CONFIG_SMP=n, or
when an SMP kernel is limited to one possible CPU with nr_cpus=1 or
possible_cpus=1, there are no other possible CPUs to examine. The loop
therefore makes no lock acquisition attempt and can never make progress.

The following stack was observed on a UP system:

  NMI context:
    pcpu_freelist_push
    free_htab_elem
    htab_map_delete_elem
    [perf-event BPF program]
    __perf_event_overflow
    perf_event_nmi_handler
    exc_nmi

  Interrupted context:
    __pcpu_freelist_push
    pcpu_freelist_push
    free_htab_elem
    htab_map_delete_elem
    [raw_tp/sys_enter BPF program]
    __bpf_trace_sys_enter
    do_syscall_64

raw_res_spin_lock() detects the same-CPU recursive acquisition and
returns -EDEADLK, but the subsequent fallback loop has no candidate head
on a system with one possible CPU.

Restore the extra fallback head that existed before the rqspinlock
conversion. Keep the current-CPU fast path, then try the other possible
CPUs and finally the extra head. The additional head lets a push, which
cannot fail without losing a preallocated element, make progress when the
only per-CPU head is held by the interrupted context.

Also check the extra head from the pop path so that nodes placed there
can be reused.

Fixes: f2ac0e5d1c4d ("bpf: Convert percpu_freelist.c to rqspinlock")
Signed-off-by: Hui Su 
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/20260806175600.1993595-1-sh_def@163.com
Signed-off-by: Kumar Kartikeya Dwivedi 
Signed-off-by: Greg Kroah-Hartman

bpf: Harden bloom filter sizing and indexing on 32-bit kernels

2026-09-07T15:36:56+00:00

commit 11c1e836710dcba03e50454a4eedfdbaf8d3050e upstream.

bloom_map_alloc() has two 32-bit-specific problems when the computed
bitmap reaches the U32_MAX fallback case.

First, BITS_TO_BYTES(U32_MAX) is evaluated with 32-bit arithmetic. The
addition performed by DIV_ROUND_UP wraps, so the map allocates only the
fixed-size bloom filter object while keeping bitset_mask == U32_MAX.
Subsequent updates can then write past the allocated object.

Second, fixing only the allocation size is not sufficient. The bloom hash
is a u32, but set_bit() takes a signed long bit number and x86 test_bit()
eventually feeds the index to variable_test_bit(long, ...). On 32-bit
kernels, hashes in [0x80000000, U32_MAX] therefore become negative bit
offsets. x86 bt/bts with a memory operand interpret those offsets relative
to the supplied base, so a map with bitset_mask == U32_MAX can read or
write before bloom->bitset even after allocating the full 512 MiB bitmap.

Keep the U32_MAX fallback, but split each hash into a word pointer and an
in-word bit number before calling test_bit() or set_bit(). The bitops
argument is then always in [0, BITS_PER_LONG - 1], while BIT_WORD(h) still
selects the intended word in the full bitmap.

Compute the bitset size from (u64)bitset_mask + 1 before passing the final
size to bpf_map_area_alloc(). This fixes the original under-allocation and
keeps the allocated storage consistent with the addressable bitset.

Exploitation note: local privilege escalation is possible on a 32-bit x86
kernel using the under-allocation bug from a binary with CAP_BPF.

Fixes: 9330986c0300 ("bpf: Add bloom filter map implementation")
Signed-off-by: Jérémy Jean 
Signed-off-by: Andrii Nakryiko 
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/20260805060228.2703051-1-Jeremy.Jean@oss.cyber.gouv.fr
Signed-off-by: Greg Kroah-Hartman 

Assisted-by: Codex:gpt-5

bpf: Disable preemption in __bpf_get_stack

2026-09-07T15:36:56+00:00

commit b1a47b2708d4e95dbd23aee2ec83752190897b3f upstream.

get_perf_callchain() returns a per-CPU perf_callchain_entry buffer and
releases its recursion slot via put_callchain_entry() before returning,
so nothing keeps the entry reserved while __bpf_get_stack() consumes
it below.

A preemptible BPF program (e.g. a non-sleepable raw tracepoint program
on a PREEMPT kernel, which runs under migrate_disable() but not
preempt_disable()) can be scheduled out between obtaining the entry
and the copy. Another task scheduled on the same CPU then reuses the
same per-CPU buffer and overwrites trace->nr with a larger value.
copy_len is then computed from the inflated trace->nr and can exceed
the caller's buffer, causing an out-of-bounds write in the memcpy()
and in the build_id path.

The rcu_read_lock() taken here alone does not prevent this. It is
only taken on the may_fault path, and under CONFIG_PREEMPT_RCU it does
not disable preemption; it merely keeps perf's callchain buffer array
alive (freed via call_rcu()) and does nothing to stop another task
from reusing the entry.

Disable preemption around obtaining the callchain entry and copying
it into the caller's buffer, so the entry cannot be reused underneath
us and trace->nr stays bounded by max_depth. Build ID resolution may
fault and is therefore deferred until after preemption is re-enabled;
by then the instruction pointers have already been copied into buf,
so it operates only on that private copy. Note, preempt_disable() also
subsumes the buffer-lifetime guarantee the rcu_read_lock() provided,
since a preempt-disabled section is an RCU read-side critical section
for the callchain buffers' call_rcu() reclaim.

Fixes: c195651e565a ("bpf: add bpf_get_stack helper")
Reported-by: Tao Chen 
Reported-by: STAR Labs SG 
Signed-off-by: Daniel Borkmann 
Signed-off-by: Jiri Olsa 
Signed-off-by: Andrii Nakryiko 
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/bpf/20260803210149.296496-11-jolsa@kernel.org
Signed-off-by: Greg Kroah-Hartman 

Closes: https://lore.kernel.org/bpf/20260206090653.1336687-1-chen.dylane@linux.dev/
[ changed Fixes: commit ]

rqspinlock: Reset tail when preserving queue on deadlock

2026-08-06T23:32:42+00:00

Currently, the destruction of the waiter queue is suppressed for
rqspinlock in cases where a deadlock is detected. Deadlock checks happen
relatively frequently (on entry for AA, within 1ms for ABBA), and waiter
threads may not be involved in locking scenarios involving deadlocks.
Thus, it is useful to not flush the queue and let other waiters take a
stab at acquiring the lock after we detect a deadlock and exit.

However, we need to follow the same logic as what we did previously for
the waitq_timeout label: reset the tail, and if we cannot, signal the
next waiter appropriately. In case of deadlocks, this signal would just
mark the MCS node as unlocked, and in case of timeouts, it would signal
RES_TIMEOUT_VAL. The difference thus is in the value propagated, which
decides whether the queue remains active or gets flushed.

Not doing the tail reset, and waiting for the next waiter can lead to
cases where we are the final waiter, and thus no next waiter arrives,
leading to intermittent stalls in this path. Once the next waiter does
join, we will be unblocked. In the theoretical case when the next waiter
never joins, we risk stalling indefinitely.

This can only happen for ABBA deadlocks, since entry into the wait queue
is guarded with AA checks. A precise sequence of executions leading up
to this scenario can be:

CPU 0 holds lock A.
CPU 1 holds lock B.
CPU 2 attempts lock B, becomes the pending waiter for B.
CPU 0 attempts lock B. B has locked+pending bits set, thus CPU 0 queues.
CPU 1 attempts lock A.
CPU 0 detects an ABBA deadlock.

Once deadlock detection happens for CPU 0, it will sit waiting for the
next waiter in the queue to populate node->next, which will experience
delays until such a waiter arrives.

Fix this by adjusting the logic for the check for deadlocks preceding
the waitq_timeout label. It would make sense to consolidate code for
both cases and use 'ret' to distinguish the value being propagated, but
that is left as an exercise for a future refactoring task to avoid diff
noise in this patch.

Fixes: 7bd6e5ce5be6 ("rqspinlock: Disable queue destruction for deadlocks")
Signed-off-by: Kumar Kartikeya Dwivedi 
Link: https://patch.msgid.link/20260802021759.1139457-1-memxor@gmail.com
Signed-off-by: Eduard Zingerman

bpf: Propagate untrusted pointer state in commuted arithmetic

2026-07-31T19:45:28+00:00

The untrusted PTR_TO_MEM early return skips pointer offset tracking
because accesses go through probe-read handling. Moving it after full
pointer-state propagation ensures scalar += untrusted_pointer leaves the
destination as PTR_TO_MEM instead of an unrelated scalar.

Fixes: f2362a57aeff ("bpf: allow void* cast using bpf_rdonly_cast()")
Signed-off-by: Yiyang Chen 
Tested-by: Daniel Wade 
Link: https://patch.msgid.link/20260729-c3-035-public-bpf-v4-v4-3-8ee297e2346b@mails.tsinghua.edu.cn
Signed-off-by: Eduard Zingerman