linux-stable.git/include/linux/filter.h, branch v7.3-rc2

x86/bpf: Make arch_bpf_trampoline_size allocate from EXECMEM_MODULE_DATA

2026-08-20T15:52:31+00:00

Jiri Olsa reports slowdown of tracing_multi benchmark that allocates huge
number of trampolines [1].

The slowdown caused by extra protection changes in execmem_alloc_rw() and
execmem_free().

With ROX caches enabled, all execmem allocations except EXECMEM_MODULE_DATA
are ROX after the allocation. execmem_alloc_rw() temporarily sets them to
W+NX and execmem_free() resets them back to ROX.

The only user of bpf_jit_alloc_exec_rw() is x86::arch_bpf_trampoline_size()
that only needs a temporary writable buffer in the modules address space.

On x86 executable memory and module data are constrained to the same
address range, so x86::arch_bpf_trampoline_size() can directly use
execmem_alloc(EXECMEM_MODULE_DATA)

Replace the call to bpf_jit_alloc_exec_rw() with a call to
execmem_alloc(EXECMEM_MODULE_DATA) in x86::arch_bpf_trampoline_size() and
drop bpf_jit_alloc_exec_rw() helper.

Fixes: 5bf02dbf39fa ("bpf, x86: Make sure allocation in arch_bpf_trampoline_size() is writable")
Reported-by: Jiri Olsa 
Signed-off-by: Mike Rapoport (Microsoft) 
Signed-off-by: Daniel Borkmann 
Tested-by: Jiri Olsa 
Link: https://lore.kernel.org/all/an8r7EODLIL-bZM3@krava
Link: https://lore.kernel.org/bpf/20260818130510.3110054-1-rppt@kernel.org

bpf: Eliminate dup/restore of insn_aux_data

2026-08-13T02:36:30+00:00

The dup/restore of insn_aux_data was introduced to resolve the
inconsistency between insnsi and insn_aux_data arrays, which occurs
on the failure path where insnsi was rolled back to the original
state before constants blinding, while insn_aux_data was not.

After JIT failure, there is only one user, bpf_clear_insn_aux_data(),
that requires insnsi and insn_aux_data to be synchronized. It accesses
both insnsi and insn_aux_data using the same array size and index.

However, the access to insnsi in bpf_clear_insn_aux_data() is not
necessary. It is checked to skip the second slot of an ldimm64 instruction,
whose jt is never set and can be absorbed into the jt check itself.

So remove the access to insnsi from bpf_clear_insn_aux_data(), and add a
specific length field for insn_aux_data to allow it to have a different
length from the insnsi array. Then remove dup/restore of insn_aux_data.

Signed-off-by: Xu Kuohai 
Acked-by: Eduard Zingerman 
Link: https://lore.kernel.org/bpf/5a4528f019c8d2638c019a2f37475cccc16a9503.1785240296.git.xukuohai@huawei.com
Signed-off-by: Kumar Kartikeya Dwivedi

bpf: Derive the atomic load register in one place

2026-08-12T17:33:53+00:00

check_atomic_rmw() open codes the mapping from a BPF_ATOMIC to the register
it reads the old value into, the BPF_STX case of insn_def_regno() open codes
the very same mapping a second time, the const folding and the liveness
transfer functions a third and a fourth time, and BPF JITs need it as well
to know which register a faulting BPF_PROBE_ATOMIC has to clear.

Add a small helper so that all of them can share it. No functional change.
The BPF_LOAD_ACQ case is there for the JITs, which do walk all instruction
classes. const_reg_xfer() loses its explicit BPF_ATOMIC mode test since the
helper checks class and mode itself; the BPF_PROBE_ATOMIC it additionally
accepts cannot be seen there as it is only set from bpf_do_misc_fixups(),
that is, after const folding has run. arg_track_xfer() keeps its mode test
since that also guards the stack clearing next to it.

Signed-off-by: Daniel Borkmann 
Acked-by: Eduard Zingerman 
Link: https://patch.msgid.link/20260811131600.506721-1-daniel@iogearbox.net
Signed-off-by: Eduard Zingerman

bpf: Support arena and arena__nullable kfunc argument suffixes

2026-08-08T10:03:26+00:00

Passing an arena pointer to a kfunc takes two steps today. There is no
arena pointer argument type, so the pointer crosses the boundary as a
bare scalar, and the kfunc then offsets it by the arena base and casts
it before it can touch the memory. Every such kfunc open-codes the same
translation.

Add the __arena and __arena__nullable argument suffixes to make this more
convenient. The kfunc declares the parameter by its real pointer type
and dereferences it directly, with the JIT rebasing the value at the
call site, rN = kern_vm_start + (u32)rN. No bounds check is needed: the
u32 offset stays within the guard-padded arena kernel mapping, and a
fault on an unpopulated page recovers through the per-arena scratch
page. A suffixed argument accepts a PTR_TO_ARENA or scalar register,
matching global subprog arena arguments.

__arena rebases unconditionally, so the kfunc never sees NULL and a
value with zero in the low 32 bits arrives as the arena base.
__arena__nullable preserves NULL for optional arguments by skipping the
rebase when the truncated value, arena offset 0, is zero. Keeping the
plain form NULL-free saves the NULL test on every call.

The double separator makes the annotations composable:
__arena__nullable also ends in __nullable and naturally follows the
common nullable argument path. Plain __arena follows that path too for
verifier type checking because both forms accept a constant zero; the
function-model flag still determines whether the JIT preserves NULL or
rebases it to the arena base.

This patch adds the verifier side: the suffixes are recognized in
check_kfunc_args() and distilled into argument flags in the function
model stored in the kfunc descriptor. JITs retrieve the model while
emitting the call, avoiding per-call state in insn_aux_data.

JITs declare support with bpf_jit_supports_arena_args() and verification
fails with -ENOTSUPP elsewhere.

Co-developed-by: Kumar Kartikeya Dwivedi 
Signed-off-by: Tejun Heo 
Signed-off-by: Kumar Kartikeya Dwivedi 
Acked-by: Eduard Zingerman 
Link: https://patch.msgid.link/20260808003938.3486067-5-memxor@gmail.com
Signed-off-by: Eduard Zingerman

bpf, riscv: Add and use bpf_atomic_is_load_acq() helper

2026-08-07T12:57:13+00:00

A load-acquire is the only BPF_STX class instruction that reads from
src_reg into dst_reg, that is, it has the operand roles of a BPF_LDX.
JIT code which tells loads from stores apart by instruction class alone
has to special case it, for example when deciding which register holds
the faulting address and which one to clear from an exception handler.

riscv64 already does so, open coded as a bare insn->imm test. Add a
bpf_atomic_is_load_acq() helper and convert riscv64 over to it, so that
the x86-64 and arm64 JITs can use the same helper in subsequent patches.

Unlike bpf_atomic_is_load_store(), which presumes that its argument is
already known to be a BPF_ATOMIC instruction, the new helper is called
from code which still sees all instruction classes, so it checks class
and mode itself.

Also, move bpf_atomic_is_load_store() to filter.h next to BPF_ATOMIC_OP,
so that both helpers stay together. No functional change intended.

Signed-off-by: Daniel Borkmann 
Link: https://lore.kernel.org/bpf/20260806201047.333389-2-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi

bpf, x86: Make sure allocation in arch_bpf_trampoline_size() is writable

2026-07-22T15:27:10+00:00

arch_bpf_trampoline_size() allocates a buffer to get actual size required
for a trampoline.

This buffer must be in the module address space because
__arch_prepare_bpf_trampoline() calculates  rel32 offsets relatively to
that buffer.

In preparation for enabling ROX mode for EXECMEM_BPF make sure that the
allocated memory is writable.

Add bpf_jit_alloc_exec_rw() wrapper for execmem_alloc_rw() and use it for
buffer allocation in arch_bpf_trampoline_size().

Signed-off-by: Mike Rapoport (Microsoft) 
Acked-by: Song Liu 
Link: https://lore.kernel.org/bpf/20260716-execmem-x86-rox-bpf-v0-v3-4-4e76158c01c5@kernel.org
Signed-off-by: Kumar Kartikeya Dwivedi

bpf: Restrict JIT predictor flush to cBPF

2026-07-01T08:33:38+00:00

Currently predictor flush on memory reuse is done for all BPF JIT
allocations, but only cBPF programs can be loaded by an unprivileged user.
eBPF is privileged by default, and flushing predictors for all CPUs on
every eBPF reuse penalizes the common case for no security benefit.

eBPF allocations can be frequent on busy systems, only flush predictors
for cBPF programs. Trampoline and dispatcher allocations also skip the
flush as they are eBPF-only.

Signed-off-by: Pawan Gupta 
Acked-by: Daniel Borkmann 
Signed-off-by: Daniel Borkmann

bpf: Support for hardening against JIT spraying

2026-07-01T08:33:35+00:00

The BPF JIT allocator packs many small programs into larger executable
allocations and reuses space within those allocations as programs are
loaded and freed. When fresh code is written into space that a previous
program occupied, an indirect jump into the new program can reuse a branch
prediction left behind by the old one.

Flush the indirect branch predictors before reusing JIT memory so that
indirect jumps into a newly written program don't reuse predictions from an
old program that occupied the same space.

Introduce bpf_arch_pred_flush_enabled static key and bpf_arch_pred_flush
static call for flushing the branch predictors on JIT memory reuse.
Architectures that need a flush, can update it to a predictor flush
function. By default, its a NOP and does not emit any CALL.

Allocations larger than a pack are not covered by this flush. That is safe
because cBPF programs (the unprivileged attack surface) are bounded well
below a pack size. Issue a warning if this assumption is ever violated
while the flush is active.

Signed-off-by: Pawan Gupta 
Acked-by: Daniel Borkmann 
Signed-off-by: Daniel Borkmann

Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf 7.1-rc5

2026-05-25T13:33:30+00:00

Cross-merge BPF and other fixes after downstream PR.

Signed-off-by: Alexei Starovoitov

bpf: Prepare architecture JIT support for stack arguments

2026-05-13T16:27:31+00:00

Add bpf_jit_supports_stack_args() as a weak function defaulting to
false. Architectures that implement JIT support for stack arguments
override it to return true.

Reject BPF functions with more than 5 parameters at verification
time if the architecture does not support stack arguments.

Acked-by: Puranjay Mohan 
Signed-off-by: Yonghong Song 
Link: https://lore.kernel.org/r/20260513045054.2390945-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov