linux.git/include/linux/bpf_verifier.h, branch v7.3-rc2

bpf: Reject untrusted allocated-object pointers

2026-09-04T14:58:36+00:00

When the final RCU read-side critical section ends, a local kptr is demoted
to PTR_UNTRUSTED but retains MEM_ALLOC. The pointer may be NULL or may refer
to an object whose lifetime is no longer protected.

type_is_ptr_alloc_obj() nevertheless recognizes any PTR_TO_BTF_ID with
MEM_ALLOC as a live allocated object. In particular, a refcount-only local
kptr never carries NON_OWN_REF, so it still passes the
bpf_refcount_acquire() argument check after RCU protection ends. The kfunc
can then dereference NULL or stale memory.

Make type_is_ptr_alloc_obj() reject PTR_UNTRUSTED pointers. Since
type_is_non_owning_ref() is based on the same predicate, graph kfunc
arguments obey the same live-object requirement. Fault-protected reads of
the demoted pointer remain valid: writes are already rejected, and read
fixups use bpf_may_fault_on_deref() rather than this predicate.

Fixes: 1b12171533a9 ("bpf: Mark direct ld of stashed bpf_{rb,list}_node as non-owning ref")
Reported-by: Nicholas Carlini 
Suggested-by: Nicholas Carlini 
Signed-off-by: Ning Ding 
[ kkd: Rewrote commit log ]
Signed-off-by: Kumar Kartikeya Dwivedi 
Link: https://lore.kernel.org/r/20260904084325.52250-8-memxor@gmail.com
Signed-off-by: Alexei Starovoitov

bpf: don't rewrite bpf_fastcall patterns entered by a jump

2026-09-04T01:55:40+00:00

mark_fastcall_pattern_for_call() must ensure that matched
"spill; call; fill" instruction series is not interrupted by a jump.
Otherwise the rewrite applied by bpf_remove_fastcall_spills_fills()
is not sound.

Record the instructions targeted by jumps in
insn_aux_data[*].jump_target when the CFG is built and use this flag
to stop growing a pattern at such an instruction. Jumps to the first
spill are fine.

Note that existing insn_aux_data[*].jmp_point field can't be reused,
as it marks subprogram return instructions.

Fixes: 5b5f51bff1b6 ("bpf: no_caller_saved_registers attribute for helper calls")
Reported-by: Nicholas Carlini 
Suggested-by: Nicholas Carlini 
Signed-off-by: Eduard Zingerman 
Link: https://lore.kernel.org/r/20260903205820.1743087-1-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov

bpf: Rewrite any fault prone load out of a mem or btf_id pointer

2026-08-17T08:06:42+00:00

bpf_convert_ctx_accesses() turns a BPF_LDX into a BPF_PROBE_MEM one by
matching the type recorded for the insn against a list of exact pointer
types. The list cannot keep up with the flag combinations the verifier
produces, and a type which is missing from it ends up as a plain load
without an exception table entry, so a bad address panics the kernel
instead of being handled.

Two such types exist today and are reachable:

  - PTR_TO_BTF_ID | PTR_UNTRUSTED | MEM_ALLOC | NON_OWN_REF
  - PTR_TO_BTF_ID | PTR_UNTRUSTED | MEM_RCU

Rather than adding the two, just drop the list and state the property
itself in the default case of the switch. This is a superset of what
the list matched, the untrusted PTR_TO_MEM does not have to carry
MEM_RDONLY for it anymore, and it stays in sync with the verifier side
which uses the same match in save_aux_ptr_type() and reg_type_mismatch_ok().

Assert that a fault prone type which does not get the rewrite for whatever
reason is rejected at load time rather than left to fault at runtime to
catch any future cases.

Fixes: 1b12171533a9 ("bpf: Mark direct ld of stashed bpf_{rb,list}_node as non-owning ref")
Fixes: 6fcd486b3a0a ("bpf: Refactor RCU enforcement in the verifier.")
Signed-off-by: Daniel Borkmann 
Acked-by: Eduard Zingerman 
Link: https://lore.kernel.org/bpf/20260814215301.709827-4-daniel@iogearbox.net

bpf: Keep fault protection when merging pointer types

2026-08-17T08:05:16+00:00

When the same BPF_LDX instruction is reached through paths that yield
different pointer types, save_aux_ptr_type() merges them into a single
type which is later used by bpf_convert_ctx_accesses() to decide whether
the load has to be rewritten into a BPF_PROBE_MEM one.

Before f2362a57aeff ("bpf: allow void* cast using bpf_rdonly_cast()")
the merge only accepted two PTR_TO_BTF_ID pointers and unconditionally
fell back to PTR_TO_BTF_ID | PTR_UNTRUSTED, so the merged type was always
one that gets the BPF_PROBE_MEM rewrite. However, the mentioned commit
widened the merge to also cover a PTR_TO_MEM base and replaced the
fallback by a union of the PTR_UNTRUSTED and MEM_RDONLY flags.

A union of flags though cannot express the property the later rewrite
is built upon, some examples:

  - PTR_TO_MEM merged with PTR_TO_BTF_ID | PTR_UNTRUSTED gets
    PTR_TO_MEM | PTR_UNTRUSTED but only the MEM_RDONLY variant is valid
  - PTR_TO_MEM merged with a plain PTR_TO_BTF_ID gets PTR_TO_MEM
    dropping the rewrite the latter type would have gotten
  - PTR_TO_MEM | MEM_RDONLY merged with a plain PTR_TO_BTF_ID gets
    PTR_TO_MEM | MEM_RDONLY which is not rewritten either since only
    its PTR_UNTRUSTED variant is

In all three cases a program can take the unsafe path at runtime with a
NULL or otherwise bad pointer and panic the kernel on the faulting load:

  BUG: kernel NULL pointer dereference, address: 0000000000000038
  RIP: 0010:bpf_prog_77531a87032eeaf1_mixed_mem_btf_id_type+0x4b/0x65
  Call Trace:
   
   bpf_test_run+0x20b/0x460
   bpf_prog_test_run_skb+0x650/0xbe0
   __sys_bpf+0xb96/0x3140
   __x64_sys_bpf+0x2c/0x40
   do_syscall_64+0xba/0x590
  Kernel panic - not syncing: Fatal exception in interrupt

Note that the last two shapes have to be fixed right here, otherwise
the merged type retains nothing which marks the load as fault prone,
thus no rule in bpf_convert_ctx_accesses() can recover it. Fix it by
normalizing the merged type instead.

Reuse it in is_load_acq_unsafe() to avoid open coding, and trim the
overly verbose comment which is more of an implementation detail of
bpf_convert_ctx_accesses() anyway.

Fixes: f2362a57aeff ("bpf: allow void* cast using bpf_rdonly_cast()")
Signed-off-by: Daniel Borkmann 
Acked-by: Eduard Zingerman 
Link: https://lore.kernel.org/bpf/20260814215301.709827-1-daniel@iogearbox.net

bpf: Track verifier register diagnostic events

2026-08-15T18:11:16+00:00

Record material register and outgoing stack argument changes so diagnostics can
explain how a value reached its current type, bounds, or unreadable state.

Store old and new register types, scalar ranges, tnum value and mask, map and
BTF type identity, and basic operand metadata in the environment-owned
diagnostic event stream.

Record invalidations when packet data moves, references are released, or
borrowed references leave their protected region. Register-scoped history
starts at the latest matching modification and then shows later branch
outcomes.

Also record fixed stack spills and overwrites, and tag register fills from
stack so register-scoped history can follow value flow through spilled stack
slots.

The type_is_map_ptr() helper previously lived as a static function in
kernel/bpf/log.c since commit 0c95c9fdb696 ("bpf: emit map name in register
state if applicable and available"). Move it verbatim to
include/linux/bpf_verifier.h as a static inline, next to the other type
classifiers, so diagnostics.c can reuse it without duplicating the case list.

Signed-off-by: Kumar Kartikeya Dwivedi 
Link: https://patch.msgid.link/20260815064612.378577-6-memxor@gmail.com
Signed-off-by: Eduard Zingerman

bpf: Add source and instruction diagnostic context

2026-08-15T18:11:16+00:00

Teach verifier diagnostics to annotate an instruction with BTF source
line information and nearby BPF instructions. The renderer keeps source
text in a fixed-width lane and prints instructions in a stable right-hand
gutter.

Wrap annotation text under the source line so long error labels remain
readable while the source and instruction lanes keep their fixed layout.

Keeping source and instruction context in one commit preserves the visual
layout contract that later diagnostic reports rely on.

Signed-off-by: Kumar Kartikeya Dwivedi 
Link: https://patch.msgid.link/20260815064612.378577-3-memxor@gmail.com
Signed-off-by: Eduard Zingerman

bpf: Eliminate dup/restore of insn_aux_data

2026-08-13T02:36:30+00:00

The dup/restore of insn_aux_data was introduced to resolve the
inconsistency between insnsi and insn_aux_data arrays, which occurs
on the failure path where insnsi was rolled back to the original
state before constants blinding, while insn_aux_data was not.

After JIT failure, there is only one user, bpf_clear_insn_aux_data(),
that requires insnsi and insn_aux_data to be synchronized. It accesses
both insnsi and insn_aux_data using the same array size and index.

However, the access to insnsi in bpf_clear_insn_aux_data() is not
necessary. It is checked to skip the second slot of an ldimm64 instruction,
whose jt is never set and can be absorbed into the jt check itself.

So remove the access to insnsi from bpf_clear_insn_aux_data(), and add a
specific length field for insn_aux_data to allow it to have a different
length from the insnsi array. Then remove dup/restore of insn_aux_data.

Signed-off-by: Xu Kuohai 
Acked-by: Eduard Zingerman 
Link: https://lore.kernel.org/bpf/5a4528f019c8d2638c019a2f37475cccc16a9503.1785240296.git.xukuohai@huawei.com
Signed-off-by: Kumar Kartikeya Dwivedi

bpf: Track verifier instruction stats for each subprogram

2026-08-13T00:45:07+00:00

The verifier currently records one instruction count for the main program
and each global subprogram checked independently. Static subprograms are
explored within callers, so their verification cost cannot be reported
separately.

Track both self and inclusive instruction counts for every subprogram.
Charge each processed instruction as self work to the current subprogram and
to a path-local subtotal in its function frame. When a function returns, add
the callee subtotal to its inclusive count and to its parent subtotal. Fold
any remaining frames when a path terminates or is pruned.

Instruction subtotals are accounting state, not semantic verifier state.
Clear them when a verifier state is copied so work before a path fork is
charged once, rather than again when a saved branch is explored. If copying
a saved state fails before all frames are allocated, skip missing frames
while folding the current path.

This generic frame accounting also records self and inclusive totals when an
asynchronous callback starts as a fresh frame-zero state. It does not yet
charge that independently explored callback path back to the main or global
exploration root which scheduled it. That will be done in subsequent
changes.

This does not change the verification statistics output format. It only
prepares the counters for per-subprogram reporting.

Signed-off-by: Kumar Kartikeya Dwivedi 
Acked-by: Eduard Zingerman 
Link: https://patch.msgid.link/20260812221925.3358041-2-memxor@gmail.com
Signed-off-by: Eduard Zingerman

bpf: Support arena and arena__nullable on struct_ops arguments

2026-08-08T10:03:26+00:00

A struct_ops callback cannot receive an arena pointer directly, so
passing one takes two steps. The pointer arrives as a bare u64 that the
callback casts, and because the two sides address the arena through
different bases it also has to be rebased by hand on the way in.

Add the __arena and __arena__nullable stub argument suffixes to make this
convenient. The callback declares the parameter as an arena pointer,
receives it as a PTR_TO_ARENA register, and dereferences it directly,
while the kernel caller just passes the natural kernel arena address
(kaddr). The trampoline converts the value while saving the arguments
into the BPF ctx, ctx[slot] = (u32)(kaddr - kern_vm_start), so the
program never sees a kernel address and nothing rewrites the ctx after
the fact. The converted value keeps the upper 32 bits clear as the JITs
require of arena pointer registers and behaves like any cast_kern'ed
arena pointer, so cast_user recovers the full user-visible address.

__arena converts unconditionally and the kernel caller must not pass
NULL. __arena__nullable preserves NULL, tested on the full 64-bit kernel
pointer, and surfaces to the verifier as PTR_TO_ARENA (but not as a
PTR_TO_ARENA | PTR_MAYBE_NULL). The reason is that PTR_TO_ARENA in the
program's type state already encompasses NULL-ness, so it is not
meaningful to force a NULL check for the program.

The composite suffix intentionally ends in __nullable. Classify
__arena__nullable before the generic suffix so scalar arena pointees do
not take the generic nullable BTF pointer path.

This patch adds the generic side. prepare_arg_info() records arena and
nullable argument flags in the struct_ops function model, and
bpf_tramp_arena_base() returns the arena base for a single-program
struct_ops indirect trampoline. Only that trampoline converts: its
program's arena is fixed at generation time. Generic trampolines can mix
programs with different arenas and reject arena context arguments
defensively, which is unreachable today as only struct_ops programs
carry them. Architectures that do not implement the conversion are
gated out at verification time with bpf_jit_supports_arena_args().

Co-developed-by: Kumar Kartikeya Dwivedi 
Signed-off-by: Tejun Heo 
Signed-off-by: Kumar Kartikeya Dwivedi 
Acked-by: Eduard Zingerman 
Link: https://patch.msgid.link/20260808003938.3486067-6-memxor@gmail.com
Signed-off-by: Eduard Zingerman

bpf: Rename 'early' BTF checking as a preparation phase

2026-08-08T10:03:25+00:00

BTF processing is split around subprogram discovery. The first phase gets
program BTF and imports func_info because a BTF-tagged exception callback
may not be referenced by any instruction. Subprogram discovery needs this
metadata to find it.

The later phase validates func_info and line_info against the complete
subprogram table and applies CO-RE relocations. This split breaks a real
dependency cycle rather than merely running the same checks early.

Rename bpf_check_btf_info_early() and check_btf_func_early() to preparation
names that reflect this role. Add short call-site comments to make the two
phases and their responsibilities clear.

No functional change is intended.

Signed-off-by: Kumar Kartikeya Dwivedi 
Reviewed-by: Amery Hung 
Link: https://patch.msgid.link/20260808003938.3486067-2-memxor@gmail.com
Signed-off-by: Eduard Zingerman