linux-stable.git/kernel/bpf/disasm.c, branch v7.3-rc2

bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics

2026-09-04T01:54:45+00:00

bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from
BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, this patch adjusts
print_bpf_insn() to print such instructions as regular atomics with a
'probe_' prefix (instead of printing them as BUG_XX).

Signed-off-by: Eduard Zingerman 
Link: https://lore.kernel.org/r/20260903171542.1438050-2-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov

bpf: Reject invalid LDSX instruction in disassembly

2026-08-20T16:45:44+00:00

The signed-load mnemonic table has entries for byte, half-word, and word
loads because BPF_MEMSX does not support double-word loads. A BPF_MEMSX
| BPF_DW instruction nevertheless selects index 3, past the end of this
table.

Program Structure diagnostics can disassemble a malformed instruction
before check_and_resolve_insns() rejects its opcode. Placing the invalid
signed double-word load at the end of a program therefore triggers an
out-of-bounds access while reporting subprogram fallthrough.

Treat signed double-word loads as invalid in the disassembler and use
the existing BUG_ldx fallback instead.

Fixes: a8f427835394 ("bpf: Report Program Structure CFG errors")
Reported-by: syzbot+3544d9b2a9206be8ba37@syzkaller.appspotmail.com
Signed-off-by: Kumar Kartikeya Dwivedi 
Signed-off-by: Daniel Borkmann 
Reviewed-by: Jiayuan Chen 
Link: https://lore.kernel.org/bpf/20260820022020.3450479-2-memxor@gmail.com

bpf: Do not print a newline after disassembly in bpf_verbose_insn()

2026-08-08T09:05:49+00:00

At the moment there are more callsites that want bpf_verbose_insn() to
not print a newline after the instruction, than callsites that want a
newline. Drop '\n' from disasm.c. Non-functional change.

The changes in bpftool are verified by writing a bpf program using a
variety of instructions and comparing `prog dump xlated` output in the
following modes: plain, opcodes, visual, visual opcodes. The output
before and after the changes is identical.

Signed-off-by: Eduard Zingerman 
Signed-off-by: Daniel Borkmann 
Reviewed-by: Quentin Monnet 
Acked-by: Daniel Borkmann 
Link: https://lore.kernel.org/bpf/20260807-static-zext-v4-1-b6c270013c77@gmail.com

bpf: Mask pseudo pointer values in verifier logs

2026-06-26T00:59:04+00:00

print_bpf_insn() masks ldimm64 immediates for pointer-bearing pseudo
sources when pointer leaks are not allowed, but the mask only covers
BPF_PSEUDO_MAP_FD and BPF_PSEUDO_MAP_VALUE.

BPF_PSEUDO_MAP_IDX, BPF_PSEUDO_MAP_IDX_VALUE, and BPF_PSEUDO_BTF_ID can
also be resolved to kernel pointer values before the verifier log prints
the instruction. Include them in the existing pointer classification so
the log prints 0x0 instead of the rewritten address.

Fixes: 4976b718c355 ("bpf: Introduce pseudo_btf_id")
Fixes: 387544bfa291 ("bpf: Introduce fd_idx")
Signed-off-by: Nuoqi Gui 
Link: https://lore.kernel.org/r/20260623-f01-13-pseudo-btf-id-cap-bpf-v2-1-a190ebb8f3e2@mails.tsinghua.edu.cn
Signed-off-by: Alexei Starovoitov 
Acked-by: Eduard Zingerman

bpf: disasm: add support for BPF_JMP|BPF_JA|BPF_X

2025-11-06T01:53:23+00:00

Add support for indirect jump instruction.

Example output from bpftool:

   0: (79) r3 = *(u64 *)(r1 +0)
   1: (25) if r3 > 0x4 goto pc+666
   2: (67) r3 <<= 3
   3: (18) r1 = 0xffffbeefspameggs
   5: (0f) r1 += r3
   6: (79) r1 = *(u64 *)(r1 +0)
   7: (0d) gotox r1

Signed-off-by: Anton Protopopov 
Acked-by: Eduard Zingerman 
Link: https://lore.kernel.org/r/20251105090410.1250500-10-a.s.protopopov@gmail.com
Signed-off-by: Alexei Starovoitov

bpftool: Using the right format specifiers

2025-03-17T20:50:56+00:00

Fixed some formatting specifiers errors, such as using %d for int and %u
for unsigned int, as well as other byte-length types.

Perform type cast using the type derived from the data type itself, for
example, if it's originally an int, it will be cast to unsigned int if
forced to unsigned.

Signed-off-by: Jiayuan Chen 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20250311112809.81901-3-jiayuan.chen@linux.dev

bpf: Introduce load-acquire and store-release instructions

2025-03-15T18:48:28+00:00

Introduce BPF instructions with load-acquire and store-release
semantics, as discussed in [1].  Define 2 new flags:

  #define BPF_LOAD_ACQ    0x100
  #define BPF_STORE_REL   0x110

A "load-acquire" is a BPF_STX | BPF_ATOMIC instruction with the 'imm'
field set to BPF_LOAD_ACQ (0x100).

Similarly, a "store-release" is a BPF_STX | BPF_ATOMIC instruction with
the 'imm' field set to BPF_STORE_REL (0x110).

Unlike existing atomic read-modify-write operations that only support
BPF_W (32-bit) and BPF_DW (64-bit) size modifiers, load-acquires and
store-releases also support BPF_B (8-bit) and BPF_H (16-bit).  As an
exception, however, 64-bit load-acquires/store-releases are not
supported on 32-bit architectures (to fix a build error reported by the
kernel test robot).

An 8- or 16-bit load-acquire zero-extends the value before writing it to
a 32-bit register, just like ARM64 instruction LDARH and friends.

Similar to existing atomic read-modify-write operations, misaligned
load-acquires/store-releases are not allowed (even if
BPF_F_ANY_ALIGNMENT is set).

As an example, consider the following 64-bit load-acquire BPF
instruction (assuming little-endian):

  db 10 00 00 00 01 00 00  r0 = load_acquire((u64 *)(r1 + 0x0))

  opcode (0xdb): BPF_ATOMIC | BPF_DW | BPF_STX
  imm (0x00000100): BPF_LOAD_ACQ

Similarly, a 16-bit BPF store-release:

  cb 21 00 00 10 01 00 00  store_release((u16 *)(r1 + 0x0), w2)

  opcode (0xcb): BPF_ATOMIC | BPF_H | BPF_STX
  imm (0x00000110): BPF_STORE_REL

In arch/{arm64,s390,x86}/net/bpf_jit_comp.c, have
bpf_jit_supports_insn(..., /*in_arena=*/true) return false for the new
instructions, until the corresponding JIT compiler supports them in
arena.

[1] https://lore.kernel.org/all/20240729183246.4110549-1-yepeilin@google.com/

Acked-by: Eduard Zingerman 
Acked-by: Ilya Leoshkevich 
Cc: kernel test robot 
Signed-off-by: Peilin Ye 
Link: https://lore.kernel.org/r/a217f46f0e445fbd573a1a024be5c6bf1d5fe716.1741049567.git.yepeilin@google.com
Signed-off-by: Alexei Starovoitov

bpf: add special internal-only MOV instruction to resolve per-CPU addrs

2024-04-03T17:29:55+00:00

Add a new BPF instruction for resolving absolute addresses of per-CPU
data from their per-CPU offsets. This instruction is internal-only and
users are not allowed to use them directly. They will only be used for
internal inlining optimizations for now between BPF verifier and BPF JITs.

We use a special BPF_MOV | BPF_ALU64 | BPF_X form with insn->off field
set to BPF_ADDR_PERCPU = -1. I used negative offset value to distinguish
them from positive ones used by user-exposed instructions.

Such instruction performs a resolution of a per-CPU offset stored in
a register to a valid kernel address which can be dereferenced. It is
useful in any use case where absolute address of a per-CPU data has to
be resolved (e.g., in inlining bpf_map_lookup_elem()).

BPF disassembler is also taught to recognize them to support dumping
final BPF assembly code (non-JIT'ed version).

Add arch-specific way for BPF JITs to mark support for this instructions.

This patch also adds support for these instructions in x86-64 BPF JIT.

Signed-off-by: Andrii Nakryiko 
Acked-by: John Fastabend 
Link: https://lore.kernel.org/r/20240402021307.1012571-2-andrii@kernel.org
Signed-off-by: Alexei Starovoitov

bpf: Disasm support for addr_space_cast instruction.

2024-03-11T22:37:24+00:00

LLVM generates rX = addr_space_cast(rY, dst_addr_space, src_addr_space)
instruction when pointers in non-zero address space are used by the bpf
program. Recognize this insn in uapi and in bpf disassembler.

Signed-off-by: Alexei Starovoitov 
Signed-off-by: Andrii Nakryiko 
Acked-by: Kumar Kartikeya Dwivedi 
Link: https://lore.kernel.org/bpf/20240308010812.89848-3-alexei.starovoitov@gmail.com

bpf: Introduce may_goto instruction

2024-03-06T23:17:31+00:00

Introduce may_goto instruction that from the verifier pov is similar to
open coded iterators bpf_for()/bpf_repeat() and bpf_loop() helper, but it
doesn't iterate any objects.
In assembly 'may_goto' is a nop most of the time until bpf runtime has to
terminate the program for whatever reason. In the current implementation
may_goto has a hidden counter, but other mechanisms can be used.
For programs written in C the later patch introduces 'cond_break' macro
that combines 'may_goto' with 'break' statement and has similar semantics:
cond_break is a nop until bpf runtime has to break out of this loop.
It can be used in any normal "for" or "while" loop, like

  for (i = zero; i < cnt; cond_break, i++) {

The verifier recognizes that may_goto is used in the program, reserves
additional 8 bytes of stack, initializes them in subprog prologue, and
replaces may_goto instruction with:
aux_reg = *(u64 *)(fp - 40)
if aux_reg == 0 goto pc+off
aux_reg -= 1
*(u64 *)(fp - 40) = aux_reg

may_goto instruction can be used by LLVM to implement __builtin_memcpy,
__builtin_strcmp.

may_goto is not a full substitute for bpf_for() macro.
bpf_for() doesn't have induction variable that verifiers sees,
so 'i' in bpf_for(i, 0, 100) is seen as imprecise and bounded.

But when the code is written as:
for (i = 0; i < 100; cond_break, i++)
the verifier see 'i' as precise constant zero,
hence cond_break (aka may_goto) doesn't help to converge the loop.
A static or global variable can be used as a workaround:
static int zero = 0;
for (i = zero; i < 100; cond_break, i++) // works!

may_goto works well with arena pointers that don't need to be bounds
checked on access. Load/store from arena returns imprecise unbounded
scalar and loops with may_goto pass the verifier.

Reserve new opcode BPF_JMP | BPF_JCOND for may_goto insn.
JCOND stands for conditional pseudo jump.
Since goto_or_nop insn was proposed, it may use the same opcode.
may_goto vs goto_or_nop can be distinguished by src_reg:
code = BPF_JMP | BPF_JCOND
src_reg = 0 - may_goto
src_reg = 1 - goto_or_nop

Signed-off-by: Alexei Starovoitov 
Signed-off-by: Andrii Nakryiko 
Acked-by: Andrii Nakryiko 
Acked-by: Eduard Zingerman 
Acked-by: John Fastabend 
Tested-by: John Fastabend 
Link: https://lore.kernel.org/bpf/20240306031929.42666-2-alexei.starovoitov@gmail.com