linux.git/tools/testing/selftests/bpf/benchs, branch master Linux kernel source tree Merge tag 'bpf-next-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2026-08-20T14:36:20+00:00 Linus Torvalds torvalds@linux-foundation.org 2026-08-20T14:36:20+00:00 5a8cd539ac19f7a68e68e1d25ef9ca2ff55b8500 Pull bpf updates from Daniel Borkmann: "Major changes: - Redesign the verifier error reporting: failures now carry source and instruction annotations along with the causal event history that led to them, making program rejections far easier to debug and repair (Kumar Kartikeya Dwivedi) - Add arena argument support to kfuncs and struct_ops through the new __arena and __arena__nullable suffixes (Tejun Heo, Puranjay Mohan, Kumar Kartikeya Dwivedi, Ihor Solodrai) - Signed BPF program loader rework to accommodate both BPF and security community needs where the kernel runs the signature verification at BPF_PROG_LOAD time before the LSM admission hook (Daniel Borkmann) - Add a set of ksock kfuncs which let BPF LSM and syscall programs create, connect and send on UDP sockets in order to emit telemetry data (Mahe Tardy) - Unify helper and kfunc call argument verification and classify kfunc arguments purely from BTF into a generated bpf_func_proto which is computed once at add-call time (Amery Hung) Other features and fixes: - Enable EXECMEM_ROX_CACHE for BPF allocations on x86 (Mike Rapoport) - Add bidirectional VLAN support to bpf_fib_lookup() through the new BPF_FIB_LOOKUP_VLAN and BPF_FIB_LOOKUP_VLAN_INPUT flags (Avinash Duduskar) - Infer zext_dst from static register liveness analysis to fix 32-bit zero-extension semantics, and remove the artificial limitations on pointer types eligible for spilling (Eduard Zingerman) - Inline the numeric open-coded iterator kfuncs so that bpf_for() loops no longer pay a kfunc call on every iteration (Puranjay Mohan) - Add an arena-based bitmap data structure to libarena along with serial and parallel selftests (Emil Tsalapatis) - Teach resolve_btfids to discover kfuncs from the kernel's BTF ID sets and to emit kfunc BTF decl tags, reducing the kernel build's dependency on pahole features (Ihor Solodrai) - Add BPF_F_ADJ_ROOM_DECAP_* flags to bpf_skb_adjust_room() so that tunnel decapsulation can update the GSO and encapsulation state of the skb (Nick Hudson) - Fix the ring buffer pending_pos walk and the available-data accounting on 32-bit position wrap (Israel Téllez García) - Add memory usage accounting for arena maps and fix an mmap_lock deadlock on arena lock failure (Jiayuan Chen) - Add tracing_multi link info support to the kernel UAPI and bpftool, and refactor the stack map code to run with preemption disabled (Jiri Olsa) - Support BPF_F_EGRESS in bpf_redirect_peer() to emit the skb in the egress direction of the target's peer device (Jordan Rife) - Add a KF_SPINLOCK_SAFE kfunc flag so that providers, in particular modules, can declare kfuncs safe to call under bpf_spin_lock instead of relying on the verifier's hard-coded allowlist (Kaitao Cheng) - Introduce global percpu data for BPF programs with libbpf probing and bpftool skeleton support, and stop exposing uninitialized kernel heap memory when copying per-CPU map values (Leon Hwang) - Add s390 JIT support for load-acquire and store-release instructions (Maxim Khmelevskii) - Fix a CFI mismatch in the task work callback and an arm64 KASAN false positive after bpf_throw() (Mykyta Yatsenko) - Reject writes through untrusted BTF pointers and bound the rdonly/rdwr_buf_size kfunc arguments (Nicholas Dudar) - Invalidate RCU pointers only after the final spin unlock and account for preempt and IRQ disabled regions as overlapping RCU protection (Ning Ding) - Support mixing bpf2bpf calls and tail calls on RV64, add signed operations and 32-bit atomics to the RV32 JIT, and add timed may_goto support (Pu Lehui, Kuan-Wei Chiu, Feng Jiang) - Fix a use-after-free on mm_struct in bpf_find_vma() for foreign tasks and an mmap_lock leak in the irq_work path (Sanghyun Park) - Populate mmap-able BPF array map memory lazily which makes mmap() O(1) instead of proportional to the map size (Song Liu) - Introduce a jit_required flag and reject programs with inlined helpers when no JIT is available, where the interpreter would otherwise jump into an invalid address (Tiezhu Yang) - Fix the x86 JIT per-CPU address resolution into an extended register where the REX prefix dropped the high destination register bit (Vineet Gupta) - Reject MEM_ALLOC BTF accesses past object bounds, arena frees below the arena base, and mixed arena and ordinary atomic paths (Yiyang Chen) - Fix the trampoline handling of 128-bit arguments and of return values larger than 8 bytes (Yonghong Song) - Ensure that any fault prone load is rewritten with exception table handling, and fix the arena load-acquire and atomic fetch handling in the x86, arm64, riscv and s390 JITs (Daniel Borkmann) - Many more fixes and cleanups across the verifier, arena, trampolines, sockmap, cgroup, ring buffer, x86/arm64/riscv/s390 JITs, libbpf, bpftool, resolve_btfids and selftests" * tag 'bpf-next-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (373 commits) selftests/bpf: Add tests for a store on a fault prone qdisc pointer selftests/bpf: Add tests for fault prone loads out of RCU pointers selftests/bpf: Add tests for pointer type merge at a shared load selftests/bpf: Remove duplicate copies of the arena spinlock qnodes selftests/bpf: Retry stat generation in cgroup_iter_memcg selftests/bpf: Test pseudo-function policy diagnostics bpf: Distinguish function references in policy diagnostics bpf: Preserve source attribution without source text selftests/bpf: Test kfunc argument diagnostics bpf: Correct kfunc argument diagnostics bpf: Use canonical stack argument names in diagnostics bpf: Preserve R0 lineage across helper calls selftests/bpf: Exercise negative optlen in cgroup getsockopt hook bpf: Reject negative optlen in cgroup getsockopt hook selftests/bpf: tc_tunnel - validate decap GSO and encapsulation state bpf: Clear decap state on skb_adjust_room shrink path bpf: Allow new DECAP flags and add guard rails bpf: Add BPF_F_ADJ_ROOM_DECAP_* flags for tunnel decapsulation bpf: Refactor masks for ADJ_ROOM flags and encap validation bpf: Name the enum for BPF_FUNC_skb_adjust_room flags ... 
Pull bpf updates from Daniel Borkmann:
 "Major changes:

   - Redesign the verifier error reporting: failures now carry source
     and instruction annotations along with the causal event history
     that led to them, making program rejections far easier to debug and
     repair (Kumar Kartikeya Dwivedi)

   - Add arena argument support to kfuncs and struct_ops through the new
     __arena and __arena__nullable suffixes (Tejun Heo, Puranjay Mohan,
     Kumar Kartikeya Dwivedi, Ihor Solodrai)

   - Signed BPF program loader rework to accommodate both BPF and
     security community needs where the kernel runs the signature
     verification at BPF_PROG_LOAD time before the LSM admission hook
     (Daniel Borkmann)

   - Add a set of ksock kfuncs which let BPF LSM and syscall programs
     create, connect and send on UDP sockets in order to emit telemetry
     data (Mahe Tardy)

   - Unify helper and kfunc call argument verification and classify
     kfunc arguments purely from BTF into a generated bpf_func_proto
     which is computed once at add-call time (Amery Hung)

  Other features and fixes:

   - Enable EXECMEM_ROX_CACHE for BPF allocations on x86 (Mike Rapoport)

   - Add bidirectional VLAN support to bpf_fib_lookup() through the new
     BPF_FIB_LOOKUP_VLAN and BPF_FIB_LOOKUP_VLAN_INPUT flags (Avinash
     Duduskar)

   - Infer zext_dst from static register liveness analysis to fix 32-bit
     zero-extension semantics, and remove the artificial limitations on
     pointer types eligible for spilling (Eduard Zingerman)

   - Inline the numeric open-coded iterator kfuncs so that bpf_for()
     loops no longer pay a kfunc call on every iteration (Puranjay
     Mohan)

   - Add an arena-based bitmap data structure to libarena along with
     serial and parallel selftests (Emil Tsalapatis)

   - Teach resolve_btfids to discover kfuncs from the kernel's BTF ID
     sets and to emit kfunc BTF decl tags, reducing the kernel build's
     dependency on pahole features (Ihor Solodrai)

   - Add BPF_F_ADJ_ROOM_DECAP_* flags to bpf_skb_adjust_room() so that
     tunnel decapsulation can update the GSO and encapsulation state of
     the skb (Nick Hudson)

   - Fix the ring buffer pending_pos walk and the available-data
     accounting on 32-bit position wrap (Israel Téllez García)

   - Add memory usage accounting for arena maps and fix an mmap_lock
     deadlock on arena lock failure (Jiayuan Chen)

   - Add tracing_multi link info support to the kernel UAPI and bpftool,
     and refactor the stack map code to run with preemption disabled
     (Jiri Olsa)

   - Support BPF_F_EGRESS in bpf_redirect_peer() to emit the skb in the
     egress direction of the target's peer device (Jordan Rife)

   - Add a KF_SPINLOCK_SAFE kfunc flag so that providers, in particular
     modules, can declare kfuncs safe to call under bpf_spin_lock
     instead of relying on the verifier's hard-coded allowlist (Kaitao
     Cheng)

   - Introduce global percpu data for BPF programs with libbpf probing
     and bpftool skeleton support, and stop exposing uninitialized
     kernel heap memory when copying per-CPU map values (Leon Hwang)

   - Add s390 JIT support for load-acquire and store-release
     instructions (Maxim Khmelevskii)

   - Fix a CFI mismatch in the task work callback and an arm64 KASAN
     false positive after bpf_throw() (Mykyta Yatsenko)

   - Reject writes through untrusted BTF pointers and bound the
     rdonly/rdwr_buf_size kfunc arguments (Nicholas Dudar)

   - Invalidate RCU pointers only after the final spin unlock and
     account for preempt and IRQ disabled regions as overlapping RCU
     protection (Ning Ding)

   - Support mixing bpf2bpf calls and tail calls on RV64, add signed
     operations and 32-bit atomics to the RV32 JIT, and add timed
     may_goto support (Pu Lehui, Kuan-Wei Chiu, Feng Jiang)

   - Fix a use-after-free on mm_struct in bpf_find_vma() for foreign
     tasks and an mmap_lock leak in the irq_work path (Sanghyun Park)

   - Populate mmap-able BPF array map memory lazily which makes mmap()
     O(1) instead of proportional to the map size (Song Liu)

   - Introduce a jit_required flag and reject programs with inlined
     helpers when no JIT is available, where the interpreter would
     otherwise jump into an invalid address (Tiezhu Yang)

   - Fix the x86 JIT per-CPU address resolution into an extended
     register where the REX prefix dropped the high destination register
     bit (Vineet Gupta)

   - Reject MEM_ALLOC BTF accesses past object bounds, arena frees below
     the arena base, and mixed arena and ordinary atomic paths (Yiyang
     Chen)

   - Fix the trampoline handling of 128-bit arguments and of return
     values larger than 8 bytes (Yonghong Song)

   - Ensure that any fault prone load is rewritten with exception table
     handling, and fix the arena load-acquire and atomic fetch handling
     in the x86, arm64, riscv and s390 JITs (Daniel Borkmann)

   - Many more fixes and cleanups across the verifier, arena,
     trampolines, sockmap, cgroup, ring buffer, x86/arm64/riscv/s390
     JITs, libbpf, bpftool, resolve_btfids and selftests"

* tag 'bpf-next-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (373 commits)
  selftests/bpf: Add tests for a store on a fault prone qdisc pointer
  selftests/bpf: Add tests for fault prone loads out of RCU pointers
  selftests/bpf: Add tests for pointer type merge at a shared load
  selftests/bpf: Remove duplicate copies of the arena spinlock qnodes
  selftests/bpf: Retry stat generation in cgroup_iter_memcg
  selftests/bpf: Test pseudo-function policy diagnostics
  bpf: Distinguish function references in policy diagnostics
  bpf: Preserve source attribution without source text
  selftests/bpf: Test kfunc argument diagnostics
  bpf: Correct kfunc argument diagnostics
  bpf: Use canonical stack argument names in diagnostics
  bpf: Preserve R0 lineage across helper calls
  selftests/bpf: Exercise negative optlen in cgroup getsockopt hook
  bpf: Reject negative optlen in cgroup getsockopt hook
  selftests/bpf: tc_tunnel - validate decap GSO and encapsulation state
  bpf: Clear decap state on skb_adjust_room shrink path
  bpf: Allow new DECAP flags and add guard rails
  bpf: Add BPF_F_ADJ_ROOM_DECAP_* flags for tunnel decapsulation
  bpf: Refactor masks for ADJ_ROOM flags and encap validation
  bpf: Name the enum for BPF_FUNC_skb_adjust_room flags
  ...
selftests/bpf: Add bpf_for() benchmark 2026-08-05T17:44:35+00:00 Puranjay Mohan puranjay@kernel.org 2026-08-04T13:45:58+00:00 5a1de41147b0b604e49144671937e9178bf5b5c1 Add a bpf_for() benchmark modelled on bench_bpf_loop so the per-iteration iterator cost can be measured and compared against bpf_loop. It runs an empty bpf_for(i, 0, nr_loops) loop 1000 times per trigger and accounts nr_loops hits per outer iteration: $ ./bench -p 1 --nr_loops 1000 bpf-for Signed-off-by: Puranjay Mohan <puranjay@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20260804134601.2305303-7-puranjay@kernel.org 
Add a bpf_for() benchmark modelled on bench_bpf_loop so the per-iteration
iterator cost can be measured and compared against bpf_loop. It runs an
empty bpf_for(i, 0, nr_loops) loop 1000 times per trigger and accounts
nr_loops hits per outer iteration:

  $ ./bench -p 1 --nr_loops 1000 bpf-for

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260804134601.2305303-7-puranjay@kernel.org
selftests/bpf: Change uprobe/usdt trigger bench code to use nop10 2026-07-04T08:57:13+00:00 Jiri Olsa jolsa@kernel.org 2026-07-03T11:49:14+00:00 b2cf7c41e4f12f76f6f7b6a21194367cf8d3e7b9 Changing uprobe/usdt trigger bench code to use nop10 instead of nop5. Also changing run_bench_uprobes.sh to use nop10 triggers. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com> Link: https://patch.msgid.link/20260703114917.238144-11-jolsa@kernel.org 
Changing uprobe/usdt trigger bench code to use nop10 instead
of nop5. Also changing run_bench_uprobes.sh to use nop10 triggers.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>
Link: https://patch.msgid.link/20260703114917.238144-11-jolsa@kernel.org
selftests/bpf: Change uprobe syscall tests to use nop10 2026-07-04T08:57:04+00:00 Jiri Olsa jolsa@kernel.org 2026-07-03T11:49:13+00:00 ec0596a0208318d66be7ae946c8964c41dccaa8d Optimized uprobes are now on top of 10-bytes nop instructions, reflect that in existing tests. Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com> Link: https://patch.msgid.link/20260703114917.238144-10-jolsa@kernel.org 
Optimized uprobes are now on top of 10-bytes nop instructions,
reflect that in existing tests.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>
Link: https://patch.msgid.link/20260703114917.238144-10-jolsa@kernel.org
selftests/bpf: Add resizable hashmap to benchmarks 2026-06-05T15:00:08+00:00 Mykyta Yatsenko yatsenko@meta.com 2026-06-05T11:41:29+00:00 84f7a49e76ec8e0a1e18f3758e89800f8cf8cfc6 Support resizable hashmap in BPF map benchmarks. 1. LOOKUP (single producer, M events/sec) key | max | nr | htab | rhtab | ratio | delta ----+-----+-------+---------+---------+-------+------- 8 | 1K | 750 | 99.85 | 81.92 | 0.82x | -18 % 8 | 1K | 1K | 100.71 | 80.19 | 0.80x | -20 % 8 | 1M | 750K | 23.37 | 72.09 | 3.08x | +208 % 8 | 1M | 1M | 13.39 | 53.72 | 4.01x | +301 % 32 | 1K | 750 | 51.57 | 42.78 | 0.83x | -17 % 32 | 1K | 1K | 50.81 | 45.83 | 0.90x | -10 % 32 | 1M | 750K | 11.27 | 15.29 | 1.36x | +36 % 32 | 1M | 1M | 7.32 | 8.75 | 1.19x | +19 % 256 | 1K | 750 | 7.58 | 7.88 | 1.04x | +4 % 256 | 1K | 1K | 7.43 | 7.81 | 1.05x | +5 % 256 | 1M | 750K | 3.69 | 4.27 | 1.16x | +16 % 256 | 1M | 1M | 2.60 | 3.12 | 1.20x | +20 % Pattern: * Small map (1K): htab wins for 8 / 32 byte keys by 10-20% * Large map (1M): rhtab wins everywhere, up to 4x at high load factor with 8 byte keys. * Higher load factor amplifies rhtab's lead: rhtab grows the bucket array; htab stays at user-declared max. 2. FULL UPDATE (M events/sec per producer) htab per-producer: 20.33 22.02 19.27 23.61 24.18 23.17 21.07 mean 21.94 range 19.27 - 24.18 rhtab per-producer: 133.51 129.47 74.52 129.29 102.26 129.98 107.64 mean 115.24 range 74.52 - 133.51 speedup (mean): 5.25x (+425 %) In-place memcpy avoids the per-update alloc + RCU pointer swap that htab pays. 3. MEMORY value_size | htab ops/s | rhtab ops/s | htab mem | rhtab mem -----------+-------------+-------------+----------+---------- 32 B | 122.87 k/s | 133.04 k/s | 2.47 MiB | 2.49 MiB 4096 B | 64.43 k/s | 65.38 k/s | 6.74 MiB | 6.44 MiB rhtab/htab : +8 % ops, +0.8 % mem (32 B) +1 % ops, -4 % mem (4096 B) Throughput effectively tied SUMMARY * Small / well-fitting map: htab is faster (cache-friendly fixed bucket array), but only by ~10-20 %. * Large / high-load-factor map: rhtab is dramatically faster (1.2x to 4x) because rhashtable resizes to keep the load factor sane while htab stays stuck at user-declared max. * Update-heavy workloads: rhtab is ~5x faster per producer via in-place memcpy. * Memory benchmark: effectively on par. Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com> Link: https://lore.kernel.org/r/20260605-rhash-v7-12-5b8e05f8630d@meta.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> 
Support resizable hashmap in BPF map benchmarks.

1. LOOKUP  (single producer, M events/sec)

  key | max | nr    |    htab |   rhtab | ratio | delta
  ----+-----+-------+---------+---------+-------+-------
    8 |  1K |   750 |   99.85 |   81.92 | 0.82x |  -18 %
    8 |  1K |    1K |  100.71 |   80.19 | 0.80x |  -20 %
    8 |  1M |  750K |   23.37 |   72.09 | 3.08x | +208 %
    8 |  1M |    1M |   13.39 |   53.72 | 4.01x | +301 %
   32 |  1K |   750 |   51.57 |   42.78 | 0.83x |  -17 %
   32 |  1K |    1K |   50.81 |   45.83 | 0.90x |  -10 %
   32 |  1M |  750K |   11.27 |   15.29 | 1.36x |  +36 %
   32 |  1M |    1M |    7.32 |    8.75 | 1.19x |  +19 %
  256 |  1K |   750 |    7.58 |    7.88 | 1.04x |   +4 %
  256 |  1K |    1K |    7.43 |    7.81 | 1.05x |   +5 %
  256 |  1M |  750K |    3.69 |    4.27 | 1.16x |  +16 %
  256 |  1M |    1M |    2.60 |    3.12 | 1.20x |  +20 %

Pattern:
  * Small map (1K): htab wins for 8 / 32 byte keys by 10-20%
  * Large map (1M): rhtab wins everywhere, up to 4x at high load
    factor with 8 byte keys.
  * Higher load factor amplifies rhtab's lead: rhtab grows the
    bucket array; htab stays at user-declared max.

2. FULL UPDATE  (M events/sec per producer)

  htab  per-producer:
    20.33   22.02   19.27   23.61   24.18   23.17   21.07
    mean  21.94   range  19.27 - 24.18

  rhtab per-producer:
   133.51  129.47   74.52  129.29  102.26  129.98  107.64
    mean 115.24   range  74.52 - 133.51

  speedup (mean): 5.25x   (+425 %)

In-place memcpy avoids the per-update alloc + RCU pointer swap
that htab pays.

3. MEMORY

  value_size |  htab ops/s | rhtab ops/s | htab mem | rhtab mem
  -----------+-------------+-------------+----------+----------
       32 B  |  122.87 k/s |  133.04 k/s | 2.47 MiB | 2.49 MiB
     4096 B  |   64.43 k/s |   65.38 k/s | 6.74 MiB | 6.44 MiB
  rhtab/htab :  +8 % ops, +0.8 % mem   (32 B)
                +1 % ops,  -4  % mem (4096 B)

Throughput effectively tied

SUMMARY

  * Small / well-fitting map: htab is faster (cache-friendly
    fixed bucket array), but only by ~10-20 %.
  * Large / high-load-factor map: rhtab is dramatically faster
    (1.2x to 4x) because rhashtable resizes to keep the load
    factor sane while htab stays stuck at user-declared max.
  * Update-heavy workloads: rhtab is ~5x faster per producer
    via in-place memcpy.
  * Memory benchmark: effectively on par.

Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
Link: https://lore.kernel.org/r/20260605-rhash-v7-12-5b8e05f8630d@meta.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
selftests/bpf: Filter timing outliers with IQR in batch-timing library 2026-05-20T16:25:47+00:00 Puranjay Mohan puranjay@kernel.org 2026-05-20T13:33:32+00:00 abac8acb633a9448369d658889ac2bcfbd96f54b System noise (timer interrupts, scheduling) can inflate the reported stddev. tcp-v4-syn showed stddev 37.86 ns without filtering vs 0.16 ns with filtering on the same run data. Filter samples outside [Q1 - 1.5*IQR, Q3 + 1.5*IQR] before computing statistics. Scenarios with genuinely wide distributions have large IQR so the fences stay wide and the filter has minimal effect. Signed-off-by: Puranjay Mohan <puranjay@kernel.org> Link: https://lore.kernel.org/r/20260520133338.3392667-4-puranjay@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org> 
System noise (timer interrupts, scheduling) can inflate the reported
stddev.  tcp-v4-syn showed stddev 37.86 ns without filtering vs
0.16 ns with filtering on the same run data.

Filter samples outside [Q1 - 1.5*IQR, Q3 + 1.5*IQR] before computing
statistics.  Scenarios with genuinely wide distributions have large IQR
so the fences stay wide and the filter has minimal effect.

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Link: https://lore.kernel.org/r/20260520133338.3392667-4-puranjay@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
selftests/bpf: Fix expired UDP LRU entries in XDP LB benchmark 2026-05-20T16:25:46+00:00 Puranjay Mohan puranjay@kernel.org 2026-05-20T13:33:31+00:00 12e896b9794bbd88f56aeac2a5807ae8d4bb5ad8 populate_lru() zero-initializes atime: struct real_pos_lru lru = { .pos = real_idx }; connection_table_lookup() treats UDP entries with cur_time - atime > 30s as expired, so every pre-populated entry expires immediately. Calibration masks this on the CPU it runs on, but if validation migrates to another CPU: [udp-v4-lru-hit] COUNTER FAIL: LRU misses=1, expected 0 Initialize atime from CLOCK_MONOTONIC for UDP flows. Fixes: a4b5ba8187cb ("selftests/bpf: Add XDP load-balancer benchmark driver") Signed-off-by: Puranjay Mohan <puranjay@kernel.org> Link: https://lore.kernel.org/r/20260520133338.3392667-3-puranjay@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org> 
populate_lru() zero-initializes atime:

    struct real_pos_lru lru = { .pos = real_idx };

connection_table_lookup() treats UDP entries with
cur_time - atime > 30s as expired, so every pre-populated entry
expires immediately.  Calibration masks this on the CPU it runs on,
but if validation migrates to another CPU:

    [udp-v4-lru-hit] COUNTER FAIL: LRU misses=1, expected 0

Initialize atime from CLOCK_MONOTONIC for UDP flows.

Fixes: a4b5ba8187cb ("selftests/bpf: Add XDP load-balancer benchmark driver")
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Link: https://lore.kernel.org/r/20260520133338.3392667-3-puranjay@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
selftests/bpf: Add XDP load-balancer benchmark run script 2026-05-11T22:25:24+00:00 Puranjay Mohan puranjay@kernel.org 2026-04-27T23:23:04+00:00 51312b6360a92e7bccd7b05b028ba2066b093305 Add a convenience script that runs all 24 XDP load-balancer scenarios and formats the results as a table with median, stddev, and p99 columns. ./benchs/run_bench_xdp_lb.sh Signed-off-by: Puranjay Mohan <puranjay@kernel.org> Link: https://lore.kernel.org/r/20260427232313.1582588-8-puranjay@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org> 
Add a convenience script that runs all 24 XDP load-balancer scenarios
and formats the results as a table with median, stddev, and p99
columns.

  ./benchs/run_bench_xdp_lb.sh

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Link: https://lore.kernel.org/r/20260427232313.1582588-8-puranjay@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
selftests/bpf: Add XDP load-balancer benchmark driver 2026-05-11T22:25:24+00:00 Puranjay Mohan puranjay@kernel.org 2026-04-27T23:23:03+00:00 a4b5ba8187cb184aacac4ac8c86b4ef4821a4aa6 Wire up the userspace side of the XDP load-balancer benchmark. 24 scenarios cover the full code-path matrix: TCP/UDP, IPv4/IPv6, cross-AF encap, LRU hit/miss/diverse/cold, consistent-hash bypass, SYN/RST flag handling, and early exits (unknown VIP, non-IP, ICMP, fragments, IP options). Before benchmarking each scenario validates correctness: the output packet is compared byte-for-byte against a pre-built expected packet and BPF map counters are checked against the expected values. Usage: sudo ./bench -a -w3 -p1 xdp-lb --scenario tcp-v4-lru-hit sudo ./bench xdp-lb --list-scenarios Signed-off-by: Puranjay Mohan <puranjay@kernel.org> Link: https://lore.kernel.org/r/20260427232313.1582588-7-puranjay@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org> 
Wire up the userspace side of the XDP load-balancer benchmark.

24 scenarios cover the full code-path matrix: TCP/UDP, IPv4/IPv6,
cross-AF encap, LRU hit/miss/diverse/cold, consistent-hash bypass,
SYN/RST flag handling, and early exits (unknown VIP, non-IP, ICMP,
fragments, IP options).

Before benchmarking each scenario validates correctness: the output
packet is compared byte-for-byte against a pre-built expected packet
and BPF map counters are checked against the expected values.

Usage:
  sudo ./bench -a -w3 -p1 xdp-lb --scenario tcp-v4-lru-hit
  sudo ./bench xdp-lb --list-scenarios

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Link: https://lore.kernel.org/r/20260427232313.1582588-7-puranjay@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
selftests/bpf: Add bpf-nop benchmark for timing overhead baseline 2026-05-11T22:25:24+00:00 Puranjay Mohan puranjay@kernel.org 2026-04-27T23:23:00+00:00 dcf11479c2a8d3520953e8366f587ec2a36505a8 Add a minimal benchmark that measures the overhead of the batch-timing infrastructure itself. The BPF program runs an empty BENCH_BPF_LOOP body (~1.5-2 ns/op), establishing the floor cost that all timing-library benchmarks include. [root@virtme-ng tools/testing/selftests/bpf]# sudo ./bench -a -p8 bpf-nop Setting up benchmark 'bpf-nop'... Benchmark 'bpf-nop' started. bpf-nop: median 1.82 ns/op, stddev 0.01, p99 1.86 (1754 samples) Signed-off-by: Puranjay Mohan <puranjay@kernel.org> Link: https://lore.kernel.org/r/20260427232313.1582588-4-puranjay@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org> 
Add a minimal benchmark that measures the overhead of the batch-timing
infrastructure itself. The BPF program runs an empty BENCH_BPF_LOOP body
(~1.5-2 ns/op), establishing the floor cost that all timing-library
benchmarks include.

[root@virtme-ng tools/testing/selftests/bpf]# sudo ./bench -a -p8 bpf-nop
Setting up benchmark 'bpf-nop'...
Benchmark 'bpf-nop' started.
bpf-nop: median 1.82 ns/op, stddev 0.01, p99 1.86 (1754 samples)

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Link: https://lore.kernel.org/r/20260427232313.1582588-4-puranjay@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>