linux.git/arch/mips/net, branch v5.16

mips, bpf: Fix reference to non-existing Kconfig symbol

2021-11-30T16:19:36+00:00

The Kconfig symbol for R10000 ll/sc errata workaround in the MIPS JIT was
misspelled, causing the workaround to not take effect when enabled.

Fixes: 72570224bb8f ("mips, bpf: Add JIT workarounds for CPU errata")
Reported-by: Lukas Bulwahn 
Signed-off-by: Johan Almbladh 
Signed-off-by: Daniel Borkmann 
Link: https://lore.kernel.org/bpf/20211130160824.3781635-1-johan.almbladh@anyfinetworks.com

bpf, mips: Fix comment on tail call count limiting

2021-10-11T13:29:38+00:00

In emit_tail_call() of bpf_jit_comp32.c, "blez t2" (t2 <= 0) is
not consistent with the comment "t2 < 0", update the comment to
keep consistency.

Signed-off-by: Tiezhu Yang 
Signed-off-by: Daniel Borkmann 
Acked-by: Johan Almbladh 
Link: https://lore.kernel.org/bpf/1633915150-13220-3-git-send-email-yangtiezhu@loongson.cn

bpf, mips: Clean up config options about JIT

2021-10-11T13:29:37+00:00

The config options MIPS_CBPF_JIT and MIPS_EBPF_JIT are useless, remove
them in arch/mips/Kconfig, and then modify arch/mips/net/Makefile.

Signed-off-by: Tiezhu Yang 
Signed-off-by: Daniel Borkmann 
Acked-by: Johan Almbladh 
Link: https://lore.kernel.org/bpf/1633915150-13220-2-git-send-email-yangtiezhu@loongson.cn

mips, bpf: Optimize loading of 64-bit constants

2021-10-07T21:51:29+00:00

This patch shaves off a few instructions when loading sparse 64-bit
constants to register. The change is covered by additional tests in
lib/test_bpf.c.

Signed-off-by: Johan Almbladh 
Signed-off-by: Daniel Borkmann 
Link: https://lore.kernel.org/bpf/20211007142828.634182-1-johan.almbladh@anyfinetworks.com

mips, bpf: Fix Makefile that referenced a removed file

2021-10-07T21:51:13+00:00

This patch removes a stale Makefile reference to the cBPF JIT that was
removed.

Fixes: ebcbacfa50ec ("mips, bpf: Remove old BPF JIT implementations")
Signed-off-by: Johan Almbladh 
Signed-off-by: Daniel Borkmann 
Link: https://lore.kernel.org/bpf/20211007142339.633899-1-johan.almbladh@anyfinetworks.com

mips, bpf: Remove old BPF JIT implementations

2021-10-06T19:28:34+00:00

This patch removes the old 32-bit cBPF and 64-bit eBPF JIT implementations.
They are replaced by a new eBPF implementation that supports both 32-bit
and 64-bit MIPS CPUs.

Signed-off-by: Johan Almbladh 
Signed-off-by: Daniel Borkmann 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20211005165408.2305108-8-johan.almbladh@anyfinetworks.com

mips, bpf: Enable eBPF JITs

2021-10-06T19:28:30+00:00

This patch enables the new eBPF JITs for 32-bit and 64-bit MIPS. It also
disables the old cBPF JIT to so cBPF programs are converted to use the
new JIT.

Workarounds for R4000 CPU errata are not implemented by the JIT, so the
JIT is disabled if any of those workarounds are configured.

Signed-off-by: Johan Almbladh 
Signed-off-by: Daniel Borkmann 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20211005165408.2305108-7-johan.almbladh@anyfinetworks.com

mips, bpf: Add JIT workarounds for CPU errata

2021-10-06T19:28:25+00:00

This patch adds workarounds for the following CPU errata to the MIPS
eBPF JIT, if enabled in the kernel configuration.

  - R10000 ll/sc weak ordering
  - Loongson-3 ll/sc weak ordering
  - Loongson-2F jump hang

The Loongson-2F nop errata is implemented in uasm, which the JIT uses,
so no additional mitigations are needed for that.

Signed-off-by: Johan Almbladh 
Signed-off-by: Daniel Borkmann 
Signed-off-by: Andrii Nakryiko 
Reviewed-by: Jiaxun Yang 
Link: https://lore.kernel.org/bpf/20211005165408.2305108-6-johan.almbladh@anyfinetworks.com

mips, bpf: Add new eBPF JIT for 64-bit MIPS

2021-10-06T19:28:20+00:00

This is an implementation on of an eBPF JIT for 64-bit MIPS III-V and
MIPS64r1-r6. It uses the same framework introduced by the 32-bit JIT.

Signed-off-by: Johan Almbladh 
Signed-off-by: Daniel Borkmann 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20211005165408.2305108-5-johan.almbladh@anyfinetworks.com

mips, bpf: Add eBPF JIT for 32-bit MIPS

2021-10-06T19:28:14+00:00

This is an implementation of an eBPF JIT for 32-bit MIPS I-V and MIPS32.
The implementation supports all 32-bit and 64-bit ALU and JMP operations,
including the recently-added atomics. 64-bit div/mod and 64-bit atomics
are implemented using function calls to math64 and atomic64 functions,
respectively. All 32-bit operations are implemented natively by the JIT,
except if the CPU lacks ll/sc instructions.

Register mapping
================
All 64-bit eBPF registers are mapped to native 32-bit MIPS register pairs,
and does not use any stack scratch space for register swapping. This means
that all eBPF register data is kept in CPU registers all the time, and
this simplifies the register management a lot. It also reduces the JIT's
pressure on temporary registers since we do not have to move data around.

Native register pairs are ordered according to CPU endiannes, following
the O32 calling convention for passing 64-bit arguments and return values.
The eBPF return value, arguments and callee-saved registers are mapped to
their native MIPS equivalents.

Since the 32 highest bits in the eBPF FP (frame pointer) register are
always zero, only one general-purpose register is actually needed for the
mapping. The MIPS fp register is used for this purpose. The high bits are
mapped to MIPS register r0. This saves us one CPU register, which is much
needed for temporaries, while still allowing us to treat the R10 (FP)
register just like any other eBPF register in the JIT.

The MIPS gp (global pointer) and at (assembler temporary) registers are
used as internal temporary registers for constant blinding. CPU registers
t6-t9 are used internally by the JIT when constructing more complex 64-bit
operations. This is precisely what is needed - two registers to store an
operand value, and two more as scratch registers when performing the
operation.

The register mapping is shown below.

    R0 - $v1, $v0   return value
    R1 - $a1, $a0   argument 1, passed in registers
    R2 - $a3, $a2   argument 2, passed in registers
    R3 - $t1, $t0   argument 3, passed on stack
    R4 - $t3, $t2   argument 4, passed on stack
    R5 - $t4, $t3   argument 5, passed on stack
    R6 - $s1, $s0   callee-saved
    R7 - $s3, $s2   callee-saved
    R8 - $s5, $s4   callee-saved
    R9 - $s7, $s6   callee-saved
    FP - $r0, $fp   32-bit frame pointer
    AX - $gp, $at   constant-blinding
         $t6 - $t9  unallocated, JIT temporaries

Jump offsets
============
The JIT tries to map all conditional JMP operations to MIPS conditional
PC-relative branches. The MIPS branch offset field is 18 bits, in bytes,
which is equivalent to the eBPF 16-bit instruction offset. However, since
the JIT may emit more than one CPU instruction per eBPF instruction, the
field width may overflow. If that happens, the JIT converts the long
conditional jump to a short PC-relative branch with the condition
inverted, jumping over a long unconditional absolute jmp (j).

This conversion will change the instruction offset mapping used for jumps,
and may in turn result in more branch offset overflows. The JIT therefore
dry-runs the translation until no more branches are converted and the
offsets do not change anymore. There is an upper bound on this of course,
and if the JIT hits that limit, the last two iterations are run with all
branches being converted.

Tail call count
===============
The current tail call count is stored in the 16-byte area of the caller's
stack frame that is reserved for the callee in the o32 ABI. The value is
initialized in the prologue, and propagated to the tail-callee by skipping
the initialization instructions when emitting the tail call.

Signed-off-by: Johan Almbladh 
Signed-off-by: Daniel Borkmann 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20211005165408.2305108-4-johan.almbladh@anyfinetworks.com