KPROBE_HIT_SS and KPROBE_REENTER are two types of fatal recursions that
can not be safely recovered in kprobes.

KPROBE_HIT_SS means that a kprobe is hit during single-stepping. At
this point, the architecture-specific single-step context is already
active. Nested single-stepping would corrupt the state, as the kprobe
control block (kcb) and hardware registers cannot safely store multiple
levels of stepping state.

KPROBE_REENTER means that a third-level recursion occurs when a probe
is hit while the system is already handling a nested probe (second-
level). The kcb only provides a single slot (prev_kprobe) to backup the
state. When a third probe is hit, there is no more space to save the
state without corrupting the first-level backup.

Kprobes work by replacing instructions with breakpoints. In order to
execute the original instruction and continue, it must be moved to a
temporary "single-step" slot. Since there is no backup space left to
set up this slot safely, the CPU would be forced to return to the same
original breakpoint address, triggering an endless loop.

Currently, the code only prints a warning and returns. This leads to
an infinite re-entry loop as the CPU repeatedly hits the same trap and
a "stuck" CPU core because preemption was disabled at the start of the
handler and never re-enabled in this early return path.

Fix the logic by:
1. Merging KPROBE_HIT_SS and KPROBE_REENTER cases, as both represent
   fatal recursions that cannot be safely recovered.
2. Replacing WARN_ON_ONCE() with BUG() to terminate the system. This
   aligns LoongArch with other architectures (x86, arm64, riscv) and
   prevents stack overflow while providing diagnostic information.

Fixes: 6d4cc40fb5f5 ("LoongArch: Add kprobes support")
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>

On SMP systems, kprobe handlers would occasionally fail to execute on
certain CPU cores. The issue is hard to reproduce and typically occurs
randomly under high system load.

The root cause is a software-side instruction hazard. According to the
LoongArch Reference Manual, while the cache coherency is maintained by
hardware, software must explicitly use the "IBAR" instruction to ensure
the instruction fetch unit (IFU) observes the effects of recent stores.

The current arch_arm_kprobe() and arch_disarm_kprobe() only execute the
"IBAR" barrier (via flush_insn_slot -> local_flush_icache_range) on the
local CPU. This leaves a vulnerable window where remote CPU cores may
continue executing stale instructions from their pipelines or prefetch
buffers, as they have not executed an "IBAR" since the code modification.

Switch to larch_insn_text_copy() to fix this:
1. Synchronization: It uses stop_machine_cpuslocked() to synchronize all
   online CPUs, ensuring no CPU is executing the target code area during
   modification.
2. Visibility: By passing cpu_online_mask to stop_machine_cpuslocked(),
   the callback text_copy_cb() is executed on all online cores. Each CPU
   core invokes local_flush_icache_range() to execute "IBAR", clearing
   instruction hazards system-wide and ensuring the "break" instruction
   is visible to the fetch units of all cores.
3. Robustness: It properly manages memory write permissions (ROX/RW) for
   the kernel text segment during patching, ensuring compatibility with
   CONFIG_STRICT_KERNEL_RWX.

Cc: <stable@vger.kernel.org>  # 6.18+
Fixes: 6d4cc40fb5f5 ("LoongArch: Add kprobes support")
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>

Validate the relocation address against the initrd region specified via
"initrd=" or "initrdmem=" on the command line. Reject relocation targets
that overlap the initrd to prevent memory corruption during early boot.

Acked-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: WANG Rui <wangrui@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>

When the kernel is relocated during early boot (efistub or kexec_file),
a randomized load address may has already been selected and applied. In
this case, performing KASLR again in relocate.c is unnecessary.

Note: strictly-defined KASLR means the kernel's final runtime address
has a random offset from the kernel's load address, which is implemented
in relocate.c; broadly-defined KALSR means the kernel's final runtime
address has a random offset from the kernel's link address (a.k.a.
VMLINUX_LOAD_ADDRESS), which also include the efistlub implementation,
kexec_file implementation and QEMU direct kernel boot. kaslr_disabled()
return true only means strictly-defined KASLR is disabled.

Acked-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: WANG Rui <wangrui@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>

Pull LoongArch updates from Huacai Chen:

 - Adjust build infrastructure for 32BIT/64BIT

 - Add HIGHMEM (PKMAP and FIX_KMAP) support

 - Show and handle CPU vulnerabilites correctly

 - Batch the icache maintenance for jump_label

 - Add more atomic instructions support for BPF JIT

 - Add more features (e.g. fsession) support for BPF trampoline

 - Some bug fixes and other small changes

* tag 'loongarch-7.1' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson: (21 commits)
  selftests/bpf: Enable CAN_USE_LOAD_ACQ_STORE_REL for LoongArch
  LoongArch: BPF: Add fsession support for trampolines
  LoongArch: BPF: Introduce emit_store_stack_imm64() helper
  LoongArch: BPF: Support up to 12 function arguments for trampoline
  LoongArch: BPF: Support small struct arguments for trampoline
  LoongArch: BPF: Open code and remove invoke_bpf_mod_ret()
  LoongArch: BPF: Support load-acquire and store-release instructions
  LoongArch: BPF: Support 8 and 16 bit read-modify-write instructions
  LoongArch: BPF: Add the default case in emit_atomic() and rename it
  LoongArch: Define instruction formats for AM{SWAP/ADD}.{B/H} and DBAR
  LoongArch: Batch the icache maintenance for jump_label
  LoongArch: Add flush_icache_all()/local_flush_icache_all()
  LoongArch: Add spectre boundry for syscall dispatch table
  LoongArch: Show CPU vulnerabilites correctly
  LoongArch: Make arch_irq_work_has_interrupt() true only if IPI HW exist
  LoongArch: Use get_random_canary() for stack canary init
  LoongArch: Improve the logging of disabling KASLR
  LoongArch: Align FPU register state to 32 bytes
  LoongArch: Handle CONFIG_32BIT in syscall_get_arch()
  LoongArch: Add HIGHMEM (PKMAP and FIX_KMAP) support
  ...

The 8 and 16 bit read-modify-write atomic instructions amadd.{b/h} and
amswap.{b/h} were newly added in the latest LoongArch Reference Manual,
define the instruction format and check whether support via CPUCFG.

Furthermore, define the instruction format for DBAR which will be used
to support BPF load-acquire and store-release instructions.

This is preparation for later patches.

Acked-by: Hengqi Chen <hengqi.chen@gmail.com>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>

Switch to the batched version of the jump label update functions so
instruction cache maintenance is deferred until the end of the update.

Signed-off-by: Youling Tang <tangyouling@kylinos.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>

The LoongArch syscall number is directly controlled by userspace, but
does not have a array_index_nospec() boundry to prevent access past the
syscall function pointer tables.

Cc: stable@vger.kernel.org
Assisted-by: gkh_clanker_2000
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>

Most LoongArch processors are vulnerable to Spectre-V1 Proof-of-Concept
(PoC). And the generic mechanism, __user pointer sanitization, can be
used as a mitigation. This means to use array_index_nospec() to prevent
out of boundry access in syscall and other critical paths.

Implement the arch-specific cpu_show_spectre_v1() to show CPU Spectre-V1
vulnerabilites correctly.

Cc: stable@vger.kernel.org
Link: https://cc-sw.com/chinese-loongarch-architecture-evaluation-part-3-of-3/
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>

Whether KASLR is disabled is not handled in nokaslr() which is the early
param "nokaslr" setup function, but in kaslr_disabled(). However, the
logging was previously done in nokaslr() and lack detail. So we move the
logging to the right place and add more specific infomation about why it
is disabled.

Suggested-by: Wentao Guan <guanwentao@uniontech.com>
Signed-off-by: Yuqian Yang <yangyuqian@uniontech.com>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>