On XIP kernels it makes sense to have exception vectors and fast
exception handlers together (in a fast memory). In addition, with MTD
XIP support both vectors and fast exception handlers must be outside of
the FLASH.

Add section .exception.text and move fast exception handlers to it.
Put it together with vectors when vectors are outside of the .text.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

Now that xtensa assembly sources are compiled with -mlongcalls let the
assembler and linker relax call instructions into l32r + callx where
needed. This change makes the code cleaner and potentially a bit faster.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

vmlinux.lds.S doesn't do anything special with literals, so instead of
keeping them separate put them into the corresponding text sections.
Drop explicit .literal sections from the vmlinux.lds.S, use standard
section macros. Mark literal pool locations in the assembly sources.
Unfortunately assembler doesn't put literals into .init sections and
external libgcc may still have .literal sections, so sed transformation
to the linker script is still needed.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

With implementation of data breakpoints debug exceptions raised when
PS.EXCM is set need to be handled, e.g. window overflow code can write
to watched userspace address. Currently debug exception handler uses
EXCSAVE and DEPC SRs to save temporary registers, but DEPC may not be
available when PS.EXCM is set and more space will be needed to save
additional state.
Reorganize debug context: create per-CPU structure debug_table instance
and store its address in the EXCSAVE<debug level> instead of
debug_exception function address. Expand this structure when more save
space is needed.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

There are multiple factors adding to the issue in different
configurations:

- commit 17290231df16eeee ("xtensa: add fixup for double exception raised
  in window overflow") added function window_overflow_restore_a0_fixup to
  double exception vector overlapping reset vector location of secondary
  processor cores.
- on MMUv2 cores RESET_VECTOR1_VADDR may point to uncached kernel memory
  making code overlapping depend on cache type and size, so that without
  cache or with WT cache reset vector code overwrites double exception
  code, making issue even harder to detect.
- on MMUv3 cores RESET_VECTOR1_VADDR may point to unmapped area, as
  MMUv3 cores change virtual address map to match MMUv2 layout, but
  reset vector virtual address is given for the original MMUv3 mapping.
- physical memory region of the secondary reset vector is not reserved
  in the physical memory map, and thus may be allocated and overwritten
  at arbitrary moment.

Fix it as follows:

- move window_overflow_restore_a0_fixup code to .text section.
- define RESET_VECTOR1_VADDR so that it points to reset vector in the
  cacheable MMUv2 map for cores with MMU.
- reserve reset vector region in the physical memory map. Drop separate
  literal section and build mxhead.S with text section literals.

Cc: <stable@vger.kernel.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

In case perf IRQ is the highest of the medium-level IRQs, and is alone
on its level, it may be treated as NMI:
- LOCKLEVEL is defined to be one level less than EXCM level,
- IRQ masking never lowers current IRQ level,
- new fake exception cause code, EXCCAUSE_MAPPED_NMI is assigned to that
  IRQ; new second level exception handler, do_nmi, assigned to it
  handles it as NMI,
- atomic operations in configurations without s32c1i still need to mask
  all interrupts.

Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

Double exceptions that happen during register window overflow/underflow
are handled in the topmost stack frame, as if it was the only exception
that occured. However unaligned access exception handler is special
because it needs to analyze instruction that caused the exception, but
the userspace instruction that triggered window exception is completely
irrelevant. Unaligned data access is rather normal in the generic
userspace code, but stack pointer manipulation must always be done by
architecture-aware code and thus unaligned stack means a serious problem
anyway.
Use the default unaligned access handler that raises SIGBUS in case
of unaligned access in window overflow/underflow handler.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

There are two FIXMEs in the double exception handler 'for the extremely
unlikely case'. This case gets hit by gcc during kernel build once in
a few hours, resulting in an unrecoverable exception condition.

Provide missing fixup routine to handle this case. Double exception
literals now need 8 more bytes, add them to the linker script.

Also replace bbsi instructions with bbsi.l as we're branching depending
on 8th and 7th LSB-based bits of exception address.

This may be tested by adding the explicit DTLB invalidation to window
overflow handlers, like the following:

    --- a/arch/xtensa/kernel/vectors.S
    +++ b/arch/xtensa/kernel/vectors.S
    @@ -592,6 +592,14 @@ ENDPROC(_WindowUnderflow4)
     ENTRY_ALIGN64(_WindowOverflow8)

    	s32e	a0, a9, -16
    +	bbsi.l	a9, 31, 1f
    +	rsr	a0, ccount
    +	bbsi.l	a0, 4, 1f
    +	pdtlb	a0, a9
    +	idtlb	a0
    +	movi	a0, 9
    +	idtlb	a0
    +1:
    	l32e    a0, a1, -12
    	s32e    a2, a9,  -8
    	s32e    a1, a9, -12

Cc: stable@vger.kernel.org
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>

The original implementation could clobber registers under certain conditions.

The Xtensa processor architecture uses windowed registers and the original
implementation was using a4 as a temporary register, which under certain
conditions could be register a0 of the oldest window frame, and didn't always
restore the content correctly.

By moving the _spill_registers routine inside the fast system call, it frees
up one more register (the return address is not required anymore) for the
spill routine.

Signed-off-by: Chris Zankel <chris@zankel.net>

Based on the SMP patch by Joe Taylor and subsequent fixes.
Preserve exception table pointer (normally stored in excsave1 SR) as it
cannot be easily restored in SMP environment.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Chris Zankel <chris@zankel.net>