linux-stable.git/arch/s390/kernel/fpu.c, branch linux-rolling-stable

s390: Explicitly include

2025-06-17T16:18:02+00:00

Explicitly include  in files which contain an
EXPORT_SYMBOL().

See commit a934a57a42f6 ("scripts/misc-check: check missing #include
 when W=1") for more details.

Acked-by: Alexander Gordeev 
Signed-off-by: Heiko Carstens 
Signed-off-by: Alexander Gordeev

s390/fpu: Re-add exception handling in load_fpu_state()

2024-07-31T14:30:20+00:00

With the recent rewrite of the fpu code exception handling for the
lfpc instruction within load_fpu_state() was erroneously removed.

Add it again to prevent that loading invalid floating point register
values cause an unhandled specification exception.

Fixes: 8c09871a950a ("s390/fpu: limit save and restore to used registers")
Cc: stable@vger.kernel.org
Reported-by: Aristeu Rozanski 
Tested-by: Aristeu Rozanski 
Reviewed-by: Vasily Gorbik 
Signed-off-by: Heiko Carstens 
Signed-off-by: Vasily Gorbik

s390/fpu: limit save and restore to used registers

2024-02-16T13:30:16+00:00

The first invocation of kernel_fpu_begin() after switching from user to
kernel context will save all vector registers, even if only parts of the
vector registers are used within the kernel fpu context. Given that save
and restore of all vector registers is quite expensive change the current
approach in several ways:

- Instead of saving and restoring all user registers limit this to those
  registers which are actually used within an kernel fpu context.

- On context switch save all remaining user fpu registers, so they can be
  restored when the task is rescheduled.

- Saving user registers within kernel_fpu_begin() is done without disabling
  and enabling interrupts - which also slightly reduces runtime. In worst
  case (e.g. interrupt context uses the same registers) this may lead to
  the situation that registers are saved several times, however the
  assumption is that this will not happen frequently, so that the new
  method is faster in nearly all cases.

- save_user_fpu_regs() can still be called from all contexts and saves all
  (or all remaining) user registers to a tasks ufpu user fpu save area.

Overall this reduces the time required to save and restore the user fpu
context for nearly all cases.

Signed-off-by: Heiko Carstens

s390/fpu: decrease stack usage for some cases

2024-02-16T13:30:16+00:00

The kernel_fpu structure has a quite large size of 520 bytes. In order to
reduce stack footprint introduce several kernel fpu structures with
different and also smaller sizes. This way every kernel fpu user must use
the correct variant. A compile time check verifies that the correct variant
is used.

There are several users which use only 16 instead of all 32 vector
registers. For those users the new kernel_fpu_16 structure with a size of
only 266 bytes can be used.

Signed-off-by: Heiko Carstens

s390/fpu: remove anonymous union from struct fpu

2024-02-16T13:30:16+00:00

The anonymous union within struct fpu contains a floating point register
array and a vector register array. Given that the vector register is always
present remove the floating point register array. For configurations
without vector registers save the floating point register contents within
their corresponding vector register location.

This allows to remove the union, and also to simplify ptrace and perf code.

Signed-off-by: Heiko Carstens

s390/fpu: remove regs member from struct fpu

2024-02-16T13:30:16+00:00

KVM was the only user which modified the regs pointer in struct fpu. Remove
the pointer and convert the rest of the core fpu code to directly access
the save area embedded within struct fpu.

Reviewed-by: Claudio Imbrenda 
Signed-off-by: Heiko Carstens

s390/fpu: change type of fpu mask from u32 to int

2024-02-16T13:30:15+00:00

Change type of fpu mask consistently from u32 to int. This is a
prerequisite to make the kernel fpu usage preemptible. Upcoming code
uses __atomic* ops which work with int pointers.

Reviewed-by: Claudio Imbrenda 
Signed-off-by: Heiko Carstens

s390/fpu: rename save_fpu_regs() to save_user_fpu_regs(), etc

2024-02-16T13:30:15+00:00

Rename save_fpu_regs(), load_fpu_regs(), and struct thread_struct's fpu
member to save_user_fpu_regs(), load_user_fpu_regs(), and ufpu. This way
the function and variable names reflect for which context they are supposed
to be used.

This large and trivial conversion is a prerequisite for making the kernel
fpu usage preemptible.

Reviewed-by: Claudio Imbrenda 
Signed-off-by: Heiko Carstens

s390/fpu: convert FPU CIF flag to regular TIF flag

2024-02-16T13:30:15+00:00

The FPU state, as represented by the CIF_FPU flag reflects the FPU state of
a task, not the CPU it is running on. Therefore convert the flag to a
regular TIF flag.

This removes the magic in switch_to() where a save_fpu_regs() call for the
currently (previous) running task sets the per-cpu CIF_FPU flag, which is
required to restore FPU register contents of the next task, when it returns
to user space.

Signed-off-by: Heiko Carstens

s390/fpu: convert __kernel_fpu_begin()/__kernel_fpu_end() to C

2024-02-16T13:30:15+00:00

Convert the rather large __kernel_fpu_begin()/__kernel_fpu_end() inline
assemblies to C. The C variant is much more readable, and this also allows
to get rid of the non-obvious usage of KERNEL_VXR_* constants within the
inline assemblies. E.g. "tmll %[m],6" correlates with the two bits set in
KERNEL_VXR_LOW. If the corresponding defines would be changed, the inline
assembles would break in a subtle way.

Therefore convert to C, use the proper defines, and allow the compiler to
generate code using the (hopefully) most efficient instructions.

Signed-off-by: Heiko Carstens