linux-stable.git/arch/s390/kernel, branch linux-rolling-stable Linux kernel stable tree s390/pai: Fix missing PAI counter increments under heavy load 2026-06-01T15:54:29+00:00 Thomas Richter tmricht@linux.ibm.com 2026-05-05T10:34:33+00:00 b972d9b0f8c1625e89eff6cb857a7533d753ba2f commit 99269799bf2448aebccee164df56c22a7b85b02c upstream. Machines with a larger number of CPUs and under heavy load sometimes loose PAI counter increments during recording using events -e CRYPTO_ÂLL or -e NNPA_ALL. Counting is not affected. This happens when several PAI crypto counters are incremented during the same cryptographic operation. During schedule out the functions paiXXX_sched_task() (with XXX either crypt or ext) +--> pai_have_samples() +--> pai_have_sample() +--> pai_copy() +--> pai_push_sample() are called to read out PAI counter values. In pai_copy() the current values of PAI counters are read from the PMU memory mapped page and compared to the values read during last schedule out operation, which have been saved in a backup page named PAI_SAVE_AREA(event). For each PAI counter a delta is calculated and when the delta is positive, that PAI counter was incremented by hardware. This positve delta is reported as raw data record attached to a sample. After all deltas have been calculated, the new PAI counter values are saved in the backup page PAI_SAVE_AREA(event). However this is done in pai_push_sample(), leaving a small window for missing hardware triggered updates. Here is one scenario: PAI counter idx: 0 1 2 3 4 5 6 7 .... N +---+---+---+---+---+---+---+---+ +---+ PAI counter page:| | | X | | | | | |....| Y | +---+---+---+---+---+---+---+---+ +---+ In pai_copy() each PAI counter value is read and compared to its old value. This is done in a loop. When PAI counter indexed N is read, the hardware might increment PAI counter indexed 2 again, updating its value from X to X+1. Later pai_push_sample() simply mem-copies the complete PAI counter page to a backup page and the increment of X+1 is lost, because the backup page now contains the new value. Read each PAI counter and save this value in the backup page when there is a positive delta. This omits any time window between read and store. This also reduced the work load as only modified PAI counters are saved. Cc: stable@vger.kernel.org Fixes: fe861b0c8d06 ("s390/pai: save PAI counter value page in event structure") Signed-off-by: Thomas Richter <tmricht@linux.ibm.com> Reviewed-by: Sumanth Korikkar <sumanthk@linux.ibm.com> Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit 99269799bf2448aebccee164df56c22a7b85b02c upstream.

Machines with a larger number of CPUs and under heavy load sometimes
loose PAI counter increments during recording using events
-e CRYPTO_ÂLL or -e NNPA_ALL. Counting is not affected.
This happens when several PAI crypto counters are incremented during
the same cryptographic operation.

During schedule out the functions

paiXXX_sched_task() (with XXX either crypt or ext)
+--> pai_have_samples()
   +--> pai_have_sample()
	+--> pai_copy()
	+--> pai_push_sample()

are called to read out PAI counter values.
In pai_copy() the current values of PAI counters are read from the
PMU memory mapped page and compared to the values read during last
schedule out operation, which have been saved in a backup page
named PAI_SAVE_AREA(event). For each PAI counter a delta is calculated
and when the delta is positive, that PAI counter was incremented by
hardware. This positve delta is reported as raw data record attached
to a sample.
After all deltas have been calculated, the new PAI counter values
are saved in the backup page PAI_SAVE_AREA(event). However this is
done in pai_push_sample(), leaving a small window for missing hardware
triggered updates. Here is one scenario:

  PAI counter idx:   0   1   2   3   4   5   6   7  ....  N
                   +---+---+---+---+---+---+---+---+    +---+
  PAI counter page:|   |   | X |   |   |   |   |   |....| Y |
                   +---+---+---+---+---+---+---+---+    +---+

In pai_copy() each PAI counter value is read and compared
to its old value. This is done in a loop. When PAI counter indexed
N is read, the hardware might increment PAI counter indexed 2 again,
updating its value from X to X+1.
Later pai_push_sample() simply mem-copies the complete PAI counter
page to a backup page and the increment of X+1 is lost, because the
backup page now contains the new value.

Read each PAI counter and save this value in the backup page when
there is a positive delta. This omits any time window between read
and store. This also reduced the work load as only modified PAI
counters are saved.

Cc: stable@vger.kernel.org
Fixes: fe861b0c8d06 ("s390/pai: save PAI counter value page in event structure")
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
s390/pai: Disable duplicate read of kernel PAI counter value 2026-06-01T15:54:28+00:00 Thomas Richter tmricht@linux.ibm.com 2026-04-27T05:17:19+00:00 8d176ace35089807b39eb41678f92bc094d7a709 commit 3fe7ecab1a0856aafe1026a35af1621a5c18d53f upstream. The PAI crypto counter design allows for user space and kernel space PAI counter increment recording. This is achieved by splitting the recording page in half. The upper part of the 4KB page records user space increments of PAI crypto counter and the lower half records kernel space increments. The page itself looks like: lowcore ptr ---> ++++++++++++++++++++++++ |user space area | +----------------------+ |kernel space area | ++++++++++++++++++++++++ User space and kernel space entries are handled via a kernel_offset value when wrting. For PAI crypto counters this offset is 2048 or half of a page size. For PAI NNPA counter design this distinction was not needed. There is no user and kernel space part for the page pointed to by lowcore. The set up is: lowcore ptr ---> ++++++++++++++++++++++++ |user + kernel space | |area | | | ++++++++++++++++++++++++ There is always only one counter value recorded and saved. Depending on number of CPUs and machine load, the number of PAI NNPA counter increment differs between counting (perf stat) and recording (perf record). The number reported by sampling was double the number shown by counting. This was caused by a double read of the PAI NNPA values in function pai_copy(). The first part of that function reads the kernel space part. The offset into the kernel page part must be larger than zero. The second part of that function reads the user space part, which begins of offset zero. This works fine for PAI crypto counters. It fails for PAI NNPA counters because the PMU device driver does not support that feature and has a kernel_offset value of 0x0. Executing both user and kernel space read out might end up reading user space value twice. For the PAI NNPA PMU prohibit the kernel space part read out. Cc: stable@vger.kernel.org Fixes: f12473541356 ("s390/pai_crypto: Rename paicrypt_copy() to pai_copy()") Signed-off-by: Thomas Richter <tmricht@linux.ibm.com> Reviewed-by: Sumanth Korikkar <sumanthk@linux.ibm.com> Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit 3fe7ecab1a0856aafe1026a35af1621a5c18d53f upstream.

The PAI crypto counter design allows for user space and kernel space
PAI counter increment recording. This is achieved by splitting the
recording page in half. The upper part of the 4KB page records user
space increments of PAI crypto counter and the lower half records
kernel space increments. The page itself looks like:

 lowcore ptr ---> ++++++++++++++++++++++++
                  |user space area       |
                  +----------------------+
                  |kernel space area     |
                  ++++++++++++++++++++++++

User space and kernel space entries are handled via a kernel_offset
value when wrting. For PAI crypto counters this offset is 2048 or
half of a page size.

For PAI NNPA counter design this distinction was not needed. There is
no user and kernel space part for the page pointed to by lowcore.
The set up is:

 lowcore ptr ---> ++++++++++++++++++++++++
                  |user + kernel space   |
                  |area                  |
                  |                      |
                  ++++++++++++++++++++++++

There is always only one counter value recorded and saved.

Depending on number of CPUs and machine load, the number of PAI NNPA
counter increment differs between counting (perf stat) and recording
(perf record). The number reported by sampling was double the number
shown by counting.

This was caused by a double read of the PAI NNPA values in function
pai_copy(). The first part of that function reads the kernel space part.
The offset into the kernel page part must be larger than zero.
The second part of that function reads the user space part, which
begins of offset zero. This works fine for PAI crypto counters.

It fails for PAI NNPA counters because the PMU device driver does
not support that feature and has a kernel_offset value of 0x0.
Executing both user and kernel space read out might end up reading
user space value twice.
For the PAI NNPA PMU prohibit the kernel space part read out.

Cc: stable@vger.kernel.org
Fixes: f12473541356 ("s390/pai_crypto: Rename paicrypt_copy() to pai_copy()")
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
s390/debug: Reject zero-length input before trimming a newline 2026-05-14T13:31:14+00:00 Pengpeng Hou pengpeng@iscas.ac.cn 2026-04-17T07:35:30+00:00 89126e69ee43c006f166ad3f982b2b27dff90719 commit c366a7b5ed7564e41345c380285bd3f6cb98971b upstream. debug_get_user_string() duplicates the userspace buffer with memdup_user_nul() and then unconditionally looks at buffer[user_len - 1] to strip a trailing newline. A zero-length write reaches this helper unchanged, so the newline trim reads before the start of the allocated buffer. Reject empty writes before accessing the last input byte. Fixes: 66a464dbc8e0 ("[PATCH] s390: debug feature changes") Cc: stable@vger.kernel.org Signed-off-by: Pengpeng Hou <pengpeng@iscas.ac.cn> Reviewed-by: Benjamin Block <bblock@linux.ibm.com> Reviewed-by: Vasily Gorbik <gor@linux.ibm.com> Tested-by: Vasily Gorbik <gor@linux.ibm.com> Link: https://lore.kernel.org/r/20260417073530.96002-1-pengpeng@iscas.ac.cn Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit c366a7b5ed7564e41345c380285bd3f6cb98971b upstream.

debug_get_user_string() duplicates the userspace buffer with
memdup_user_nul() and then unconditionally looks at buffer[user_len - 1]
to strip a trailing newline.

A zero-length write reaches this helper unchanged, so the newline trim
reads before the start of the allocated buffer.

Reject empty writes before accessing the last input byte.

Fixes: 66a464dbc8e0 ("[PATCH] s390: debug feature changes")
Cc: stable@vger.kernel.org
Signed-off-by: Pengpeng Hou <pengpeng@iscas.ac.cn>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Tested-by: Vasily Gorbik <gor@linux.ibm.com>
Link: https://lore.kernel.org/r/20260417073530.96002-1-pengpeng@iscas.ac.cn
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
s390/debug: Reject zero-length input in debug_input_flush_fn() 2026-05-14T13:31:14+00:00 Vasily Gorbik gor@linux.ibm.com 2026-04-17T12:33:43+00:00 73461ff7441a52f1ca9b9b41ebcedfe072e2cdd7 commit e14622a7584f9608927c59a7d6ae4a0999dc545e upstream. debug_input_flush_fn() always copies one byte from the userspace buffer with copy_from_user() regardless of the supplied write length. A zero-length write therefore reads one byte beyond the caller's buffer. If the stale byte happens to be '-' or a digit the debug log is silently flushed. With an unmapped buffer the call returns -EFAULT. Reject zero-length writes before copying from userspace. Cc: stable@vger.kernel.org # v5.10+ Acked-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit e14622a7584f9608927c59a7d6ae4a0999dc545e upstream.

debug_input_flush_fn() always copies one byte from the userspace buffer
with copy_from_user() regardless of the supplied write length. A
zero-length write therefore reads one byte beyond the caller's buffer.
If the stale byte happens to be '-' or a digit the debug log is
silently flushed. With an unmapped buffer the call returns -EFAULT.

Reject zero-length writes before copying from userspace.

Cc: stable@vger.kernel.org # v5.10+
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Merge tag 's390-7.0-7' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux 2026-04-04T00:50:24+00:00 Linus Torvalds torvalds@linux-foundation.org 2026-04-04T00:50:24+00:00 3719114091cea0d3a896581e4fe5bed4eba4604b Pull s390 fixes from Vasily Gorbik: - Fix a memory leak in the zcrypt driver where the AP message buffer for clear key RSA requests was allocated twice, once by the caller and again locally, causing the first allocation to never be freed - Fix the cpum_sf perf sampling rate overflow adjustment to clamp the recalculated rate to the hardware maximum, preventing exceptions on heavily loaded systems running with HZ=1000 * tag 's390-7.0-7' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux: s390/zcrypt: Fix memory leak with CCA cards used as accelerator s390/cpum_sf: Cap sampling rate to prevent lsctl exception 
Pull s390 fixes from Vasily Gorbik:

 - Fix a memory leak in the zcrypt driver where the AP message buffer
   for clear key RSA requests was allocated twice, once by the caller
   and again locally, causing the first allocation to never be freed

 - Fix the cpum_sf perf sampling rate overflow adjustment to clamp the
   recalculated rate to the hardware maximum, preventing exceptions on
   heavily loaded systems running with HZ=1000

* tag 's390-7.0-7' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux:
  s390/zcrypt: Fix memory leak with CCA cards used as accelerator
  s390/cpum_sf: Cap sampling rate to prevent lsctl exception
Merge tag 's390-7.0-6' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux 2026-03-28T16:50:11+00:00 Linus Torvalds torvalds@linux-foundation.org 2026-03-28T16:50:11+00:00 e522b75c44f580ba8a58e8d9f263643c7936ddeb Pull s390 fixes from Vasily Gorbik: - Add array_index_nospec() to syscall dispatch table lookup to prevent limited speculative out-of-bounds access with user-controlled syscall number - Mark array_index_mask_nospec() __always_inline since GCC may emit an out-of-line call instead of the inline data dependency sequence the mitigation relies on - Clear r12 on kernel entry to prevent potential speculative use of user value in system_call, ext/io/mcck interrupt handlers * tag 's390-7.0-6' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux: s390/entry: Scrub r12 register on kernel entry s390/syscalls: Add spectre boundary for syscall dispatch table s390/barrier: Make array_index_mask_nospec() __always_inline 
Pull s390 fixes from Vasily Gorbik:

 - Add array_index_nospec() to syscall dispatch table lookup to prevent
   limited speculative out-of-bounds access with user-controlled syscall
   number

 - Mark array_index_mask_nospec() __always_inline since GCC may emit an
   out-of-line call instead of the inline data dependency sequence the
   mitigation relies on

 - Clear r12 on kernel entry to prevent potential speculative use of
   user value in system_call, ext/io/mcck interrupt handlers

* tag 's390-7.0-6' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux:
  s390/entry: Scrub r12 register on kernel entry
  s390/syscalls: Add spectre boundary for syscall dispatch table
  s390/barrier: Make array_index_mask_nospec() __always_inline
s390/entry: Scrub r12 register on kernel entry 2026-03-27T23:43:39+00:00 Vasily Gorbik gor@linux.ibm.com 2026-03-26T18:50:14+00:00 0738d395aab8fae3b5a3ad3fc640630c91693c27 Before commit f33f2d4c7c80 ("s390/bp: remove TIF_ISOLATE_BP"), all entry handlers loaded r12 with the current task pointer (lg %r12,__LC_CURRENT) for use by the BPENTER/BPEXIT macros. That commit removed TIF_ISOLATE_BP, dropping both the branch prediction macros and the r12 load, but did not add r12 to the register clearing sequence. Add the missing xgr %r12,%r12 to make the register scrub consistent across all entry points. Fixes: f33f2d4c7c80 ("s390/bp: remove TIF_ISOLATE_BP") Cc: stable@kernel.org Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> 
Before commit f33f2d4c7c80 ("s390/bp: remove TIF_ISOLATE_BP"),
all entry handlers loaded r12 with the current task pointer
(lg %r12,__LC_CURRENT) for use by the BPENTER/BPEXIT macros. That
commit removed TIF_ISOLATE_BP, dropping both the branch prediction
macros and the r12 load, but did not add r12 to the register clearing
sequence.

Add the missing xgr %r12,%r12 to make the register scrub consistent
across all entry points.

Fixes: f33f2d4c7c80 ("s390/bp: remove TIF_ISOLATE_BP")
Cc: stable@kernel.org
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
s390/syscalls: Add spectre boundary for syscall dispatch table 2026-03-27T23:43:39+00:00 Greg Kroah-Hartman gregkh@linuxfoundation.org 2026-03-24T16:34:05+00:00 48b8814e25d073dd84daf990a879a820bad2bcbd The s390 syscall number is directly controlled by userspace, but does not have an array_index_nospec() boundary to prevent access past the syscall function pointer tables. Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Christian Borntraeger <borntraeger@linux.ibm.com> Cc: Sven Schnelle <svens@linux.ibm.com> Cc: Arnd Bergmann <arnd@arndb.de> Fixes: 56e62a737028 ("s390: convert to generic entry") Cc: stable@kernel.org Assisted-by: gkh_clanker_2000 Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Vasily Gorbik <gor@linux.ibm.com> Link: https://lore.kernel.org/r/2026032404-sterling-swoosh-43e6@gregkh Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> 
The s390 syscall number is directly controlled by userspace, but does
not have an array_index_nospec() boundary to prevent access past the
syscall function pointer tables.

Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Fixes: 56e62a737028 ("s390: convert to generic entry")
Cc: stable@kernel.org
Assisted-by: gkh_clanker_2000
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Link: https://lore.kernel.org/r/2026032404-sterling-swoosh-43e6@gregkh
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
s390/cpum_sf: Cap sampling rate to prevent lsctl exception 2026-03-24T19:57:31+00:00 Thomas Richter tmricht@linux.ibm.com 2026-03-06T12:50:31+00:00 57ad0d4a00f5d3e80f33ba2da8d560c73d83dc22 commit fcc43a7e294f ("s390/configs: Set HZ=1000") changed the interrupt frequency of the system. On machines with heavy load and many perf event overflows, this might lead to an exception. Dmesg displays these entries: [112.242542] cpum_sf: Loading sampling controls failed: op 1 err -22 One line per CPU online. The root cause is the CPU Measurement sampling facility overflow adjustment. Whenever an overflow (too much samples per tick) occurs, the sampling rate is adjusted and increased. This was done without observing the maximum sampling rate limit. When the current sampling interval is higher than the maximum sampling rate limit, the lsctl instruction raises an exception. The error messages is the result of such an exception. Observe the upper limit when the new sampling rate is recalculated. Cc: stable@vger.kernel.org Fixes: 39d4a501a9ef ("s390/cpum_sf: Adjust sampling interval to avoid hitting sample limits") Signed-off-by: Thomas Richter <tmricht@linux.ibm.com> Reviewed-by: Sumanth Korikkar <sumanthk@linux.ibm.com> Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> 
commit fcc43a7e294f ("s390/configs: Set HZ=1000") changed the interrupt
frequency of the system. On machines with heavy load and many perf event
overflows, this might lead to an exception. Dmesg displays these entries:
  [112.242542] cpum_sf: Loading sampling controls failed: op 1 err -22
One line per CPU online.

The root cause is the CPU Measurement sampling facility overflow
adjustment. Whenever an overflow (too much samples per tick) occurs, the
sampling rate is adjusted and increased. This was done without observing
the maximum sampling rate limit. When the current sampling interval is
higher than the maximum sampling rate limit, the lsctl instruction raises
an exception. The error messages is the result of such an exception.
Observe the upper limit when the new sampling rate is recalculated.

Cc: stable@vger.kernel.org
Fixes: 39d4a501a9ef ("s390/cpum_sf: Adjust sampling interval to avoid hitting sample limits")
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Merge tag 'kvm-s390-master-7.0-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux into HEAD 2026-03-24T16:32:13+00:00 Paolo Bonzini pbonzini@redhat.com 2026-03-24T16:32:13+00:00 12fd96587160fe6714dd19dfdd7b6b6184eacea4 KVM: s390: Fixes for 7.0 - fix deadlock in new memory management - handle kernel faults on donated memory properly - fix bounds checking for irq routing + selftest - fix invalid machine checks + logging 
KVM: s390: Fixes for 7.0

- fix deadlock in new memory management
- handle kernel faults on donated memory properly
- fix bounds checking for irq routing + selftest
- fix invalid machine checks + logging