linux-stable.git/arch/s390/kernel/idle.c, branch linux-rolling-stable

s390/idle: Inline update_timer_idle()

2026-02-25T15:46:07+00:00

Inline update_timer_idle() again to avoid an extra function call. This
way the generated code is close to old assembler version again.

Reviewed-by: Sven Schnelle 
Signed-off-by: Heiko Carstens 
Signed-off-by: Vasily Gorbik

s390/idle: Slightly optimize idle time accounting

2026-02-25T15:46:07+00:00

Slightly optimize account_idle_time_irq() and update_timer_idle():

- Use fast single instruction __atomic64() primitives to update per
  cpu idle_time and idle_count, instead of READ_ONCE() / WRITE_ONCE()
  pairs

- stcctm() is an inline assembly with a full memory barrier. This
  leads to a not necessary extra dereference of smp_cpu_mtid in
  update_timer_idle(). Avoid this and read smp_cpu_mtid into a
  variable

- Use __this_cpu_add() instead of this_cpu_add() to avoid disabling /
  enabling of preemption several times in a loop in update_timer_idle().

Reviewed-by: Sven Schnelle 
Signed-off-by: Heiko Carstens 
Signed-off-by: Vasily Gorbik

s390/idle: Add comment for non obvious code

2026-02-25T15:46:07+00:00

Add a comment to update_timer_idle() which describes why wall time (not
steal time) is added to steal_timer. This is not obvious and was reported
by Frederic Weisbecker.

Reported-by: Frederic Weisbecker 
Closes: https://lore.kernel.org/all/aXEVM-04lj0lntMr@localhost.localdomain/
Reviewed-by: Sven Schnelle 
Signed-off-by: Heiko Carstens 
Signed-off-by: Vasily Gorbik

s390/idle: Fix cpu idle exit cpu time accounting

2026-02-25T15:46:07+00:00

With the conversion to generic entry [1] cpu idle exit cpu time accounting
was converted from assembly to C. This introduced an reversed order of cpu
time accounting.

On cpu idle exit the current accounting happens with the following call
chain:

-> do_io_irq()/do_ext_irq()
 -> irq_enter_rcu()
  -> account_hardirq_enter()
   -> vtime_account_irq()
    -> vtime_account_kernel()

vtime_account_kernel() accounts the passed cpu time since last_update_timer
as system time, and updates last_update_timer to the current cpu timer
value.

However the subsequent call of

 -> account_idle_time_irq()

will incorrectly subtract passed cpu time from timer_idle_enter to the
updated last_update_timer value from system_timer. Then last_update_timer
is updated to a sys_enter_timer, which means that last_update_timer goes
back in time.

Subsequently account_hardirq_exit() will account too much cpu time as
hardirq time. The sum of all accounted cpu times is still correct, however
some cpu time which was previously accounted as system time is now
accounted as hardirq time, plus there is the oddity that last_update_timer
goes back in time.

Restore previous behavior by extracting cpu time accounting code from
account_idle_time_irq() into a new update_timer_idle() function and call it
before irq_enter_rcu().

Fixes: 56e62a737028 ("s390: convert to generic entry") [1]
Reviewed-by: Sven Schnelle 
Signed-off-by: Heiko Carstens 
Signed-off-by: Vasily Gorbik

s390/idle: Remove duplicate get_lowcore() calls

2024-06-18T15:01:33+00:00

Assign the output from get_lowcore() to a local variable,
so the code is easier to read.

Acked-by: Heiko Carstens 
Signed-off-by: Sven Schnelle 
Signed-off-by: Vasily Gorbik

s390: Replace S390_lowcore by get_lowcore()

2024-06-18T15:01:33+00:00

Replace all S390_lowcore usages in arch/s390/ by get_lowcore().

Acked-by: Heiko Carstens 
Signed-off-by: Sven Schnelle 
Signed-off-by: Vasily Gorbik

s390/idle: Rewrite psw_idle() in C

2024-05-14T11:37:07+00:00

To ease maintenance and further enhancements, convert
the psw_idle() function to C.

Reviewed-by: Heiko Carstens 
Signed-off-by: Sven Schnelle 
Signed-off-by: Alexander Gordeev

sched/idle: Mark arch_cpu_idle_dead() __noreturn

2023-03-08T16:44:28+00:00

Before commit 076cbf5d2163 ("x86/xen: don't let xen_pv_play_dead()
return"), in Xen, when a previously offlined CPU was brought back
online, it unexpectedly resumed execution where it left off in the
middle of the idle loop.

There were some hacks to make that work, but the behavior was surprising
as do_idle() doesn't expect an offlined CPU to return from the dead (in
arch_cpu_idle_dead()).

Now that Xen has been fixed, and the arch-specific implementations of
arch_cpu_idle_dead() also don't return, give it a __noreturn attribute.

This will cause the compiler to complain if an arch-specific
implementation might return.  It also improves code generation for both
caller and callee.

Also fixes the following warning:

  vmlinux.o: warning: objtool: do_idle+0x25f: unreachable instruction

Reported-by: Paul E. McKenney 
Tested-by: Paul E. McKenney 
Link: https://lore.kernel.org/r/60d527353da8c99d4cf13b6473131d46719ed16d.1676358308.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf

Merge tag 's390-6.3-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux

2023-02-21T23:09:17+00:00

Pull s390 updates from Heiko Carstens:

 - Large cleanup of the con3270/tty3270 driver. Among others this fixes:
     - Background Color Support
     - ASCII Line Character Support
     - VT100 Support
     - Geometries other than 80x24

 - Cleanup and improve cmpxchg() code. Also add cmpxchg_user_key() to
   uaccess functions, which will be used by KVM to access KVM guest
   memory with a specific storage key

 - Add support for user space events counting to CPUMF

 - Cleanup the vfio/ccw code, which also allows now to properly support
   2K Format-2 IDALs

 - Move kernel page table allocation and initialization to decompressor,
   which finally allows to enter the kernel with dynamic address
   translation enabled. This in turn allows to get rid of code with
   special handling in the kernel, which has to distinguish if DAT is on
   or off

 - Replace kretprobe with rethook

 - Various improvements to vfio/ap queue resets:
     - Use TAPQ to verify completion of a reset in progress rather than
       multiple invocations of ZAPQ.
     - Check TAPQ response codes when verifying successful completion of
       ZAPQ.
     - Fix erroneous handling of some error response codes.
     - Increase the maximum amount of time to wait for successful
       completion of ZAPQ

 - Rework system call wrappers to get rid of alias functions, which were
   only left on s390

 - Cleanup diag288_wdt watchdog driver. It has been agreed on with
   Guenter Roeck that this goes upstream via the s390 tree

 - Add missing loadparm parameter handling for list-directed ECKD
   ipl/reipl

 - Various improvements to memory detection code

 - Remove arch_cpu_idle_time() since the current implementation is
   broken, and allows user space observable accounted idle times which
   can temporarily decrease

 - Add Reset DAT-Protection support: (only) allow to change PTEs from RO
   to RW with a new RDP instruction. Unlike the currently used IPTE
   instruction, this does not necessarily guarantee that TLBs of all
   CPUs are synchronously flushed; and that remote CPUs can see spurious
   protection faults. The overall improvement for not requiring an all
   CPU synchronization, like it is required with IPTE, should be
   beneficial

 - Fix KFENCE page fault reporting

 - Smaller cleanups and improvement all over the place

* tag 's390-6.3-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux: (182 commits)
  s390/irq,idle: simplify idle check
  s390/processor: add test_and_set_cpu_flag() and test_and_clear_cpu_flag()
  s390/processor: let cpu helper functions return boolean values
  s390/kfence: fix page fault reporting
  s390/zcrypt: introduce ctfm field in struct CPRBX
  s390: remove confusing comment from uapi types header file
  vfio/ccw: remove WARN_ON during shutdown
  s390/entry: remove toolchain dependent micro-optimization
  s390/mem_detect: do not truncate online memory ranges info
  s390/vx: remove __uint128_t type from __vector128 struct again
  s390/mm: add support for RDP (Reset DAT-Protection)
  s390/mm: define private VM_FAULT_* reasons from top bits
  Documentation: s390: correct spelling
  s390/ap: fix status returned by ap_qact()
  s390/ap: fix status returned by ap_aqic()
  s390: vfio-ap: tighten the NIB validity check
  Revert "s390/mem_detect: do not update output parameters on failure"
  s390/idle: remove arch_cpu_idle_time() and corresponding code
  s390/vx: use simple assignments to access __vector128 members
  s390/vx: add 64 and 128 bit members to __vector128 struct
  ...

s390/irq,idle: simplify idle check

2023-02-15T10:07:01+00:00

Use the per-cpu CIF_ENABLED_WAIT flag to decide if an interrupt
occurred while a cpu was idle, instead of checking two conditions
within the old psw.

Also move clearing of the CIF_ENABLED_WAIT bit to the early interrupt
handler, which in turn makes arch_vcpu_is_preempted() also a bit more
precise, since the flag is now cleared before interrupt handlers have
been called.

Reviewed-by: Sven Schnelle 
Signed-off-by: Heiko Carstens