linux.git/include/linux/kernel_stat.h, branch v7.2-rc1

s390/idle: Provide arch specific kcpustat_field_idle()/kcpustat_field_iowait()

2026-06-15T14:33:40+00:00

The former s390 specific arch_cpu_idle_time() implementation was
removed, since its implementation was racy and reported idle time
could go backwards [1].

However this removal was not necessary, since independently of the s390
architecture specific races there exists the iowait counter update race,
which can also lead to reported idle time going backwards [2].

With Frederic Weisbecker's recent cpu idle time accounting refactoring
kernel_cpustat got a sequence counter. Use this to implement s390 specific
variants of kcpustat_field_idle() and kcpustat_field_iowait(). This is
logically a revert of [1] and moves cpu idle time accounting back into s390
architecture code, which is also more precise than the dyntick idle time
accounting by nohz/scheduler.

For comparing cross cpu time stamps it is necessary to use the stcke
instead of the stckf instruction in irq entry path. Furthermore this
open-codes a sequence lock in assembler and C code, which is required to
update the irq entry time stamp to the per cpu idle_data structure in a
race free manner.

[1] commit be76ea614460 ("s390/idle: remove arch_cpu_idle_time() and corresponding code")
[2] commit ead70b752373 ("timers/nohz: Add a comment about broken iowait counter update race")

Signed-off-by: Heiko Carstens 
Acked-by: Frederic Weisbecker 
Signed-off-by: Alexander Gordeev

sched/cputime: Handle dyntick-idle steal time correctly

2026-06-02T19:27:26+00:00

The dyntick-idle steal time is currently accounted when the tick restarts
but the stolen idle time is not subtracted from the idle time that was
already accounted. This is to avoid observing the idle time going backward
as the dyntick-idle cputime accessors can't reliably know in advance the
stolen idle time.

In order to maintain a forward progressing idle cputime while subtracting
idle steal time from it, keep track of the previously accounted idle stolen
time and substract it from _later_ idle cputime accounting.

Signed-off-by: Frederic Weisbecker 
Signed-off-by: Thomas Gleixner 
Tested-by: Shrikanth Hegde 
Link: https://patch.msgid.link/20260508131647.43868-16-frederic@kernel.org

sched/cputime: Provide get_cpu_[idle|iowait]_time_us() off-case

2026-06-02T19:27:26+00:00

The last reason why get_cpu_idle/iowait_time_us() may return -1 now is if
the config doesn't support nohz.

The ad-hoc replacement solution by cpufreq is to compute jiffies minus the
whole busy cputime. Although the intention should provide a coherent low
resolution estimation of the idle and iowait time, the implementation is
buggy because jiffies don't start at 0.

Just provide instead a real get_cpu_[idle|iowait]_time_us() offcase.

Signed-off-by: Frederic Weisbecker 
Signed-off-by: Thomas Gleixner 
Tested-by: Shrikanth Hegde 
Link: https://patch.msgid.link/20260508131647.43868-14-frederic@kernel.org

tick/sched: Consolidate idle time fetching APIs

2026-06-02T19:27:26+00:00

Fetching the idle cputime is available through a variety of accessors all
over the place depending on the different accounting flavours and needs:

  - idle vtime generic accounting can be accessed by kcpustat_field(),
    kcpustat_cpu_fetch(), get_idle/iowait_time() and
    get_cpu_idle/iowait_time_us()

  - dynticks-idle accounting can only be accessed by get_idle/iowait_time()
    or get_cpu_idle/iowait_time_us()

  - CONFIG_NO_HZ_COMMON=n idle accounting can be accessed by kcpustat_field()
    kcpustat_cpu_fetch(), or get_idle/iowait_time() but not by
    get_cpu_idle/iowait_time_us()

Moreover get_idle/iowait_time() relies on get_cpu_idle/iowait_time_us()
with a non-sensical conversion to microseconds and back to nanoseconds on
the way.

Start consolidating the APIs with removing get_idle/iowait_time() and make
kcpustat_field() and kcpustat_cpu_fetch() work for all cases.

Signed-off-by: Frederic Weisbecker 
Signed-off-by: Thomas Gleixner 
Tested-by: Shrikanth Hegde 
Link: https://patch.msgid.link/20260508131647.43868-13-frederic@kernel.org

tick/sched: Move dyntick-idle cputime accounting to cputime code

2026-06-02T19:27:25+00:00

Although the dynticks-idle cputime accounting is necessarily tied to the
tick subsystem, the actual related accounting code has no business residing
there and should be part of the scheduler cputime code.

Move away the relevant pieces and state machine to where they belong.

Signed-off-by: Frederic Weisbecker 
Signed-off-by: Thomas Gleixner 
Tested-by: Shrikanth Hegde 
Link: https://patch.msgid.link/20260508131647.43868-10-frederic@kernel.org

tick/sched: Unify idle cputime accounting

2026-06-02T19:27:25+00:00

The non-vtime dynticks-idle cputime accounting is a big mess that
accumulates within two concurrent statistics, each having their own
shortcomings:

 * The accounting for online CPUs which is based on the delta between
   tick_nohz_start_idle() and tick_nohz_stop_idle().

   Pros:
       - Works when the tick is off

       - Has nsecs granularity

   Cons:
       - Account idle steal time but doesn't substract it from idle
         cputime.

       - Assumes CONFIG_IRQ_TIME_ACCOUNTING by not accounting IRQs but
         the IRQ time is simply ignored when
         CONFIG_IRQ_TIME_ACCOUNTING=n

       - The windows between 1) idle task scheduling and the first call
         to tick_nohz_start_idle() and 2) idle task between the last
         tick_nohz_stop_idle() and the rest of the idle time are
         blindspots wrt. cputime accounting (though mostly insignificant
         amount)

       - Relies on private fields outside of kernel stats, with specific
         accessors.

 * The accounting for offline CPUs which is based on ticks and the
   jiffies delta during which the tick was stopped.

   Pros:
       - Handles steal time correctly

       - Handle CONFIG_IRQ_TIME_ACCOUNTING=y and
         CONFIG_IRQ_TIME_ACCOUNTING=n correctly.

       - Handles the whole idle task

       - Accounts directly to kernel stats, without midlayer accumulator.

    Cons:
       - Doesn't elapse when the tick is off, which doesn't make it
         suitable for online CPUs.

       - Has TICK_NSEC granularity (jiffies)

       - Needs to track the dyntick-idle ticks that were accounted and
         substract them from the total jiffies time spent while the tick
         was stopped. This is an ugly workaround.

Having two different accounting for a single context is not the only
problem: since those accountings are of different natures, it is
possible to observe the global idle time going backward after a CPU goes
offline.

Clean up the situation with introducing a hybrid approach that stays
coherent and works for both online and offline CPUs:

  * Tick based or native vtime accounting operate before the idle loop
    is entered and resume once the idle loop prepares to exit.

  * When the idle loop starts, switch to dynticks-idle accounting as is
    done currently, except that the statistics accumulate directly to the
    relevant kernel stat fields.

  * Private dyntick cputime accounting fields are removed.

  * Works on both online and offline case.

Further improvement will include:

  * Only switch to dynticks-idle cputime accounting when the tick actually
    goes in dynticks mode.

  * Handle CONFIG_IRQ_TIME_ACCOUNTING=n correctly such that the
    dynticks-idle accounting still elapses while on IRQs.

  * Correctly substract idle steal cputime from idle time

Reported-by: Xin Zhao 
Signed-off-by: Frederic Weisbecker 
Signed-off-by: Thomas Gleixner 
Tested-by: Shrikanth Hegde 
Link: https://patch.msgid.link/20260508131647.43868-8-frederic@kernel.org

sched/cputime: Remove superfluous and error prone kcpustat_field() parameter

2026-06-02T19:27:25+00:00

The first parameter to kcpustat_field() is a pointer to the cpu kcpustat to
be fetched from. This parameter is error prone because a copy to a kcpustat
could be passed by accident instead of the original one. Also the kcpustat
structure can already be retrieved with the help of the mandatory CPU
argument.

Remove the needless parameter.

Signed-off-by: Frederic Weisbecker 
Signed-off-by: Thomas Gleixner 
Tested-by: Shrikanth Hegde 
Reviewed-by: Shrikanth Hegde 
Link: https://patch.msgid.link/20260508131647.43868-4-frederic@kernel.org

cpumask: cleanup core headers inclusion

2024-06-25T05:25:02+00:00

Many core headers include cpumask.h for nothing. Drop it.

Link: https://lkml.kernel.org/r/20240528005648.182376-6-yury.norov@gmail.com
Signed-off-by: Yury Norov 
Cc: Amit Daniel Kachhap 
Cc: Anna-Maria Behnsen 
Cc: Christoph Lameter 
Cc: Daniel Lezcano 
Cc: Dennis Zhou 
Cc: Frederic Weisbecker 
Cc: Johannes Weiner 
Cc: Juri Lelli 
Cc: Kees Cook 
Cc: Mathieu Desnoyers 
Cc: Paul E. McKenney 
Cc: Peter Zijlstra 
Cc: Rafael J. Wysocki 
Cc: Rasmus Villemoes 
Cc: Tejun Heo 
Cc: Thomas Gleixner 
Cc: Ulf Hansson 
Cc: Vincent Guittot 
Cc: Viresh Kumar 
Cc: Yury Norov 
Signed-off-by: Andrew Morton

genirq: Provide a snapshot mechanism for interrupt statistics

2024-04-12T15:08:05+00:00

The soft lockup detector lacks a mechanism to identify interrupt storms as
root cause of a lockup. To enable this the detector needs a mechanism to
snapshot the interrupt count statistics on a CPU when the detector observes
a potential lockup scenario and compare that against the interrupt count
when it warns about the lockup later on. The number of interrupts in that
period give a hint whether the lockup might have been caused by an interrupt
storm.

Instead of having extra storage in the lockup detector and accessing the
internals of the interrupt descriptor directly, add a snapshot member to
the per CPU irq_desc::kstat_irq structure and provide interfaces to take a
snapshot of all interrupts on the current CPU and to retrieve the delta of
a specific interrupt later on.

Originally-by: Thomas Gleixner 
Signed-off-by: Bitao Hu 
Signed-off-by: Thomas Gleixner 
Link: https://lore.kernel.org/r/20240411074134.30922-3-yaoma@linux.alibaba.com

sched: Add helper nr_context_switches_cpu()

2023-01-05T20:21:11+00:00

Add a function nr_context_switches_cpu() that returns  number of context
switches since boot on the specified CPU.  This information will be used
to diagnose RCU CPU stalls.

Signed-off-by: Zhen Lei 
Cc: Ingo Molnar 
Cc: Peter Zijlstra 
Cc: Juri Lelli 
Cc: Vincent Guittot 
Cc: Dietmar Eggemann 
Cc: Steven Rostedt 
Cc: Ben Segall 
Cc: Mel Gorman 
Cc: Daniel Bristot de Oliveira 
Cc: Valentin Schneider 
Reviewed-by: Frederic Weisbecker 
Signed-off-by: Paul E. McKenney