linux.git/kernel/sched/cputime.c, branch v5.10

sched/cputime: Improve cputime_adjust()

2020-06-15T12:10:00+00:00

People report that utime and stime from /proc//stat become very
wrong when the numbers are big enough, especially if you watch these
counters incrementally.

Specifically, the current implementation of: stime*rtime/total,
results in a saw-tooth function on top of the desired line, where the
teeth grow in size the larger the values become. IOW, it has a
relative error.

The result is that, when watching incrementally as time progresses
(for large values), we'll see periods of pure stime or utime increase,
irrespective of the actual ratio we're striving for.

Replace scale_stime() with a math64.h helper: mul_u64_u64_div_u64()
that is far more accurate. This also allows architectures to override
the implementation -- for instance they can opt for the old algorithm
if this new one turns out to be too expensive for them.

Signed-off-by: Oleg Nesterov 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://lkml.kernel.org/r/20200519172506.GA317395@hirez.programming.kicks-ass.net

sched/vtime: Work around an unitialized variable warning

2020-04-15T09:06:50+00:00

Work around this warning:

  kernel/sched/cputime.c: In function ‘kcpustat_field’:
  kernel/sched/cputime.c:1007:6: warning: ‘val’ may be used uninitialized in this function [-Wmaybe-uninitialized]

because GCC can't see that val is used only when err is 0.

Acked-by: Peter Zijlstra 
Signed-off-by: Borislav Petkov 
Signed-off-by: Ingo Molnar 
Link: https://lore.kernel.org/r/20200327214334.GF8015@zn.tnic

sched/vtime: Prevent unstable evaluation of WARN(vtime->state)

2020-03-06T11:57:16+00:00

As the vtime is sampled under loose seqcount protection by kcpustat, the
vtime fields may change as the code flows. Where logic dictates a field
has a static value, use a READ_ONCE.

Signed-off-by: Chris Wilson 
Signed-off-by: Frederic Weisbecker 
Signed-off-by: Peter Zijlstra (Intel) 
Signed-off-by: Ingo Molnar 
Fixes: 74722bb223d0 ("sched/vtime: Bring up complete kcpustat accessor")
Link: https://lkml.kernel.org/r/20200123180849.28486-1-frederic@kernel.org

sched/cputime: move rq parameter in irqtime_account_process_tick

2020-01-17T09:19:21+00:00

Every time we call irqtime_account_process_tick() is in a interrupt,
Every caller will get and assign a parameter rq = this_rq(), This is
unnecessary and increase the code size a little bit. Move the rq getting
action to irqtime_account_process_tick internally is better.

             base               with this patch
cputime.o    578792 bytes        577888 bytes

Signed-off-by: Alex Shi 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://lkml.kernel.org/r/1577959674-255537-1-git-send-email-alex.shi@linux.alibaba.com

sched/vtime: Bring up complete kcpustat accessor

2019-11-21T06:33:24+00:00

Many callsites want to fetch the values of system, user, user_nice, guest
or guest_nice kcpustat fields altogether or at least a pair of these.

In that case calling kcpustat_field() for each requested field brings
unecessary overhead when we could fetch all of them in a row.

So provide kcpustat_cpu_fetch() that fetches the whole kcpustat array
in a vtime safe way under the same RCU and seqcount block.

Signed-off-by: Frederic Weisbecker 
Cc: Peter Zijlstra 
Cc: Wanpeng Li 
Cc: Yauheni Kaliuta 
Link: https://lkml.kernel.org/r/20191121024430.19938-3-frederic@kernel.org
Signed-off-by: Ingo Molnar

sched/cputime: Support other fields on kcpustat_field()

2019-11-21T06:33:23+00:00

Provide support for user, nice, guest and guest_nice fields through
kcpustat_field().

Whether we account the delta to a nice or not nice field is decided on
top of the nice value snapshot taken at the time we call kcpustat_field().
If the nice value of the task has been changed since the last vtime
update, we may have inacurrate distribution of the nice VS unnice
cputime.

However this is considered as a minor issue compared to the proper fix
that would involve interrupting the target on nice updates, which is
undesired on nohz_full CPUs.

Signed-off-by: Frederic Weisbecker 
Cc: Peter Zijlstra 
Cc: Wanpeng Li 
Cc: Yauheni Kaliuta 
Link: https://lkml.kernel.org/r/20191121024430.19938-2-frederic@kernel.org
Signed-off-by: Ingo Molnar

sched/kcpustat: Introduce vtime-aware kcpustat accessor for CPUTIME_SYSTEM

2019-10-29T09:01:17+00:00

Kcpustat is not correctly supported on nohz_full CPUs. The tick doesn't
fire and the cputime therefore doesn't move forward. The issue has shown
up after the vanishing of the remaining 1Hz which has made the stall
visible.

We are solving that with checking the task running on a CPU through RCU
and reading its vtime delta that we add to the raw kcpustat values.

We make sure that we fetch a coherent raw-kcpustat/vtime-delta couple
sequence while checking that the CPU referred by the target vtime is the
correct one, under the locked vtime seqcount.

Only CPUTIME_SYSTEM is handled here as a start because it's the trivial
case. User and guest time will require more preparation work to
correctly handle niceness.

Reported-by: Yauheni Kaliuta 
Signed-off-by: Frederic Weisbecker 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Rik van Riel 
Cc: Thomas Gleixner 
Cc: Wanpeng Li 
Link: https://lkml.kernel.org/r/20191025020303.19342-1-frederic@kernel.org
Signed-off-by: Ingo Molnar

sched/vtime: Rename vtime_accounting_cpu_enabled() to vtime_accounting_enabled_this_cpu()

2019-10-29T09:01:14+00:00

Standardize the naming on top of the vtime_accounting_enabled_*() base.
Also make it clear we are checking the vtime state of the
*current* CPU with this function. We'll need to add an API to check that
state on remote CPUs as well, so we must disambiguate the naming.

Signed-off-by: Frederic Weisbecker 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Jacek Anaszewski 
Cc: Linus Torvalds 
Cc: Pavel Machek 
Cc: Peter Zijlstra 
Cc: Rafael J . Wysocki 
Cc: Rik van Riel 
Cc: Thomas Gleixner 
Cc: Viresh Kumar 
Cc: Wanpeng Li 
Cc: Yauheni Kaliuta 
Link: https://lkml.kernel.org/r/20191016025700.31277-9-frederic@kernel.org
Signed-off-by: Ingo Molnar

sched/cputime: Add vtime guest task state

2019-10-29T09:01:11+00:00

Record guest as a VTIME state instead of guessing it from VTIME_SYS and
PF_VCPU. This is going to simplify the cputime read side especially as
its state machine is going to further expand in order to fully support
kcpustat on nohz_full.

Signed-off-by: Frederic Weisbecker 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Jacek Anaszewski 
Cc: Linus Torvalds 
Cc: Pavel Machek 
Cc: Peter Zijlstra 
Cc: Rafael J . Wysocki 
Cc: Rik van Riel 
Cc: Thomas Gleixner 
Cc: Viresh Kumar 
Cc: Wanpeng Li 
Cc: Yauheni Kaliuta 
Link: https://lkml.kernel.org/r/20191016025700.31277-4-frederic@kernel.org
Signed-off-by: Ingo Molnar

sched/cputime: Add vtime idle task state

2019-10-29T09:01:10+00:00

Record idle as a VTIME state instead of guessing it from VTIME_SYS and
is_idle_task(). This is going to simplify the cputime read side
especially as its state machine is going to further expand in order to
fully support kcpustat on nohz_full.

Signed-off-by: Frederic Weisbecker 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Jacek Anaszewski 
Cc: Linus Torvalds 
Cc: Pavel Machek 
Cc: Peter Zijlstra 
Cc: Rafael J . Wysocki 
Cc: Rik van Riel 
Cc: Thomas Gleixner 
Cc: Viresh Kumar 
Cc: Wanpeng Li 
Cc: Yauheni Kaliuta 
Link: https://lkml.kernel.org/r/20191016025700.31277-3-frederic@kernel.org
Signed-off-by: Ingo Molnar