linux.git/kernel/sched/cputime.c, branch v4.13

sched/cputime: Don't use smp_processor_id() in preemptible context

2017-07-14T08:27:15+00:00

Recent kernels trigger this warning:

 BUG: using smp_processor_id() in preemptible [00000000] code: 99-trinity/181
 caller is debug_smp_processor_id+0x17/0x19
 CPU: 0 PID: 181 Comm: 99-trinity Not tainted 4.12.0-01059-g2a42eb9 #1
 Call Trace:
  dump_stack+0x82/0xb8
  check_preemption_disabled()
  debug_smp_processor_id()
  vtime_delta()
  task_cputime()
  thread_group_cputime()
  thread_group_cputime_adjusted()
  wait_consider_task()
  do_wait()
  SYSC_wait4()
  do_syscall_64()
  entry_SYSCALL64_slow_path()

As Frederic pointed out:

| Although those sched_clock_cpu() things seem to only matter when the
| sched_clock() is unstable. And that stability is a condition for nohz_full
| to work anyway. So probably sched_clock() alone would be enough.

This patch fixes it by replacing sched_clock_cpu() with sched_clock() to
avoid calling smp_processor_id() in a preemptible context.

Reported-by: Xiaolong Ye 
Signed-off-by: Wanpeng Li 
Cc: Frederic Weisbecker 
Cc: Linus Torvalds 
Cc: Luiz Capitulino 
Cc: Peter Zijlstra 
Cc: Rik van Riel 
Cc: Thomas Gleixner 
Link: http://lkml.kernel.org/r/1499586028-7402-1-git-send-email-wanpeng.li@hotmail.com
[ Prettified the changelog. ]
Signed-off-by: Ingo Molnar

sched/cputime: Accumulate vtime on top of nsec clocksource

2017-07-05T07:54:15+00:00

Currently the cputime source used by vtime is jiffies. When we cross
a context boundary and jiffies have changed since the last snapshot, the
pending cputime is accounted to the switching out context.

This system works ok if the ticks are not aligned across CPUs. If they
instead are aligned (ie: all fire at the same time) and the CPUs run in
userspace, the jiffies change is only observed on tick exit and therefore
the user cputime is accounted as system cputime. This is because the
CPU that maintains timekeeping fires its tick at the same time as the
others. It updates jiffies in the middle of the tick and the other CPUs
see that update on IRQ exit:

    CPU 0 (timekeeper)                  CPU 1
    -------------------              -------------
                      jiffies = N
    ...                              run in userspace for a jiffy
    tick entry                       tick entry (sees jiffies = N)
    set jiffies = N + 1
    tick exit                        tick exit (sees jiffies = N + 1)
                                                account 1 jiffy as stime

Fix this with using a nanosec clock source instead of jiffies. The
cputime is then accumulated and flushed everytime the pending delta
reaches a jiffy in order to mitigate the accounting overhead.

[ fweisbec: changelog, rebase on struct vtime, field renames, add delta
  on cputime readers, keep idle vtime as-is (low overhead accounting),
  harmonize clock sources. ]

Suggested-by: Thomas Gleixner 
Reported-by: Luiz Capitulino 
Tested-by: Luiz Capitulino 
Signed-off-by: Wanpeng Li 
Signed-off-by: Frederic Weisbecker 
Reviewed-by: Thomas Gleixner 
Acked-by: Rik van Riel 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Wanpeng Li 
Link: http://lkml.kernel.org/r/1498756511-11714-6-git-send-email-fweisbec@gmail.com
Signed-off-by: Ingo Molnar

sched/cputime: Move the vtime task fields to their own struct

2017-07-05T07:54:15+00:00

We are about to add vtime accumulation fields to the task struct. Let's
avoid more bloatification and gather vtime information to their own
struct.

Tested-by: Luiz Capitulino 
Signed-off-by: Frederic Weisbecker 
Reviewed-by: Thomas Gleixner 
Acked-by: Rik van Riel 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Wanpeng Li 
Link: http://lkml.kernel.org/r/1498756511-11714-5-git-send-email-fweisbec@gmail.com
Signed-off-by: Ingo Molnar

sched/cputime: Rename vtime fields

2017-07-05T07:54:14+00:00

The current "snapshot" based naming on vtime fields suggests we record
some past event but that's a low level picture of their actual purpose
which comes out blurry. The real point of these fields is to run a basic
state machine that tracks down cputime entry while switching between
contexts.

So lets reflect that with more meaningful names.

Tested-by: Luiz Capitulino 
Signed-off-by: Frederic Weisbecker 
Reviewed-by: Thomas Gleixner 
Acked-by: Rik van Riel 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Wanpeng Li 
Link: http://lkml.kernel.org/r/1498756511-11714-4-git-send-email-fweisbec@gmail.com
Signed-off-by: Ingo Molnar

sched/cputime: Always set tsk->vtime_snap_whence after accounting vtime

2017-07-05T07:54:14+00:00

Even though it doesn't have functional consequences, setting
the task's new context state after we actually accounted the pending
vtime from the old context state makes more sense from a review
perspective.

vtime_user_exit() is the only function that doesn't follow that rule
and that can bug the reviewer for a little while until he realizes there
is no reason for this special case.

Tested-by: Luiz Capitulino 
Signed-off-by: Frederic Weisbecker 
Reviewed-by: Thomas Gleixner 
Acked-by: Rik van Riel 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Wanpeng Li 
Link: http://lkml.kernel.org/r/1498756511-11714-3-git-send-email-fweisbec@gmail.com
Signed-off-by: Ingo Molnar

vtime, sched/cputime: Remove vtime_account_user()

2017-07-05T07:54:14+00:00

It's an unnecessary function between vtime_user_exit() and
account_user_time().

Tested-by: Luiz Capitulino 
Signed-off-by: Frederic Weisbecker 
Reviewed-by: Thomas Gleixner 
Acked-by: Rik van Riel 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Wanpeng Li 
Link: http://lkml.kernel.org/r/1498756511-11714-2-git-send-email-fweisbec@gmail.com
Signed-off-by: Ingo Molnar

Revert "sched/cputime: Refactor the cputime_adjust() code"

2017-07-04T09:58:05+00:00

This reverts commit 72298e5c92c50edd8cb7cfda4519483ce65fa166.

As Peter explains:

> Argh, no... That code was perfectly fine. The new code otoh is
> convoluted.
>
> The old code had the following form:
>
>         if (exception1)
>           deal with exception1
>
>         if (execption2)
>           deal with exception2
>
>         do normal stuff
>
> Which is as simple and straight forward as it gets.
>
> The new code otoh reads like:
>
>         if (!exception1) {
>                 if (exception2)
>                   deal with exception 2
>                 else
>                   do normal stuff
>         }

So restore the old form.

Also fix the comment describing the logic, as it was confusing.

Requested-by: Peter Zijlstra 
Cc: Gustavo A. R. Silva 
Cc: Frans Klaver 
Cc: Frederic Weisbecker 
Cc: Linus Torvalds 
Cc: Rik van Riel 
Cc: Stanislaw Gruszka 
Cc: Thomas Gleixner 
Cc: Wanpeng Li 
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar

sched/cputime: Refactor the cputime_adjust() code

2017-06-30T07:37:59+00:00

Address a Coverity false positive, which is caused by overly
convoluted code:

Value assigned to variable 'utime' at line 619:utime = rtime;
is overwritten at line 642:utime = rtime - stime; before it
can be used. This makes such variable assignment useless.

Remove this variable assignment and refactor the code related.

Addresses-Coverity-ID: 1371643
Signed-off-by: Gustavo A. R. Silva 
Cc: Frans Klaver 
Cc: Frederic Weisbecker 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Rik van Riel 
Cc: Stanislaw Gruszka 
Cc: Thomas Gleixner 
Cc: Wanpeng Li 
Link: http://lkml.kernel.org/r/20170629184128.GA5271@embeddedgus
Signed-off-by: Ingo Molnar

sched/cputime: Fix ksoftirqd cputime accounting regression

2017-04-27T07:08:26+00:00

irq_time_read() returns the irqtime minus the ksoftirqd time. This
is necessary because irq_time_read() is used to substract the IRQ time
from the sum_exec_runtime of a task. If we were to include the softirq
time of ksoftirqd, this task would substract its own CPU time everytime
it updates ksoftirqd->sum_exec_runtime which would therefore never
progress.

But this behaviour got broken by:

  a499a5a14db ("sched/cputime: Increment kcpustat directly on irqtime account")

... which now includes ksoftirqd softirq time in the time returned by
irq_time_read().

This has resulted in wrong ksoftirqd cputime reported to userspace
through /proc/stat and thus "top" not showing ksoftirqd when it should
after intense networking load.

ksoftirqd->stime happens to be correct but it gets scaled down by
sum_exec_runtime through task_cputime_adjusted().

To fix this, just account the strict IRQ time in a separate counter and
use it to report the IRQ time.

Reported-and-tested-by: Jesper Dangaard Brouer 
Signed-off-by: Frederic Weisbecker 
Reviewed-by: Rik van Riel 
Acked-by: Jesper Dangaard Brouer 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Stanislaw Gruszka 
Cc: Thomas Gleixner 
Cc: Wanpeng Li 
Link: http://lkml.kernel.org/r/1493129448-5356-1-git-send-email-fweisbec@gmail.com
Signed-off-by: Ingo Molnar

sched/headers: Prepare to move cputime functionality from into

2017-03-02T07:42:39+00:00

Introduce a trivial, mostly empty  header
to prepare for the moving of cputime functionality out of sched.h.

Update all code that relies on these facilities.

Acked-by: Linus Torvalds 
Cc: Mike Galbraith 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar