linux.git/kernel/sched/idle.c, branch v4.11

sched/headers: Prepare for new header dependencies before moving code to

2017-03-02T07:42:26+00:00

We are going to split   out of , which
will have to be picked up from other headers and a couple of .c files.

Create a trivial placeholder  file that just
maps to  to make this patch obviously correct and
bisectable.

Include the new header in the files that are going to need it.

Acked-by: Linus Torvalds 
Cc: Mike Galbraith 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar

sched/idle: Add support for tasks that inject idle

2016-11-29T13:02:21+00:00

Idle injection drivers such as Intel powerclamp and ACPI PAD drivers use
realtime tasks to take control of CPU then inject idle. There are two
issues with this approach:

 1. Low efficiency: injected idle task is treated as busy so sched ticks
    do not stop during injected idle period, the result of these
    unwanted wakeups can be ~20% loss in power savings.

 2. Idle accounting: injected idle time is presented to user as busy.

This patch addresses the issues by introducing a new PF_IDLE flag which
allows any given task to be treated as idle task while the flag is set.
Therefore, idle injection tasks can run through the normal flow of NOHZ
idle enter/exit to get the correct accounting as well as tick stop when
possible.

The implication is that idle task is then no longer limited to PID == 0.

Acked-by: Ingo Molnar 
Signed-off-by: Peter Zijlstra 
Signed-off-by: Jacob Pan 
Signed-off-by: Rafael J. Wysocki

cpuidle: Allow enforcing deepest idle state selection

2016-11-29T13:02:21+00:00

When idle injection is used to cap power, we need to override the
governor's choice of idle states.

For this reason, make it possible the deepest idle state selection to
be enforced by setting a flag on a given CPU to achieve the maximum
potential power draw reduction.

Signed-off-by: Jacob Pan 
[ rjw: Subject & changelog ]
Signed-off-by: Rafael J. Wysocki

nmi_backtrace: generate one-line reports for idle cpus

2016-10-08T01:46:30+00:00

When doing an nmi backtrace of many cores, most of which are idle, the
output is a little overwhelming and very uninformative.  Suppress
messages for cpus that are idling when they are interrupted and just
emit one line, "NMI backtrace for N skipped: idling at pc 0xNNN".

We do this by grouping all the cpuidle code together into a new
.cpuidle.text section, and then checking the address of the interrupted
PC to see if it lies within that section.

This commit suitably tags x86 and tile idle routines, and only adds in
the minimal framework for other architectures.

Link: http://lkml.kernel.org/r/1472487169-14923-5-git-send-email-cmetcalf@mellanox.com
Signed-off-by: Chris Metcalf 
Acked-by: Peter Zijlstra (Intel) 
Tested-by: Peter Zijlstra (Intel) 
Tested-by: Daniel Thompson  [arm]
Tested-by: Petr Mladek 
Cc: Aaron Tomlin 
Cc: Peter Zijlstra (Intel) 
Cc: "Rafael J. Wysocki" 
Cc: Russell King 
Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

Merge branch 'sched/urgent' into sched/core, to pick up fixes

2016-06-14T09:04:13+00:00

Signed-off-by: Ingo Molnar

sched/idle: Optimize the generic idle loop

2016-06-03T07:18:56+00:00

Currently, smp_processor_id() is used to fetch the current CPU in
cpu_idle_loop(). Every time the idle thread runs, it fetches the
current CPU using smp_processor_id().

Since the idle thread is per CPU, the current CPU is constant, so we
can lift the load out of the loop, saving execution cycles/time in the
loop.

x86-64:

Before patch (execution in loop):
	148:    0f ae e8                lfence
	14b:    65 8b 04 25 00 00 00 00 mov %gs:0x0,%eax
	152:    00
	153:    89 c0                   mov %eax,%eax
	155:    49 0f a3 04 24          bt %rax,(%r12)

After patch (execution in loop):
	150:    0f ae e8                lfence
	153:    4d 0f a3 34 24          bt %r14,(%r12)

ARM64:

Before patch (execution in loop):
	168:    d5033d9f        dsb     ld
	16c:    b9405661        ldr     w1,[x19,#84]
	170:    1100fc20        add     w0,w1,#0x3f
	174:    6b1f003f        cmp     w1,wzr
	178:    1a81b000        csel    w0,w0,w1,lt
	17c:    130c7000        asr     w0,w0,#6
	180:    937d7c00        sbfiz   x0,x0,#3,#32
	184:    f8606aa0        ldr     x0,[x21,x0]
	188:    9ac12401        lsr     x1,x0,x1
	18c:    36000e61        tbz     w1,#0,358

After patch (execution in loop):
	1a8:    d50339df        dsb     ld
	1ac:    f8776ac0        ldr     x0,[x22,x23]
	ab0:    ea18001f        tst     x0,x24
	1b4:    54000ea0        b.eq    388

Further observance on ARM64 for 4 seconds shows that cpu_idle_loop is
called 8672 times. Shifting the code will save instructions executed
in loop and eventually time as well.

Signed-off-by: Gaurav Jindal 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Sanjeev Yadav 
Cc: Linus Torvalds 
Cc: Mike Galbraith 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Link: http://lkml.kernel.org/r/20160512101330.GA488@gauravjindalubtnb.del.spreadtrum.com
Signed-off-by: Ingo Molnar

cpuidle: Do not access cpuidle_devices when !CONFIG_CPU_IDLE

2016-06-02T21:05:27+00:00

The cpuidle_devices per-CPU variable is only defined when CPU_IDLE is
enabled. Commit c8cc7d4de7a4 ("sched/idle: Reorganize the idle loop")
removed the #ifdef CONFIG_CPU_IDLE around cpuidle_idle_call() with the
compiler optimising away __this_cpu_read(cpuidle_devices). However, with
CONFIG_UBSAN && !CONFIG_CPU_IDLE, this optimisation no longer happens
and the kernel fails to link since cpuidle_devices is not defined.

This patch introduces an accessor function for the current CPU cpuidle
device (returning NULL when !CONFIG_CPU_IDLE) and uses it in
cpuidle_idle_call().

Signed-off-by: Catalin Marinas 
Cc: 4.5+  # 4.5+
Signed-off-by: Rafael J. Wysocki

rcu: Make CPU_DYING_IDLE an explicit call

2016-03-01T19:36:58+00:00

Make the RCU CPU_DYING_IDLE callback an explicit function call, so it gets
invoked at the proper place.

Signed-off-by: Thomas Gleixner 
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel 
Cc: Rafael Wysocki 
Cc: "Srivatsa S. Bhat" 
Cc: Peter Zijlstra 
Cc: Arjan van de Ven 
Cc: Sebastian Siewior 
Cc: Rusty Russell 
Cc: Steven Rostedt 
Cc: Oleg Nesterov 
Cc: Tejun Heo 
Cc: Andrew Morton 
Cc: Paul McKenney 
Cc: Linus Torvalds 
Cc: Paul Turner 
Link: http://lkml.kernel.org/r/20160226182341.870167933@linutronix.de
Signed-off-by: Thomas Gleixner

cpu/hotplug: Make wait for dead cpu completion based

2016-03-01T19:36:58+00:00

Kill the busy spinning on the control side and just wait for the hotplugged
cpu to tell that it reached the dead state.

Signed-off-by: Thomas Gleixner 
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel 
Cc: Rafael Wysocki 
Cc: "Srivatsa S. Bhat" 
Cc: Peter Zijlstra 
Cc: Arjan van de Ven 
Cc: Sebastian Siewior 
Cc: Rusty Russell 
Cc: Steven Rostedt 
Cc: Oleg Nesterov 
Cc: Tejun Heo 
Cc: Andrew Morton 
Cc: Paul McKenney 
Cc: Linus Torvalds 
Cc: Paul Turner 
Link: http://lkml.kernel.org/r/20160226182341.776157858@linutronix.de
Signed-off-by: Thomas Gleixner

cpu/hotplug: Let upcoming cpu bring itself fully up

2016-03-01T19:36:57+00:00

Let the upcoming cpu kick the hotplug thread and let itself complete the
bringup. That way the controll side can just wait for the completion or later
when we made the hotplug machinery async not care at all.

Signed-off-by: Thomas Gleixner 
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel 
Cc: Rafael Wysocki 
Cc: "Srivatsa S. Bhat" 
Cc: Peter Zijlstra 
Cc: Arjan van de Ven 
Cc: Sebastian Siewior 
Cc: Rusty Russell 
Cc: Steven Rostedt 
Cc: Oleg Nesterov 
Cc: Tejun Heo 
Cc: Andrew Morton 
Cc: Paul McKenney 
Cc: Linus Torvalds 
Cc: Paul Turner 
Link: http://lkml.kernel.org/r/20160226182341.697655464@linutronix.de
Signed-off-by: Thomas Gleixner