linux.git/kernel/sched/idle.c, branch v5.10

sched/idle: Fix arch_cpu_idle() vs tracing

2020-11-24T15:47:35+00:00

We call arch_cpu_idle() with RCU disabled, but then use
local_irq_{en,dis}able(), which invokes tracing, which relies on RCU.

Switch all arch_cpu_idle() implementations to use
raw_local_irq_{en,dis}able() and carefully manage the
lockdep,rcu,tracing state like we do in entry.

(XXX: we really should change arch_cpu_idle() to not return with
interrupts enabled)

Reported-by: Sven Schnelle 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Mark Rutland 
Tested-by: Mark Rutland 
Link: https://lkml.kernel.org/r/20201120114925.594122626@infradead.org

treewide: Convert macro and uses of section(foo) to section("foo")

2020-10-25T21:51:49+00:00

Use a more generic form for __section that requires quotes to avoid
complications with clang and gcc differences.

Remove the quote operator # from compiler_attributes.h __section macro.

Convert all unquoted __section(foo) uses to quoted __section("foo").
Also convert __attribute__((section("foo"))) uses to __section("foo")
even if the __attribute__ has multiple list entry forms.

Conversion done using the script at:

    https://lore.kernel.org/lkml/75393e5ddc272dc7403de74d645e6c6e0f4e70eb.camel@perches.com/2-convert_section.pl

Signed-off-by: Joe Perches 
Reviewed-by: Nick Desaulniers 
Reviewed-by: Miguel Ojeda 
Signed-off-by: Linus Torvalds

cpuidle: Move trace_cpu_idle() into generic code

2020-08-26T10:41:54+00:00

Remove trace_cpu_idle() from the arch_cpu_idle() implementations and
put it in the generic code, right before disabling RCU. Gets rid of
more trace_*_rcuidle() users.

Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Steven Rostedt (VMware) 
Reviewed-by: Thomas Gleixner 
Acked-by: Rafael J. Wysocki 
Tested-by: Marco Elver 
Link: https://lkml.kernel.org/r/20200821085348.428433395@infradead.org

sched,idle,rcu: Push rcu_idle deeper into the idle path

2020-08-26T10:41:53+00:00

Lots of things take locks, due to a wee bug, rcu_lockdep didn't notice
that the locking tracepoints were using RCU.

Push rcu_idle_{enter,exit}() as deep as possible into the idle paths,
this also resolves a lot of _rcuidle()/RCU_NONIDLE() usage.

Specifically, sched_clock_idle_wakeup_event() will use ktime which
will use seqlocks which will tickle lockdep, and
stop_critical_timings() uses lock.

Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Steven Rostedt (VMware) 
Reviewed-by: Thomas Gleixner 
Acked-by: Rafael J. Wysocki 
Tested-by: Marco Elver 
Link: https://lkml.kernel.org/r/20200821085348.310943801@infradead.org

Merge branch 'sched/urgent'

2020-07-08T09:38:59+00:00

cpuidle: Rearrange s2idle-specific idle state entry code

2020-06-25T11:52:53+00:00

Implement call_cpuidle_s2idle() in analogy with call_cpuidle()
for the s2idle-specific idle state entry and invoke it from
cpuidle_idle_call() to make the s2idle-specific idle entry code
path look more similar to the "regular" idle entry one.

No intentional functional impact.

Signed-off-by: Rafael J. Wysocki 
Acked-by: Chen Yu

sched: Remove struct sched_class::next field

2020-06-25T11:45:44+00:00

Now that the sched_class descriptors are defined in order via the linker
script vmlinux.lds.h, there's no reason to have a "next" pointer to the
previous priroity structure. The order of the sturctures can be aligned as
an array, and used to index and find the next sched_class descriptor.

Signed-off-by: Steven Rostedt (VMware) 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://lkml.kernel.org/r/20191219214558.845353593@goodmis.org

sched: Force the address order of each sched class descriptor

2020-06-25T11:45:43+00:00

In order to make a micro optimization in pick_next_task(), the order of the
sched class descriptor address must be in the same order as their priority
to each other. That is:

 &idle_sched_class < &fair_sched_class < &rt_sched_class <
 &dl_sched_class < &stop_sched_class

In order to guarantee this order of the sched class descriptors, add each
one into their own data section and force the order in the linker script.

Signed-off-by: Steven Rostedt (VMware) 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://lore.kernel.org/r/157675913272.349305.8936736338884044103.stgit@localhost.localdomain

sched/idle,stop: Remove .get_rr_interval from sched_class

2020-06-15T12:10:01+00:00

The idle task and stop task sched_classes return 0 in this function.

The single call site in sched_rr_get_interval() calls
p->sched_class->get_rr_interval() only conditional in case it is
defined. Otherwise time_slice=0 will be used.

The deadline sched class does not define it. Commit a57beec5d427
("sched: Make sched_class::get_rr_interval() optional") introduced
the default time-slice=0 for sched classes which do not provide this
function.

So .get_rr_interval for idle and stop sched_class can be removed to
shrink the code a little.

Signed-off-by: Dietmar Eggemann 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://lkml.kernel.org/r/20200603080304.16548-4-dietmar.eggemann@arm.com

sched: Replace rq::wake_list

2020-05-28T08:54:16+00:00

The recent commit: 90b5363acd47 ("sched: Clean up scheduler_ipi()")
got smp_call_function_single_async() subtly wrong. Even though it will
return -EBUSY when trying to re-use a csd, that condition is not
atomic and still requires external serialization.

The change in ttwu_queue_remote() got this wrong.

While on first reading ttwu_queue_remote() has an atomic test-and-set
that appears to serialize the use, the matching 'release' is not in
the right place to actually guarantee this serialization.

The actual race is vs the sched_ttwu_pending() call in the idle loop;
that can run the wakeup-list without consuming the CSD.

Instead of trying to chain the lists, merge them.

Signed-off-by: Peter Zijlstra (Intel) 
Signed-off-by: Ingo Molnar 
Link: https://lore.kernel.org/r/20200526161908.129371594@infradead.org