linux.git/kernel/rcu/tree.h, branch v4.11

sched/headers: Prepare for the removal of from

2017-03-02T07:42:32+00:00

Fix up missing #includes in other places that rely on sched.h doing that for them.

Acked-by: Linus Torvalds 
Cc: Mike Galbraith 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar

Merge branches 'doc.2017.01.15b', 'dyntick.2017.01.23a', 'fixes.2017.01.23a', 'srcu.2017.01.25a' and 'torture.2017.01.15b' into HEAD

2017-01-25T20:56:05+00:00

doc.2017.01.15b: Documentation updates
dyntick.2017.01.23a: Dyntick tracking consolidation
fixes.2017.01.23a: Miscellaneous fixes
srcu.2017.01.25a: SRCU rewrite, fixes, and verification
torture.2017.01.15b: Torture-test updates

srcu: Force full grace-period ordering

2017-01-25T20:54:22+00:00

If a process invokes synchronize_srcu(), is delayed just the right amount
of time, and thus does not sleep when waiting for the grace period to
complete, there is no ordering between the end of the grace period and
the code following the synchronize_srcu().  Similarly, there can be a
lack of ordering between the end of the SRCU grace period and callback
invocation.

This commit adds the necessary ordering.

Reported-by: Lance Roy 
Signed-off-by: Paul E. McKenney 
[ paulmck: Further smp_mb() adjustment per email with Lance Roy. ]

rcu: Abstract extended quiescent state determination

2017-01-23T19:44:18+00:00

This commit is the fourth step towards full abstraction of all accesses
to the ->dynticks counter, implementing previously open-coded checks and
comparisons in new rcu_dynticks_in_eqs() and rcu_dynticks_in_eqs_since()
functions.  This abstraction will ease changes to the ->dynticks counter
operation.

Signed-off-by: Paul E. McKenney 
Reviewed-by: Josh Triplett

rcu: Eliminate unused expedited_normal counter

2017-01-23T19:37:14+00:00

Expedited grace periods no longer fall back to normal grace periods
in response to lock contention, given that expedited grace periods
now use the rcu_node tree so as to avoid contention.  This commit
therfore removes the expedited_normal counter.

Signed-off-by: Paul E. McKenney 
Reviewed-by: Josh Triplett

rcu: Make expedited grace periods recheck dyntick idle state

2016-11-14T18:46:31+00:00

Expedited grace periods check dyntick-idle state, and avoid sending
IPIs to idle CPUs, including those running guest OSes, and, on NOHZ_FULL
kernels, nohz_full CPUs.  However, the kernel has been observed checking
a CPU while it was non-idle, but sending the IPI after it has gone
idle.  This commit therefore rechecks idle state immediately before
sending the IPI, refraining from IPIing CPUs that have since gone idle.

Reported-by: Rik van Riel 
Signed-off-by: Paul E. McKenney

rcu: Drive expedited grace periods from workqueue

2016-08-22T16:30:25+00:00

The current implementation of expedited grace periods has the user
task drive the grace period.  This works, but has downsides: (1) The
user task must awaken tasks piggybacking on this grace period, which
can result in latencies rivaling that of the grace period itself, and
(2) User tasks can receive signals, which interfere with RCU CPU stall
warnings.

This commit therefore uses workqueues to drive the grace periods, so
that the user task need not do the awakening.  A subsequent commit
will remove the now-unnecessary code allowing for signals.

Signed-off-by: Paul E. McKenney

rcu: Correctly handle sparse possible cpus

2016-06-15T23:00:05+00:00

In many cases in the RCU tree code, we iterate over the set of cpus for
a leaf node described by rcu_node::grplo and rcu_node::grphi, checking
per-cpu data for each cpu in this range. However, if the set of possible
cpus is sparse, some cpus described in this range are not possible, and
thus no per-cpu region will have been allocated (or initialised) for
them by the generic percpu code.

Erroneous accesses to a per-cpu area for these !possible cpus may fault
or may hit other data depending on the addressed generated when the
erroneous per cpu offset is applied. In practice, both cases have been
observed on arm64 hardware (the former being silent, but detectable with
additional patches).

To avoid issues resulting from this, we must iterate over the set of
*possible* cpus for a given leaf node. This patch add a new helper,
for_each_leaf_node_possible_cpu, to enable this. As iteration is often
intertwined with rcu_node local bitmask manipulation, a new
leaf_node_cpu_bit helper is added to make this simpler and more
consistent. The RCU tree code is made to use both of these where
appropriate.

Without this patch, running reboot at a shell can result in an oops
like:

[ 3369.075979] Unable to handle kernel paging request at virtual address ffffff8008b21b4c
[ 3369.083881] pgd = ffffffc3ecdda000
[ 3369.087270] [ffffff8008b21b4c] *pgd=00000083eca48003, *pud=00000083eca48003, *pmd=0000000000000000
[ 3369.096222] Internal error: Oops: 96000007 [#1] PREEMPT SMP
[ 3369.101781] Modules linked in:
[ 3369.104825] CPU: 2 PID: 1817 Comm: NetworkManager Tainted: G        W       4.6.0+ #3
[ 3369.121239] task: ffffffc0fa13e000 ti: ffffffc3eb940000 task.ti: ffffffc3eb940000
[ 3369.128708] PC is at sync_rcu_exp_select_cpus+0x188/0x510
[ 3369.134094] LR is at sync_rcu_exp_select_cpus+0x104/0x510
[ 3369.139479] pc : [] lr : [] pstate: 200001c5
[ 3369.146860] sp : ffffffc3eb9435a0
[ 3369.150162] x29: ffffffc3eb9435a0 x28: ffffff8008be4f88
[ 3369.155465] x27: ffffff8008b66c80 x26: ffffffc3eceb2600
[ 3369.160767] x25: 0000000000000001 x24: ffffff8008be4f88
[ 3369.166070] x23: ffffff8008b51c3c x22: ffffff8008b66c80
[ 3369.171371] x21: 0000000000000001 x20: ffffff8008b21b40
[ 3369.176673] x19: ffffff8008b66c80 x18: 0000000000000000
[ 3369.181975] x17: 0000007fa951a010 x16: ffffff80086a30f0
[ 3369.187278] x15: 0000007fa9505590 x14: 0000000000000000
[ 3369.192580] x13: ffffff8008b51000 x12: ffffffc3eb940000
[ 3369.197882] x11: 0000000000000006 x10: ffffff8008b51b78
[ 3369.203184] x9 : 0000000000000001 x8 : ffffff8008be4000
[ 3369.208486] x7 : ffffff8008b21b40 x6 : 0000000000001003
[ 3369.213788] x5 : 0000000000000000 x4 : ffffff8008b27280
[ 3369.219090] x3 : ffffff8008b21b4c x2 : 0000000000000001
[ 3369.224406] x1 : 0000000000000001 x0 : 0000000000000140
...
[ 3369.972257] [] sync_rcu_exp_select_cpus+0x188/0x510
[ 3369.978685] [] synchronize_rcu_expedited+0x64/0xa8
[ 3369.985026] [] synchronize_net+0x24/0x30
[ 3369.990499] [] dev_deactivate_many+0x28c/0x298
[ 3369.996493] [] __dev_close_many+0x60/0xd0
[ 3370.002052] [] __dev_close+0x28/0x40
[ 3370.007178] [] __dev_change_flags+0x8c/0x158
[ 3370.012999] [] dev_change_flags+0x20/0x60
[ 3370.018558] [] do_setlink+0x288/0x918
[ 3370.023771] [] rtnl_newlink+0x398/0x6a8
[ 3370.029158] [] rtnetlink_rcv_msg+0xe4/0x220
[ 3370.034891] [] netlink_rcv_skb+0xc4/0xf8
[ 3370.040364] [] rtnetlink_rcv+0x2c/0x40
[ 3370.045663] [] netlink_unicast+0x160/0x238
[ 3370.051309] [] netlink_sendmsg+0x2f0/0x358
[ 3370.056956] [] sock_sendmsg+0x18/0x30
[ 3370.062168] [] ___sys_sendmsg+0x26c/0x280
[ 3370.067728] [] __sys_sendmsg+0x44/0x88
[ 3370.073027] [] SyS_sendmsg+0x10/0x20
[ 3370.078153] [] el0_svc_naked+0x24/0x28

Signed-off-by: Mark Rutland 
Reported-by: Dennis Chen 
Cc: Catalin Marinas 
Cc: Josh Triplett 
Cc: Lai Jiangshan 
Cc: Mathieu Desnoyers 
Cc: Steve Capper 
Cc: Steven Rostedt 
Cc: Will Deacon 
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Paul E. McKenney

Merge branches 'doc.2016.04.19a', 'exp.2016.03.31d', 'fixes.2016.03.31d' and 'torture.2016.04.21a' into HEAD

2016-04-21T20:48:20+00:00

doc.2016.04.19a: Documentation updates
exp.2016.03.31d: Expedited grace-period updates
fixes.2016.03.31d: Miscellaneous fixes
torture.2016.004.21a Torture-test updates

rcu: Awaken grace-period kthread if too long since FQS

2016-03-31T20:34:50+00:00

Recent kernels can fail to awaken the grace-period kthread for
quiescent-state forcing.  This commit is a crude hack that does
a wakeup if a scheduling-clock interrupt sees that it has been
too long since force-quiescent-state (FQS) processing.

Signed-off-by: Paul E. McKenney