linux-stable.git/kernel/rcutree_plugin.h, branch master

rcu: Move RCU-related source code to kernel/rcu directory

2013-10-15T19:53:31+00:00

Signed-off-by: Paul E. McKenney 
Reviewed-by: Ingo Molnar

Merge branch 'idle.2013.09.25a' into HEAD

2013-10-15T19:49:59+00:00

idle.2013.09.25a: Topic branch for idle entry-/exit-related changes.

Merge branch 'gp.2013.09.25a' into HEAD

2013-10-15T19:47:04+00:00

gp.2013.09.25a: Topic branch for grace-period updates.

rcu: Throttle invoke_rcu_core() invocations due to non-lazy callbacks

2013-09-25T13:44:33+00:00

If a non-lazy callback arrives on a CPU that has previously gone idle
with no non-lazy callbacks, invoke_rcu_core() forces the RCU core to
run.  However, it does not update the conditions, which could result
in several closely spaced invocations of the RCU core, which in turn
could result in an excessively high context-switch rate and resulting
high overhead.

This commit therefore updates the ->all_lazy and ->nonlazy_posted_snap
fields to prevent closely spaced invocations.

Reported-by: Tibor Billes 
Signed-off-by: Paul E. McKenney 
Tested-by: Tibor Billes 
Reviewed-by: Josh Triplett

rcu: Throttle rcu_try_advance_all_cbs() execution

2013-09-25T13:44:25+00:00

The rcu_try_advance_all_cbs() function is invoked on each attempted
entry to and every exit from idle.  If this function determines that
there are callbacks ready to invoke, the caller will invoke the RCU
core, which in turn will result in a pair of context switches.  If a
CPU enters and exits idle extremely frequently, this can result in
an excessive number of context switches and high CPU overhead.

This commit therefore causes rcu_try_advance_all_cbs() to throttle
itself, refusing to do work more than once per jiffy.

Reported-by: Tibor Billes 
Signed-off-by: Paul E. McKenney 
Tested-by: Tibor Billes 
Reviewed-by: Josh Triplett

rcu: Remove redundant code from rcu_cleanup_after_idle()

2013-09-25T13:44:03+00:00

The rcu_try_advance_all_cbs() function returns a bool saying whether or
not there are callbacks ready to invoke, but rcu_cleanup_after_idle()
rechecks this regardless.  This commit therefore uses the value returned
by rcu_try_advance_all_cbs() instead of making rcu_cleanup_after_idle()
do this recheck.

Reported-by: Tibor Billes 
Signed-off-by: Paul E. McKenney 
Tested-by: Tibor Billes 
Reviewed-by: Josh Triplett

rcu: Fix CONFIG_RCU_NOCB_CPU_ALL panic on machines with sparse CPU mask

2013-09-23T21:11:11+00:00

Some architectures have sparse cpu mask. UltraSparc's cpuinfo for example:

CPU0: online
CPU2: online

So, set only possible CPUs when CONFIG_RCU_NOCB_CPU_ALL is enabled.

Also, check that user passes right 'rcu_nocbs=' option.

Signed-off-by: Kirill Tkhai 
CC: Dipankar Sarma 
[ paulmck: Fix pr_info() issue noted by scripts/checkpatch.pl. ]
Signed-off-by: Paul E. McKenney

rcu: Track rcu_nocb_kthread()'s sleeping and awakening

2013-09-23T16:18:16+00:00

This commit adds event traces to track all of rcu_nocb_kthread()'s
blocking and awakening.

Signed-off-by: Paul E. McKenney

rcu: Distinguish between NOCB and non-NOCB rcu_callback trace events

2013-09-23T16:18:14+00:00

One way to distinguish between NOCB and non-NOCB rcu_callback trace
events is that the former always print zero for the lazy and non-lazy
queue lengths.  Unfortunately, this also means that we cannot see the NOCB
queue lengths.  This commit therefore accesses the NOCB queue lengths,
but negates them.  NOCB rcu_callback trace events should therefore have
negative queue lengths.

Signed-off-by: Paul E. McKenney 
[ paulmck: Match operand size per kbuild test robot's advice. ]

rcu: Add tracing for rcuo no-CBs CPU wakeup handshake

2013-09-23T16:18:13+00:00

Lost wakeups from call_rcu() to the rcuo kthreads can result in hangs
that are difficult to diagnose.  This commit therefore adds tracing to
help pin down the cause of these hangs.

Reported-by: Clark Williams 
Reported-by: Carsten Emde 
Signed-off-by: Paul E. McKenney 
[ paulmck: Add const per kbuild test robot's advice. ]