linux.git/include/linux/rcupdate.h, branch v6.10-rc2

rcu: Mollify sparse with RCU guard

2024-04-15T17:43:48+00:00

When using "guard(rcu)();" sparse will complain, because even
though it now understands the cleanup attribute, it doesn't
evaluate the calls from it at function exit, and thus doesn't
count the context correctly.

Given that there's a conditional in the resulting code:

  static inline void class_rcu_destructor(class_rcu_t *_T)
  {
      if (_T->lock) {
          rcu_read_unlock();
      }
  }

it seems that even trying to teach sparse to evalulate the
cleanup attribute function it'd still be difficult to really
make it understand the full context here.

Suppress the sparse warning by just releasing the context in
the acquisition part of the function, after all we know it's
safe with the guard, that's the whole point of it.

Signed-off-by: Johannes Berg 
Reviewed-by: Boqun Feng 
Signed-off-by: Paul E. McKenney 
Signed-off-by: Uladzislau Rezki (Sony)

rcu: Remove redundant CONFIG_PROVE_RCU #if condition

2024-04-15T16:12:20+00:00

The #if condition controlling the rcu_preempt_sleep_check() definition
has a redundant check for CONFIG_PREEMPT_RCU, which is already checked
for by an enclosing #ifndef.  This commit therefore removes this redundant
condition from the inner #if.

Signed-off-by: Paul E. McKenney 
Signed-off-by: Uladzislau Rezki (Sony)

rcu: Update lockdep while in RCU read-side critical section

2024-04-15T09:28:59+00:00

With Ankur's lazy-/auto-preemption patches applied and with a
lazy-preemptible kernel in combination with a non-preemptible RCU,
lockdep sometimes complains about context switches within RCU read-side
critical sections.  This is a false positive due to rcu_read_unlock()
updating lockdep state too late:

	__release(RCU);
	__rcu_read_unlock();
	// Context switch here results in lockdep false positive!!!
	rcu_lock_release(&rcu_lock_map); /* Keep acq info for rls diags. */

Although this complaint could also happen with preemptible RCU
in a preemptible kernel, the odds of that happening aer quite low.
In constrast, with non-preemptible RCU, a long critical section has a
high probability of performing a context switch from the preempt_enable()
in __rcu_read_unlock().

The fix is straightforward, just move the rcu_lock_release()
within rcu_read_unlock() to obtain the reverse order from that of
rcu_read_lock():

	rcu_lock_release(&rcu_lock_map); /* Keep acq info for rls diags. */
	__release(RCU);
	__rcu_read_unlock();

This commit makes this change.

Co-developed-by: Frederic Weisbecker 
Signed-off-by: Frederic Weisbecker 
Co-developed-by: Joel Fernandes (Google) 
Signed-off-by: Joel Fernandes (Google) 
Co-developed-by: Boqun Feng 
Signed-off-by: Boqun Feng 
Signed-off-by: Uladzislau Rezki (Sony) 
Reviewed-by: Paul E. McKenney 
Cc: Ankur Arora 
Cc: Thomas Gleixner

rcu: add a helper to report consolidated flavor QS

2024-03-21T04:05:42+00:00

When under heavy load, network processing can run CPU-bound for many
tens of seconds. Even in preemptible kernels (non-RT kernel), this can
block RCU Tasks grace periods, which can cause trace-event removal to
take more than a minute, which is unacceptably long.

This commit therefore creates a new helper function that passes through
both RCU and RCU-Tasks quiescent states every 100 milliseconds. This
hard-coded value suffices for current workloads.

Suggested-by: Paul E. McKenney 
Reviewed-by: Jesper Dangaard Brouer 
Signed-off-by: Yan Zhai 
Reviewed-by: Paul E. McKenney 
Acked-by: Jesper Dangaard Brouer 
Link: https://lore.kernel.org/r/90431d46ee112d2b0af04dbfe936faaca11810a5.1710877680.git.yan@cloudflare.com
Signed-off-by: Jakub Kicinski

rcu-tasks: Repair RCU Tasks Trace quiescence check

2024-02-14T16:02:08+00:00

The context-switch-time check for RCU Tasks Trace quiescence expects
current->trc_reader_special.b.need_qs to be zero, and if so, updates
it to TRC_NEED_QS_CHECKED.  This is backwards, because if this value
is zero, there is no RCU Tasks Trace grace period in flight, an thus
no need for a quiescent state.  Instead, when a grace period starts,
this field is set to TRC_NEED_QS.

This commit therefore changes the check from zero to TRC_NEED_QS.

Reported-by: Steven Rostedt 
Signed-off-by: Paul E. McKenney 
Tested-by: Steven Rostedt (Google) 
Signed-off-by: Boqun Feng

Merge branches 'doc.2023.12.13a', 'torture.2023.11.23a', 'fixes.2023.12.13a', 'rcu-tasks.2023.12.12b' and 'srcu.2023.12.13a' into rcu-merge.2023.12.13a

2023-12-13T19:51:31+00:00

srcu: Use try-lock lockdep annotation for NMI-safe access.

2023-12-13T18:11:19+00:00

It is claimed that srcu_read_lock_nmisafe() NMI-safe. However it
triggers a lockdep if used from NMI because lockdep expects a deadlock
since nothing disables NMIs while the lock is acquired.

This is because commit f0f44752f5f61 ("rcu: Annotate SRCU's update-side
lockdep dependencies") annotates synchronize_srcu() as a write lock
usage. This helps to detect a deadlocks such as
	srcu_read_lock();
	synchronize_srcu();
	srcu_read_unlock();

The side effect is that the lock srcu_struct now has a USED usage in normal
contexts, so it conflicts with a USED_READ usage in NMI. But this shouldn't
cause a real deadlock because the write lock usage from synchronize_srcu()
is a fake one and only used for read/write deadlock detection.

Use a try-lock annotation for srcu_read_lock_nmisafe() to avoid lockdep
complains if used from NMI.

Fixes: f0f44752f5f6 ("rcu: Annotate SRCU's update-side lockdep dependencies")
Link: https://lore.kernel.org/r/20230927160231.XRCDDSK4@linutronix.de
Reviewed-by: Boqun Feng 
Signed-off-by: Sebastian Andrzej Siewior 
Signed-off-by: Paul E. McKenney 
Signed-off-by: Neeraj Upadhyay (AMD)

rcu: Remove unused macros from rcupdate.h

2023-12-11T21:02:03+00:00

ulong2long, USHORT_CMP_GE and USHORT_CMP_LT are redundant and have been
unused for quite a few releases.

Signed-off-by: Pedro Falcato 
Signed-off-by: Paul E. McKenney 
Signed-off-by: Neeraj Upadhyay (AMD)

rcu: Standardize explicit CPU-hotplug calls

2023-10-04T20:29:45+00:00

rcu_report_dead() and rcutree_migrate_callbacks() have their headers in
rcupdate.h while those are pure rcutree calls, like the other CPU-hotplug
functions.

Also rcu_cpu_starting() and rcu_report_dead() have different naming
conventions while they mirror each other's effects.

Fix the headers and propose a naming that relates both functions and
aligns with the prefix of other rcutree CPU-hotplug functions.

Reviewed-by: Paul E. McKenney 
Signed-off-by: Frederic Weisbecker

rcu: Assume rcu_report_dead() is always called locally

2023-10-04T15:35:56+00:00

rcu_report_dead() has to be called locally by the CPU that is going to
exit the RCU state machine. Passing a cpu argument here is error-prone
and leaves the possibility for a racy remote call.

Use local access instead.

Reviewed-by: Paul E. McKenney 
Signed-off-by: Frederic Weisbecker