linux-stable.git/include/linux/percpu-rwsem.h, branch v6.15.2

locking/percpu-rwsem: Add guard support

2025-03-17T10:23:36+00:00

To simplify the usage of the percpu rw semaphore.

Signed-off-by: Peter Zijlstra (Intel) 
Signed-off-by: Kan Liang 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://lore.kernel.org/r/20250314172700.438923-2-kan.liang@linux.intel.com

percpu-rwsem: remove the unused parameter 'read'

2024-08-19T11:45:03+00:00

In the function percpu_rwsem_release, the parameter `read`
is unused, so remove it.

Signed-off-by: Wang Long 
Link: https://lore.kernel.org/r/20240802091901.2546797-1-w@laoqinren.net
Reviewed-by: Jan Kara 
Signed-off-by: Christian Brauner

locking/percpu-rwsem: Add percpu_is_write_locked() and percpu_is_read_locked()

2022-08-30T08:56:23+00:00

Implement simple accessors to probe percpu-rwsem's locked state:
percpu_is_write_locked(), percpu_is_read_locked().

Signed-off-by: Marco Elver 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Dmitry Vyukov 
Acked-by: Ian Rogers 
Link: https://lore.kernel.org/r/20220829124719.675715-11-elver@google.com

locking/percpu-rwsem: Use this_cpu_{inc,dec}() for read_count

2020-09-16T14:26:56+00:00

The __this_cpu*() accessors are (in general) IRQ-unsafe which, given
that percpu-rwsem is a blocking primitive, should be just fine.

However, file_end_write() is used from IRQ context and will cause
load-store issues on architectures where the per-cpu accessors are not
natively irq-safe.

Fix it by using the IRQ-safe this_cpu_*() for operations on
read_count. This will generate more expensive code on a number of
platforms, which might cause a performance regression for some of the
other percpu-rwsem users.

If any such is reported, we can consider alternative solutions.

Fixes: 70fe2f48152e ("aio: fix freeze protection of aio writes")
Signed-off-by: Hou Tao 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Acked-by: Oleg Nesterov 
Link: https://lkml.kernel.org/r/20200915140750.137881-1-houtao1@huawei.com

locking/percpu-rwsem: Fold __percpu_up_read()

2020-02-11T12:10:58+00:00

Now that __percpu_up_read() is only ever used from percpu_up_read()
merge them, it's a small function.

Signed-off-by: Davidlohr Bueso 
Signed-off-by: Peter Zijlstra (Intel) 
Signed-off-by: Ingo Molnar 
Acked-by: Will Deacon 
Acked-by: Waiman Long 
Link: https://lkml.kernel.org/r/20200131151540.212415454@infradead.org

locking/percpu-rwsem: Remove the embedded rwsem

2020-02-11T12:10:56+00:00

The filesystem freezer uses percpu-rwsem in a way that is effectively
write_non_owner() and achieves this with a few horrible hacks that
rely on the rwsem (!percpu) implementation.

When PREEMPT_RT replaces the rwsem implementation with a PI aware
variant this comes apart.

Remove the embedded rwsem and implement it using a waitqueue and an
atomic_t.

 - make readers_block an atomic, and use it, with the waitqueue
   for a blocking test-and-set write-side.

 - have the read-side wait for the 'lock' state to clear.

Have the waiters use FIFO queueing and mark them (reader/writer) with
a new WQ_FLAG. Use a custom wake_function to wake either a single
writer or all readers until a writer.

Signed-off-by: Peter Zijlstra (Intel) 
Signed-off-by: Ingo Molnar 
Reviewed-by: Davidlohr Bueso 
Acked-by: Will Deacon 
Acked-by: Waiman Long 
Tested-by: Juri Lelli 
Link: https://lkml.kernel.org/r/20200204092403.GB14879@hirez.programming.kicks-ass.net

locking/percpu-rwsem: Move __this_cpu_inc() into the slowpath

2020-02-11T12:10:54+00:00

As preparation to rework __percpu_down_read() move the
__this_cpu_inc() into it.

Signed-off-by: Peter Zijlstra (Intel) 
Signed-off-by: Ingo Molnar 
Reviewed-by: Davidlohr Bueso 
Acked-by: Will Deacon 
Acked-by: Waiman Long 
Tested-by: Juri Lelli 
Link: https://lkml.kernel.org/r/20200131151540.041600199@infradead.org

locking/percpu-rwsem: Convert to bool

2020-02-11T12:10:54+00:00

Use bool where possible.

Signed-off-by: Peter Zijlstra (Intel) 
Signed-off-by: Ingo Molnar 
Reviewed-by: Davidlohr Bueso 
Acked-by: Will Deacon 
Acked-by: Waiman Long 
Tested-by: Juri Lelli 
Link: https://lkml.kernel.org/r/20200131151539.984626569@infradead.org

locking/percpu-rwsem, lockdep: Make percpu-rwsem use its own lockdep_map

2020-02-11T12:10:53+00:00

As preparation for replacing the embedded rwsem, give percpu-rwsem its
own lockdep_map.

Signed-off-by: Peter Zijlstra (Intel) 
Signed-off-by: Ingo Molnar 
Reviewed-by: Davidlohr Bueso 
Acked-by: Will Deacon 
Acked-by: Waiman Long 
Tested-by: Juri Lelli 
Link: https://lkml.kernel.org/r/20200131151539.927625541@infradead.org

locking/lockdep: Remove unused @nested argument from lock_release()

2019-10-09T10:46:10+00:00

Since the following commit:

  b4adfe8e05f1 ("locking/lockdep: Remove unused argument in __lock_release")

@nested is no longer used in lock_release(), so remove it from all
lock_release() calls and friends.

Signed-off-by: Qian Cai 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Will Deacon 
Acked-by: Daniel Vetter 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: airlied@linux.ie
Cc: akpm@linux-foundation.org
Cc: alexander.levin@microsoft.com
Cc: daniel@iogearbox.net
Cc: davem@davemloft.net
Cc: dri-devel@lists.freedesktop.org
Cc: duyuyang@gmail.com
Cc: gregkh@linuxfoundation.org
Cc: hannes@cmpxchg.org
Cc: intel-gfx@lists.freedesktop.org
Cc: jack@suse.com
Cc: jlbec@evilplan.or
Cc: joonas.lahtinen@linux.intel.com
Cc: joseph.qi@linux.alibaba.com
Cc: jslaby@suse.com
Cc: juri.lelli@redhat.com
Cc: maarten.lankhorst@linux.intel.com
Cc: mark@fasheh.com
Cc: mhocko@kernel.org
Cc: mripard@kernel.org
Cc: ocfs2-devel@oss.oracle.com
Cc: rodrigo.vivi@intel.com
Cc: sean@poorly.run
Cc: st@kernel.org
Cc: tj@kernel.org
Cc: tytso@mit.edu
Cc: vdavydov.dev@gmail.com
Cc: vincent.guittot@linaro.org
Cc: viro@zeniv.linux.org.uk
Link: https://lkml.kernel.org/r/1568909380-32199-1-git-send-email-cai@lca.pw
Signed-off-by: Ingo Molnar