linux.git/include/linux/rwsem.h, branch v3.16-rc5

locking/rwsem: Fix warnings for CONFIG_RWSEM_GENERIC_SPINLOCK

2014-06-05T08:38:45+00:00

Optimistic spinning is only used by the xadd variant
of rw-semaphores. Make sure that we use the old version
of the __RWSEM_INITIALIZER macro for systems that rely
on the spinlock one, otherwise warnings can be triggered,
such as the following reported on an arm box:

  ipc/ipcns_notifier.c:22:8: warning: excess elements in struct initializer [enabled by default]
  ipc/ipcns_notifier.c:22:8: warning: (near initialization for 'ipcns_chain.rwsem') [enabled by default]
  ipc/ipcns_notifier.c:22:8: warning: excess elements in struct initializer [enabled by default]
  ipc/ipcns_notifier.c:22:8: warning: (near initialization for 'ipcns_chain.rwsem') [enabled by default]

Signed-off-by: Davidlohr Bueso 
Signed-off-by: Peter Zijlstra 
Cc: Tim Chen 
Cc: Linus Torvalds 
Cc: Paul McKenney 
Cc: Michel Lespinasse 
Cc: Peter Hurley 
Cc: Alex Shi 
Cc: Rik van Riel 
Cc: Andrew Morton 
Cc: Andrea Arcangeli 
Cc: "H. Peter Anvin" 
Cc: Jason Low 
Cc: Andi Kleen 
Cc: Chris Mason 
Cc: Josef Bacik 
Link: http://lkml.kernel.org/r/1400545677.6399.10.camel@buesod1.americas.hpqcorp.net
Signed-off-by: Ingo Molnar

locking/rwsem: Support optimistic spinning

2014-06-05T08:38:21+00:00

We have reached the point where our mutexes are quite fine tuned
for a number of situations. This includes the use of heuristics
and optimistic spinning, based on MCS locking techniques.

Exclusive ownership of read-write semaphores are, conceptually,
just about the same as mutexes, making them close cousins. To
this end we need to make them both perform similarly, and
right now, rwsems are simply not up to it. This was discovered
by both reverting commit 4fc3f1d6 (mm/rmap, migration: Make
rmap_walk_anon() and try_to_unmap_anon() more scalable) and
similarly, converting some other mutexes (ie: i_mmap_mutex) to
rwsems. This creates a situation where users have to choose
between a rwsem and mutex taking into account this important
performance difference. Specifically, biggest difference between
both locks is when we fail to acquire a mutex in the fastpath,
optimistic spinning comes in to play and we can avoid a large
amount of unnecessary sleeping and overhead of moving tasks in
and out of wait queue. Rwsems do not have such logic.

This patch, based on the work from Tim Chen and I, adds support
for write-side optimistic spinning when the lock is contended.
It also includes support for the recently added cancelable MCS
locking for adaptive spinning. Note that is is only applicable
to the xadd method, and the spinlock rwsem variant remains intact.

Allowing optimistic spinning before putting the writer on the wait
queue reduces wait queue contention and provided greater chance
for the rwsem to get acquired. With these changes, rwsem is on par
with mutex. The performance benefits can be seen on a number of
workloads. For instance, on a 8 socket, 80 core 64bit Westmere box,
aim7 shows the following improvements in throughput:

 +--------------+---------------------+-----------------+
 |   Workload   | throughput-increase | number of users |
 +--------------+---------------------+-----------------+
 | alltests     | 20%                 | >1000           |
 | custom       | 27%, 60%            | 10-100, >1000   |
 | high_systime | 36%, 30%            | >100, >1000     |
 | shared       | 58%, 29%            | 10-100, >1000   |
 +--------------+---------------------+-----------------+

There was also improvement on smaller systems, such as a quad-core
x86-64 laptop running a 30Gb PostgreSQL (pgbench) workload for up
to +60% in throughput for over 50 clients. Additionally, benefits
were also noticed in exim (mail server) workloads. Furthermore, no
performance regression have been seen at all.

Based-on-work-from: Tim Chen 
Signed-off-by: Davidlohr Bueso 
[peterz: rej fixup due to comment patches, sched/rt.h header]
Signed-off-by: Peter Zijlstra 
Cc: Alex Shi 
Cc: Andi Kleen 
Cc: Michel Lespinasse 
Cc: Rik van Riel 
Cc: Peter Hurley 
Cc: "Paul E.McKenney" 
Cc: Jason Low 
Cc: Aswin Chandramouleeswaran 
Cc: Andrew Morton 
Cc: Linus Torvalds 
Cc: "Scott J Norton" 
Cc: Andrea Arcangeli 
Cc: Chris Mason 
Cc: Josef Bacik 
Link: http://lkml.kernel.org/r/1399055055.6275.15.camel@buesod1.americas.hpqcorp.net
Signed-off-by: Ingo Molnar

rwsem: add rwsem_is_contended

2014-01-28T21:20:09+00:00

Btrfs needs a simple way to know if it needs to let go of it's read lock on a
rwsem.  Introduce rwsem_is_contended to check to see if there are any waiters on
this rwsem currently.  This is just a hueristic, it is meant to be light and not
100% accurate and called by somebody already holding on to the rwsem in either
read or write.  Thanks,

Signed-off-by: Josef Bacik 
Signed-off-by: Chris Mason 
Acked-by: Ingo Molnar

Revert "rw_semaphore: remove up/down_read_non_owner"

2013-03-23T22:53:52+00:00

This reverts commit 11b80f459adaf91a712f95e7734a17655a36bf30.

Bcache needs rw semaphores for cache coherency in writeback mode -
writes have to take a read lock on a per cache device rw sem, and
release it when the bio completes.

But since this is for bios it's naturally not in the context of the
process that originally took the lock.

Signed-off-by: Kent Overstreet 
CC: Christoph Hellwig 
CC: David Howells

lockdep, rwsem: fix down_write_nest_lock() if !CONFIG_DEBUG_LOCK_ALLOC

2013-01-16T20:13:20+00:00

Commit 1b963c81b145 ("lockdep, rwsem: provide down_write_nest_lock()")
contains a bug in a codepath when CONFIG_DEBUG_LOCK_ALLOC is disabled,
which causes down_read() to be called instead of down_write() by mistake
on such configurations.  Fix that.

Reported-and-tested-by: Andrew Clayton 
Reported-and-tested-by: Zlatko Calusic 
Signed-off-by: Jiri Kosina 
Reviewed-by: Rik van Riel 
Signed-off-by: Linus Torvalds

lockdep, rwsem: provide down_write_nest_lock()

2013-01-11T22:54:55+00:00

down_write_nest_lock() provides a means to annotate locking scenario
where an outer lock is guaranteed to serialize the order nested locks
are being acquired.

This is analogoue to already existing mutex_lock_nest_lock() and
spin_lock_nest_lock().

Signed-off-by: Jiri Kosina 
Cc: Rik van Riel 
Cc: Ingo Molnar 
Cc: Peter Zijlstra 
Cc: Mel Gorman 
Tested-by: Sedat Dilek 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

Remove all #inclusions of asm/system.h

2012-03-28T17:30:03+00:00

Remove all #inclusions of asm/system.h preparatory to splitting and killing
it.  Performed with the following command:

perl -p -i -e 's!^#\s*include\s*.*\n!!' `grep -Irl '^#\s*include\s*' *`

Signed-off-by: David Howells

locking, rwsem: Annotate inner lock as raw

2011-09-13T09:11:59+00:00

There is no reason to allow the lock protecting rwsems (the
ownerless variant) to be preemptible on -rt. Convert it to raw.

In mainline this change documents the low level nature of
the lock - otherwise there's no functional difference. Lockdep
and Sparse checking will work as usual.

Signed-off-by: Thomas Gleixner 
Signed-off-by: Ingo Molnar

atomic: use

2011-07-26T23:49:47+00:00

This allows us to move duplicated code in 
(atomic_inc_not_zero() for now) to 

Signed-off-by: Arun Sharma 
Reviewed-by: Eric Dumazet 
Cc: Ingo Molnar 
Cc: David Miller 
Cc: Eric Dumazet 
Acked-by: Mike Frysinger 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

rw_semaphore: remove up/down_read_non_owner

2011-07-21T00:47:47+00:00

Now that the last users is gone these can be removed.

Signed-off-by: Christoph Hellwig 
Signed-off-by: Al Viro