linux.git/ipc/util.h, branch v3.12-rc2

ipc: drop ipc_lock_check

2013-09-11T22:59:45+00:00

No remaining users, we now use ipc_obtain_object_check().

Signed-off-by: Davidlohr Bueso 
Cc: Sedat Dilek 
Cc: Rik van Riel 
Cc: Manfred Spraul 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

ipc: drop ipc_lock_by_ptr

2013-09-11T22:59:44+00:00

After previous cleanups and optimizations, this function is no longer
heavily used and we don't have a good reason to keep it.  Update the few
remaining callers and get rid of it.

Signed-off-by: Davidlohr Bueso 
Cc: Sedat Dilek 
Cc: Rik van Riel 
Cc: Manfred Spraul 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

ipc: rename ids->rw_mutex

2013-09-11T22:59:42+00:00

Since in some situations the lock can be shared for readers, we shouldn't
be calling it a mutex, rename it to rwsem.

Signed-off-by: Davidlohr Bueso 
Tested-by: Sedat Dilek 
Cc: Rik van Riel 
Cc: Manfred Spraul 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

ipc: drop ipcctl_pre_down

2013-09-11T22:59:39+00:00

Now that sem, msgque and shm, through *_down(), all use the lockless
variant of ipcctl_pre_down(), go ahead and delete it.

[akpm@linux-foundation.org: fix function name in kerneldoc, cleanups]
Signed-off-by: Davidlohr Bueso 
Tested-by: Sedat Dilek 
Cc: Rik van Riel 
Cc: Manfred Spraul 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

ipc: close open coded spin lock calls

2013-07-09T17:33:27+00:00

Signed-off-by: Davidlohr Bueso 
Cc: Andi Kleen 
Cc: Rik van Riel 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

ipc: introduce ipc object locking helpers

2013-07-09T17:33:27+00:00

Simple helpers around the (kern_ipc_perm *)->lock spinlock.

Signed-off-by: Davidlohr Bueso 
Cc: Andi Kleen 
Cc: Rik van Riel 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

ipc,sem: fine grained locking for semtimedop

2013-05-01T15:12:58+00:00

Introduce finer grained locking for semtimedop, to handle the common case
of a program wanting to manipulate one semaphore from an array with
multiple semaphores.

If the call is a semop manipulating just one semaphore in an array with
multiple semaphores, only take the lock for that semaphore itself.

If the call needs to manipulate multiple semaphores, or another caller is
in a transaction that manipulates multiple semaphores, the sem_array lock
is taken, as well as all the locks for the individual semaphores.

On a 24 CPU system, performance numbers with the semop-multi
test with N threads and N semaphores, look like this:

	vanilla		Davidlohr's	Davidlohr's +	Davidlohr's +
threads			patches		rwlock patches	v3 patches
10	610652		726325		1783589		2142206
20	341570		365699		1520453		1977878
30	288102		307037		1498167		2037995
40	290714		305955		1612665		2256484
50	288620		312890		1733453		2650292
60	289987		306043		1649360		2388008
70	291298		306347		1723167		2717486
80	290948		305662		1729545		2763582
90	290996		306680		1736021		2757524
100	292243		306700		1773700		3059159

[davidlohr.bueso@hp.com: do not call sem_lock when bogus sma]
[davidlohr.bueso@hp.com: make refcounter atomic]
Signed-off-by: Rik van Riel 
Suggested-by: Linus Torvalds 
Acked-by: Davidlohr Bueso 
Cc: Chegu Vinod 
Cc: Jason Low 
Reviewed-by: Michel Lespinasse 
Cc: Peter Hurley 
Cc: Stanislav Kinsbursky 
Tested-by: Emmanuel Benisty 
Tested-by: Sedat Dilek 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

ipc,sem: do not hold ipc lock more than necessary

2013-05-01T15:12:58+00:00

Instead of holding the ipc lock for permissions and security checks, among
others, only acquire it when necessary.

Some numbers....

1) With Rik's semop-multi.c microbenchmark we can see the following
   results:

Baseline (3.9-rc1):
cpus 4, threads: 256, semaphores: 128, test duration: 30 secs
total operations: 151452270, ops/sec 5048409

+  59.40%            a.out  [kernel.kallsyms]  [k] _raw_spin_lock
+   6.14%            a.out  [kernel.kallsyms]  [k] sys_semtimedop
+   3.84%            a.out  [kernel.kallsyms]  [k] avc_has_perm_flags
+   3.64%            a.out  [kernel.kallsyms]  [k] __audit_syscall_exit
+   2.06%            a.out  [kernel.kallsyms]  [k] copy_user_enhanced_fast_string
+   1.86%            a.out  [kernel.kallsyms]  [k] ipc_lock

With this patchset:
cpus 4, threads: 256, semaphores: 128, test duration: 30 secs
total operations: 273156400, ops/sec 9105213

+  18.54%            a.out  [kernel.kallsyms]  [k] _raw_spin_lock
+  11.72%            a.out  [kernel.kallsyms]  [k] sys_semtimedop
+   7.70%            a.out  [kernel.kallsyms]  [k] ipc_has_perm.isra.21
+   6.58%            a.out  [kernel.kallsyms]  [k] avc_has_perm_flags
+   6.54%            a.out  [kernel.kallsyms]  [k] __audit_syscall_exit
+   4.71%            a.out  [kernel.kallsyms]  [k] ipc_obtain_object_check

2) While on an Oracle swingbench DSS (data mining) workload the
   improvements are not as exciting as with Rik's benchmark, we can see
   some positive numbers.  For an 8 socket machine the following are the
   percentages of %sys time incurred in the ipc lock:

Baseline (3.9-rc1):
100 swingbench users: 8,74%
400 swingbench users: 21,86%
800 swingbench users: 84,35%

With this patchset:
100 swingbench users: 8,11%
400 swingbench users: 19,93%
800 swingbench users: 77,69%

[riel@redhat.com: fix two locking bugs]
[sasha.levin@oracle.com: prevent releasing RCU read lock twice in semctl_main]
[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: Davidlohr Bueso 
Signed-off-by: Rik van Riel 
Reviewed-by: Chegu Vinod 
Acked-by: Michel Lespinasse 
Cc: Rik van Riel 
Cc: Jason Low 
Cc: Emmanuel Benisty 
Cc: Peter Hurley 
Cc: Stanislav Kinsbursky 
Tested-by: Sedat Dilek 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

ipc: introduce lockless pre_down ipcctl

2013-05-01T15:12:58+00:00

Various forms of ipc use ipcctl_pre_down() to retrieve an ipc object and
check permissions, mostly for IPC_RMID and IPC_SET commands.

Introduce ipcctl_pre_down_nolock(), a lockless version of this function.
The locking version is retained, yet modified to call the nolock version
without affecting its semantics, thus transparent to all ipc callers.

Signed-off-by: Davidlohr Bueso 
Signed-off-by: Rik van Riel 
Suggested-by: Linus Torvalds 
Cc: Chegu Vinod 
Cc: Emmanuel Benisty 
Cc: Jason Low 
Cc: Michel Lespinasse 
Cc: Peter Hurley 
Cc: Stanislav Kinsbursky 
Tested-by: Sedat Dilek 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

ipc: introduce obtaining a lockless ipc object

2013-05-01T15:12:57+00:00

Through ipc_lock() and therefore ipc_lock_check() we currently return the
locked ipc object.  This is not necessary for all situations and can,
therefore, cause unnecessary ipc lock contention.

Introduce analogous ipc_obtain_object() and ipc_obtain_object_check()
functions that only lookup and return the ipc object.

Both these functions must be called within the RCU read critical section.

[akpm@linux-foundation.org: propagate the ipc_obtain_object() errno from ipc_lock()]
Signed-off-by: Davidlohr Bueso 
Signed-off-by: Rik van Riel 
Reviewed-by: Chegu Vinod 
Acked-by: Michel Lespinasse 
Cc: Emmanuel Benisty 
Cc: Jason Low 
Cc: Peter Hurley 
Cc: Stanislav Kinsbursky 
Tested-by: Sedat Dilek 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds