linux-stable.git/kernel/sched/membarrier.c, branch linux-5.2.y

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157

2019-05-30T18:26:37+00:00

Based on 3 normalized pattern(s):

  this program is free software you can redistribute it and or modify
  it under the terms of the gnu general public license as published by
  the free software foundation either version 2 of the license or at
  your option any later version this program is distributed in the
  hope that it will be useful but without any warranty without even
  the implied warranty of merchantability or fitness for a particular
  purpose see the gnu general public license for more details

  this program is free software you can redistribute it and or modify
  it under the terms of the gnu general public license as published by
  the free software foundation either version 2 of the license or at
  your option any later version [author] [kishon] [vijay] [abraham]
  [i] [kishon]@[ti] [com] this program is distributed in the hope that
  it will be useful but without any warranty without even the implied
  warranty of merchantability or fitness for a particular purpose see
  the gnu general public license for more details

  this program is free software you can redistribute it and or modify
  it under the terms of the gnu general public license as published by
  the free software foundation either version 2 of the license or at
  your option any later version [author] [graeme] [gregory]
  [gg]@[slimlogic] [co] [uk] [author] [kishon] [vijay] [abraham] [i]
  [kishon]@[ti] [com] [based] [on] [twl6030]_[usb] [c] [author] [hema]
  [hk] [hemahk]@[ti] [com] this program is distributed in the hope
  that it will be useful but without any warranty without even the
  implied warranty of merchantability or fitness for a particular
  purpose see the gnu general public license for more details

extracted by the scancode license scanner the SPDX license identifier

  GPL-2.0-or-later

has been chosen to replace the boilerplate/reference in 1105 file(s).

Signed-off-by: Thomas Gleixner 
Reviewed-by: Allison Randal 
Reviewed-by: Richard Fontana 
Reviewed-by: Kate Stewart 
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190527070033.202006027@linutronix.de
Signed-off-by: Greg Kroah-Hartman

sched/membarrier: synchronize_sched() with synchronize_rcu()

2018-11-27T17:21:43+00:00

Now that synchronize_rcu() waits for preempt-disable regions of code
as well as RCU read-side critical sections, synchronize_sched() can be
replaced by synchronize_rcu().  This commit therefore makes this change.

Signed-off-by: Paul E. McKenney 
Cc: Mathieu Desnoyers 
Cc: Ingo Molnar 
Cc: Peter Zijlstra

sched/membarrier: Replace synchronize_sched() with synchronize_rcu()

2018-11-09T05:43:20+00:00

Now that synchronize_rcu() waits for preempt-disable regions of code
as well as RCU read-side critical sections, the synchronize_sched()
in sys_membarrier() can be replaced by synchronize_rcu().  This commit
therefore makes this change.

Signed-off-by: Paul E. McKenney 
Cc: Mathieu Desnoyers

sched/headers: Simplify and clean up header usage in the scheduler

2018-03-04T11:39:29+00:00

Do the following cleanups and simplifications:

 - sched/sched.h already includes , so no need to
   include it in sched/core.c again.

 - order the  headers alphabetically

 - add all  headers to kernel/sched/sched.h

 - remove all unnecessary includes from the .c files that
   are already included in kernel/sched/sched.h.

Finally, make all scheduler .c files use a single common header:

  #include "sched.h"

... which now contains a union of the relied upon headers.

This makes the various .c files easier to read and easier to handle.

Cc: Linus Torvalds 
Cc: Mike Galbraith 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar

sched: Clean up and harmonize the coding style of the scheduler code base

2018-03-03T14:50:21+00:00

A good number of small style inconsistencies have accumulated
in the scheduler core, so do a pass over them to harmonize
all these details:

 - fix speling in comments,

 - use curly braces for multi-line statements,

 - remove unnecessary parentheses from integer literals,

 - capitalize consistently,

 - remove stray newlines,

 - add comments where necessary,

 - remove invalid/unnecessary comments,

 - align structure definitions and other data types vertically,

 - add missing newlines for increased readability,

 - fix vertical tabulation where it's misaligned,

 - harmonize preprocessor conditional block labeling
   and vertical alignment,

 - remove line-breaks where they uglify the code,

 - add newline after local variable definitions,

No change in functionality:

  md5:
     1191fa0a890cfa8132156d2959d7e9e2  built-in.o.before.asm
     1191fa0a890cfa8132156d2959d7e9e2  built-in.o.after.asm

Cc: Linus Torvalds 
Cc: Mike Galbraith 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar

membarrier: Provide core serializing command, *_SYNC_CORE

2018-02-05T20:35:03+00:00

Provide core serializing membarrier command to support memory reclaim
by JIT.

Each architecture needs to explicitly opt into that support by
documenting in their architecture code how they provide the core
serializing instructions required when returning from the membarrier
IPI, and after the scheduler has updated the curr->mm pointer (before
going back to user-space). They should then select
ARCH_HAS_MEMBARRIER_SYNC_CORE to enable support for that command on
their architecture.

Architectures selecting this feature need to either document that
they issue core serializing instructions when returning to user-space,
or implement their architecture-specific sync_core_before_usermode().

Signed-off-by: Mathieu Desnoyers 
Acked-by: Thomas Gleixner 
Acked-by: Peter Zijlstra (Intel) 
Cc: Andrea Parri 
Cc: Andrew Hunter 
Cc: Andy Lutomirski 
Cc: Avi Kivity 
Cc: Benjamin Herrenschmidt 
Cc: Boqun Feng 
Cc: Dave Watson 
Cc: David Sehr 
Cc: Greg Hackmann 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Cc: Maged Michael 
Cc: Michael Ellerman 
Cc: Paul E. McKenney 
Cc: Paul Mackerras 
Cc: Russell King 
Cc: Will Deacon 
Cc: linux-api@vger.kernel.org
Cc: linux-arch@vger.kernel.org
Link: http://lkml.kernel.org/r/20180129202020.8515-9-mathieu.desnoyers@efficios.com
Signed-off-by: Ingo Molnar

membarrier: Provide GLOBAL_EXPEDITED command

2018-02-05T20:34:31+00:00

Allow expedited membarrier to be used for data shared between processes
through shared memory.

Processes wishing to receive the membarriers register with
MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED. Those which want to issue
membarrier invoke MEMBARRIER_CMD_GLOBAL_EXPEDITED.

This allows extremely simple kernel-level implementation: we have almost
everything we need with the PRIVATE_EXPEDITED barrier code. All we need
to do is to add a flag in the mm_struct that will be used to check
whether we need to send the IPI to the current thread of each CPU.

There is a slight downside to this approach compared to targeting
specific shared memory users: when performing a membarrier operation,
all registered "global" receivers will get the barrier, even if they
don't share a memory mapping with the sender issuing
MEMBARRIER_CMD_GLOBAL_EXPEDITED.

This registration approach seems to fit the requirement of not
disturbing processes that really deeply care about real-time: they
simply should not register with MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED.

In order to align the membarrier command names, the "MEMBARRIER_CMD_SHARED"
command is renamed to "MEMBARRIER_CMD_GLOBAL", keeping an alias of
MEMBARRIER_CMD_SHARED to MEMBARRIER_CMD_GLOBAL for UAPI header backward
compatibility.

Signed-off-by: Mathieu Desnoyers 
Acked-by: Thomas Gleixner 
Acked-by: Peter Zijlstra (Intel) 
Cc: Andrea Parri 
Cc: Andrew Hunter 
Cc: Andy Lutomirski 
Cc: Avi Kivity 
Cc: Benjamin Herrenschmidt 
Cc: Boqun Feng 
Cc: Dave Watson 
Cc: David Sehr 
Cc: Greg Hackmann 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Cc: Maged Michael 
Cc: Michael Ellerman 
Cc: Paul E. McKenney 
Cc: Paul Mackerras 
Cc: Russell King 
Cc: Will Deacon 
Cc: linux-api@vger.kernel.org
Link: http://lkml.kernel.org/r/20180129202020.8515-5-mathieu.desnoyers@efficios.com
Signed-off-by: Ingo Molnar

powerpc, membarrier: Skip memory barrier in switch_mm()

2018-02-05T20:34:02+00:00

Allow PowerPC to skip the full memory barrier in switch_mm(), and
only issue the barrier when scheduling into a task belonging to a
process that has registered to use expedited private.

Threads targeting the same VM but which belong to different thread
groups is a tricky case. It has a few consequences:

It turns out that we cannot rely on get_nr_threads(p) to count the
number of threads using a VM. We can use
(atomic_read(&mm->mm_users) == 1 && get_nr_threads(p) == 1)
instead to skip the synchronize_sched() for cases where the VM only has
a single user, and that user only has a single thread.

It also turns out that we cannot use for_each_thread() to set
thread flags in all threads using a VM, as it only iterates on the
thread group.

Therefore, test the membarrier state variable directly rather than
relying on thread flags. This means
membarrier_register_private_expedited() needs to set the
MEMBARRIER_STATE_PRIVATE_EXPEDITED flag, issue synchronize_sched(), and
only then set MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY which allows
private expedited membarrier commands to succeed.
membarrier_arch_switch_mm() now tests for the
MEMBARRIER_STATE_PRIVATE_EXPEDITED flag.

Signed-off-by: Mathieu Desnoyers 
Acked-by: Thomas Gleixner 
Acked-by: Peter Zijlstra (Intel) 
Cc: Alan Stern 
Cc: Alexander Viro 
Cc: Andrea Parri 
Cc: Andrew Hunter 
Cc: Andy Lutomirski 
Cc: Avi Kivity 
Cc: Benjamin Herrenschmidt 
Cc: Boqun Feng 
Cc: Dave Watson 
Cc: David Sehr 
Cc: Greg Hackmann 
Cc: Linus Torvalds 
Cc: Maged Michael 
Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Cc: Paul E. McKenney 
Cc: Paul Mackerras 
Cc: Russell King 
Cc: Will Deacon 
Cc: linux-api@vger.kernel.org
Cc: linux-arch@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lkml.kernel.org/r/20180129202020.8515-3-mathieu.desnoyers@efficios.com
Signed-off-by: Ingo Molnar

membarrier: Disable preemption when calling smp_call_function_many()

2018-01-10T07:50:31+00:00

smp_call_function_many() requires disabling preemption around the call.

Signed-off-by: Mathieu Desnoyers 
Cc:  # v4.14+
Cc: Andrea Parri 
Cc: Andrew Hunter 
Cc: Avi Kivity 
Cc: Benjamin Herrenschmidt 
Cc: Boqun Feng 
Cc: Dave Watson 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Cc: Maged Michael 
Cc: Michael Ellerman 
Cc: Paul E . McKenney 
Cc: Paul E. McKenney 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Link: http://lkml.kernel.org/r/20171215192310.25293-1-mathieu.desnoyers@efficios.com
Signed-off-by: Ingo Molnar

membarrier: Provide register expedited private command

2017-10-20T02:13:40+00:00

This introduces a "register private expedited" membarrier command which
allows eventual removal of important memory barrier constraints on the
scheduler fast-paths. It changes how the "private expedited" membarrier
command (new to 4.14) is used from user-space.

This new command allows processes to register their intent to use the
private expedited command.  This affects how the expedited private
command introduced in 4.14-rc is meant to be used, and should be merged
before 4.14 final.

Processes are now required to register before using
MEMBARRIER_CMD_PRIVATE_EXPEDITED, otherwise that command returns EPERM.

This fixes a problem that arose when designing requested extensions to
sys_membarrier() to allow JITs to efficiently flush old code from
instruction caches.  Several potential algorithms are much less painful
if the user register intent to use this functionality early on, for
example, before the process spawns the second thread.  Registering at
this time removes the need to interrupt each and every thread in that
process at the first expedited sys_membarrier() system call.

Signed-off-by: Mathieu Desnoyers 
Acked-by: Paul E. McKenney 
Cc: Peter Zijlstra 
Cc: Ingo Molnar 
Cc: Alexander Viro 
Signed-off-by: Linus Torvalds