linux.git/arch/arc/include/asm/atomic.h, branch v4.19

ARC: atomics: unbork atomic_fetch_##op()

2018-08-31T17:14:51+00:00

In 4.19-rc1, Eugeniy reported weird boot and IO errors on ARC HSDK

| INFO: task syslogd:77 blocked for more than 10 seconds.
|       Not tainted 4.19.0-rc1-00007-gf213acea4e88 #40
| "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
| message.
| syslogd         D    0    77     76 0x00000000
|
| Stack Trace:
|  __switch_to+0x0/0xac
|  __schedule+0x1b2/0x730
|  io_schedule+0x5c/0xc0
|  __lock_page+0x98/0xdc
|  find_lock_entry+0x38/0x100
|  shmem_getpage_gfp.isra.3+0x82/0xbfc
|  shmem_fault+0x46/0x138
|  handle_mm_fault+0x5bc/0x924
|  do_page_fault+0x100/0x2b8
|  ret_from_exception+0x0/0x8

He bisected to 84c6591103db ("locking/atomics,
asm-generic/bitops/lock.h: Rewrite using atomic_fetch_*()")

This commit however only unmasked the real issue introduced by commit
4aef66c8ae9 ("locking/atomic, arch/arc: Fix build") which missed the
retry-if-scond-failed branch in atomic_fetch_##op() macros.

The bisected commit started using atomic_fetch_##op() macros for building
the rest of atomics.

Fixes: 4aef66c8ae9 ("locking/atomic, arch/arc: Fix build")
Reported-by: Eugeniy Paltsev 
Acked-by: Peter Zijlstra (Intel) 
Signed-off-by: Will Deacon 
Signed-off-by: Vineet Gupta 
[vgupta: wrote changelog]

atomics/treewide: Clean up '*_andnot()' ifdeffery

2018-06-21T12:25:24+00:00

The ifdeffery for atomic*_{fetch_,}andnot() is unlike that for all the
other atomics. If atomic*_andnot() is not defined, the corresponding
atomic*_fetch_andnot() is assumed to not be defined.

Additionally, the fallbacks for the various ordering cases are written
much later in atomic.h as static inlines.

This isn't problematic today, but gets in the way of scripting the
generation of atomics. To prepare for scripting, this patch:

* Switches to separate ifdefs for atomic*_andnot() and
  atomic*_fetch_andnot(), updating implementations as appropriate.

* Moves the fallbacks into the standards ifdefs, as macro expansions
  rather than static inlines.

* Removes trivial andnot implementations from architectures, where these
  are superseded by core code.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: Will Deacon 
Acked-by: Peter Zijlstra (Intel) 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Link: https://lore.kernel.org/lkml/20180621121321.4761-19-mark.rutland@arm.com
Signed-off-by: Ingo Molnar

atomics/treewide: Make conditional inc/dec ops optional

2018-06-21T12:25:24+00:00

The conditional inc/dec ops differ for atomic_t and atomic64_t:

- atomic_inc_unless_positive() is optional for atomic_t, and doesn't exist for atomic64_t.
- atomic_dec_unless_negative() is optional for atomic_t, and doesn't exist for atomic64_t.
- atomic_dec_if_positive is optional for atomic_t, and is mandatory for atomic64_t.

Let's make these consistently optional for both. At the same time, let's
clean up the existing fallbacks to use atomic_try_cmpxchg().

The instrumented atomics are updated accordingly.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: Will Deacon 
Acked-by: Peter Zijlstra (Intel) 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Link: https://lore.kernel.org/lkml/20180621121321.4761-18-mark.rutland@arm.com
Signed-off-by: Ingo Molnar

atomics/treewide: Make unconditional inc/dec ops optional

2018-06-21T12:25:24+00:00

Many of the inc/dec ops are mandatory, but for most architectures inc/dec are
simply trivial wrappers around their corresponding add/sub ops.

Let's make all the inc/dec ops optional, so that we can get rid of these
boilerplate wrappers.

The instrumented atomics are updated accordingly.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: Will Deacon 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Palmer Dabbelt 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Link: https://lore.kernel.org/lkml/20180621121321.4761-17-mark.rutland@arm.com
Signed-off-by: Ingo Molnar

atomics/treewide: Make test ops optional

2018-06-21T12:25:24+00:00

Some of the atomics return the result of a test applied after the atomic
operation, and almost all architectures implement these as trivial
wrappers around the underlying atomic. Specifically:

 * _inc_and_test(v)    is (_inc_return(v)    == 0)
 * _dec_and_test(v)    is (_dec_return(v)    == 0)
 * _sub_and_test(i, v) is (_sub_return(i, v) == 0)
 * _add_negative(i, v) is (_add_return(i, v)  < 0)

Rather than have these definitions duplicated in all architectures, with
minor inconsistencies in formatting and documentation, let's make these
operations optional, with default fallbacks as above. Implementations
must now provide a preprocessor symbol.

The instrumented atomics are updated accordingly.

Both x86 and m68k have custom implementations, which are left as-is,
given preprocessor symbols to avoid being overridden.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: Will Deacon 
Acked-by: Geert Uytterhoeven 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Palmer Dabbelt 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Link: https://lore.kernel.org/lkml/20180621121321.4761-16-mark.rutland@arm.com
Signed-off-by: Ingo Molnar

atomics/arc: Define atomic64_fetch_add_unless()

2018-06-21T12:25:24+00:00

As a step towards unifying the atomic/atomic64/atomic_long APIs, this
patch converts the arch/arc implementation of atomic64_add_unless() into
an implementation of atomic64_fetch_add_unless().

A wrapper in  will build atomic_add_unless() atop of
this, provided it is given a preprocessor definition.

No functional change is intended as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: Will Deacon 
Acked-by: Peter Zijlstra (Intel) 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Cc: Vineet Gupta 
Link: https://lore.kernel.org/lkml/20180621121321.4761-11-mark.rutland@arm.com
Signed-off-by: Ingo Molnar

atomics/treewide: Make atomic_fetch_add_unless() optional

2018-06-21T12:22:33+00:00

Several architectures these have a near-identical implementation based
on atomic_read() and atomic_cmpxchg() which we can instead define in
, so let's do so, using something close to the existing
x86 implementation with try_cmpxchg().

Where an architecture provides its own atomic_fetch_add_unless(), it
must define a preprocessor symbol for it. The instrumented atomics are
updated accordingly.

Note that arch/arc's existing atomic_fetch_add_unless() had redundant
barriers, as these are already present in its atomic_cmpxchg()
implementation.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: Geert Uytterhoeven 
Reviewed-by: Will Deacon 
Acked-by: Geert Uytterhoeven 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Palmer Dabbelt 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Cc: Vineet Gupta 
Link: https://lore.kernel.org/lkml/20180621121321.4761-7-mark.rutland@arm.com
Signed-off-by: Ingo Molnar

atomics/treewide: Make atomic64_inc_not_zero() optional

2018-06-21T12:22:33+00:00

We define a trivial fallback for atomic_inc_not_zero(), but don't do
the same for atomic64_inc_not_zero(), leading most architectures to
define the same boilerplate.

Let's add a fallback in , and remove the redundant
implementations. Note that atomic64_add_unless() is always defined in
, and promotes its arguments to the requisite types, so
we need not do this explicitly.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: Will Deacon 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Palmer Dabbelt 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Link: https://lore.kernel.org/lkml/20180621121321.4761-6-mark.rutland@arm.com
Signed-off-by: Ingo Molnar

atomics/treewide: Remove redundant atomic_inc_not_zero() definitions

2018-06-21T12:22:33+00:00

When atomic_inc_not_zero(v) isn't defined,  will define
it as falling back to atomic_add_unless((v), 1, 0), so there's no need
for arch code to do so.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: Will Deacon 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Palmer Dabbelt 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Link: https://lore.kernel.org/lkml/20180621121321.4761-3-mark.rutland@arm.com
Signed-off-by: Ingo Molnar

atomics/treewide: Rename __atomic_add_unless() => atomic_fetch_add_unless()

2018-06-21T12:22:32+00:00

While __atomic_add_unless() was originally intended as a building-block
for atomic_add_unless(), it's now used in a number of places around the
kernel. It's the only common atomic operation named __atomic*(), rather
than atomic_*(), and for consistency it would be better named
atomic_fetch_add_unless().

This lack of consistency is slightly confusing, and gets in the way of
scripting atomics. Given that, let's clean things up and promote it to
an official part of the atomics API, in the form of
atomic_fetch_add_unless().

This patch converts definitions and invocations over to the new name,
including the instrumented version, using the following script:

  ----
  git grep -w __atomic_add_unless | while read line; do
  sed -i '{s/\<__atomic_add_unless\>/atomic_fetch_add_unless/}' "${line%%:*}";
  done
  git grep -w __arch_atomic_add_unless | while read line; do
  sed -i '{s/\<__arch_atomic_add_unless\>/arch_atomic_fetch_add_unless/}' "${line%%:*}";
  done
  ----

Note that we do not have atomic{64,_long}_fetch_add_unless(), which will
be introduced by later patches.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: Will Deacon 
Acked-by: Geert Uytterhoeven 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Palmer Dabbelt 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Link: https://lore.kernel.org/lkml/20180621121321.4761-2-mark.rutland@arm.com
Signed-off-by: Ingo Molnar