linux.git/mm/percpu.c, branch v4.13-rc2

percpu: resolve err may not be initialized in pcpu_alloc

2017-06-21T16:00:45+00:00

From 4a42ecc735cff0015cc73c3d87edede631f4b885 Mon Sep 17 00:00:00 2001
From: Dennis Zhou 
Date: Wed, 21 Jun 2017 08:07:15 -0700

Add error message to out of space failure for atomic allocations in
percpu allocation path to fix -Wmaybe-uninitialized.

Signed-off-by: Dennis Zhou 
Reported-by: Stephen Rothwell 
Signed-off-by: Tejun Heo

percpu: add tracepoint support for percpu memory

2017-06-20T19:31:43+00:00

Add support for tracepoints to the following events: chunk allocation,
chunk free, area allocation, area free, and area allocation failure.
This should let us replay percpu memory requests and evaluate
corresponding decisions.

Signed-off-by: Dennis Zhou 
Signed-off-by: Tejun Heo

percpu: expose statistics about percpu memory via debugfs

2017-06-20T19:31:38+00:00

There is limited visibility into the use of percpu memory leaving us
unable to reason about correctness of parameters and overall use of
percpu memory. These counters and statistics aim to help understand
basic statistics about percpu memory such as number of allocations over
the lifetime, allocation sizes, and fragmentation.

New Config: PERCPU_STATS

Signed-off-by: Dennis Zhou 
Signed-off-by: Tejun Heo

percpu: migrate percpu data structures to internal header

2017-06-20T19:31:28+00:00

Migrates pcpu_chunk definition and a few percpu static variables to an
internal header file from mm/percpu.c. These will be used with debugfs
to expose statistics about percpu memory improving visibility regarding
allocations and fragmentation.

Signed-off-by: Dennis Zhou 
Signed-off-by: Tejun Heo

percpu: add missing lockdep_assert_held to func pcpu_free_area

2017-06-20T17:43:38+00:00

Add a missing lockdep_assert_held for pcpu_lock to improve consistency
and safety throughout mm/percpu.c.

Signed-off-by: Dennis Zhou 
Signed-off-by: Tejun Heo

mark most percpu globals as __ro_after_init

2017-05-10T19:21:49+00:00

Moving pcpu_base_addr to this section comes from PaX where it's part of
KERNEXEC. This extends it to the rest of the globals only written by the
init code.

Signed-off-by: Daniel Micay 
Acked-by: Kees Cook 
Signed-off-by: Tejun Heo

Merge branch 'sched/core' into locking/core

2017-04-04T09:31:12+00:00

Required for the rtmutex/sched_deadline patches which depend on both
branches

lockdep: Fix per-cpu static objects

2017-03-26T13:09:45+00:00

Since commit 383776fa7527 ("locking/lockdep: Handle statically initialized
PER_CPU locks properly") we try to collapse per-cpu locks into a single
class by giving them all the same key. For this key we choose the canonical
address of the per-cpu object, which would be the offset into the per-cpu
area.

This has two problems:

 - there is a case where we run !0 lock->key through static_obj() and
   expect this to pass; it doesn't for canonical pointers.

 - 0 is a valid canonical address.

Cure both issues by redefining the canonical address as the address of the
per-cpu variable on the boot CPU.

Since I didn't want to rely on CPU0 being the boot-cpu, or even existing at
all, track the boot CPU in a variable.

Fixes: 383776fa7527 ("locking/lockdep: Handle statically initialized PER_CPU locks properly")
Reported-by: kernel test robot 
Signed-off-by: Peter Zijlstra (Intel) 
Tested-by: Borislav Petkov 
Cc: Sebastian Andrzej Siewior 
Cc: linux-mm@kvack.org
Cc: wfg@linux.intel.com
Cc: kernel test robot 
Cc: LKP 
Link: http://lkml.kernel.org/r/20170320114108.kbvcsuepem45j5cr@hirez.programming.kicks-ass.net
Signed-off-by: Thomas Gleixner

locking/lockdep: Handle statically initialized PER_CPU locks properly

2017-03-16T08:57:08+00:00

If a PER_CPU struct which contains a spin_lock is statically initialized
via:

DEFINE_PER_CPU(struct foo, bla) = {
	.lock = __SPIN_LOCK_UNLOCKED(bla.lock)
};

then lockdep assigns a seperate key to each lock because the logic for
assigning a key to statically initialized locks is to use the address as
the key. With per CPU locks the address is obvioulsy different on each CPU.

That's wrong, because all locks should have the same key.

To solve this the following modifications are required:

 1) Extend the is_kernel/module_percpu_addr() functions to hand back the
    canonical address of the per CPU address, i.e. the per CPU address
    minus the per CPU offset.

 2) Check the lock address with these functions and if the per CPU check
    matches use the returned canonical address as the lock key, so all per
    CPU locks have the same key.

 3) Move the static_obj(key) check into look_up_lock_class() so this check
    can be avoided for statically initialized per CPU locks.  That's
    required because the canonical address fails the static_obj(key) check
    for obvious reasons.

Reported-by: Mike Galbraith 
Signed-off-by: Thomas Gleixner 
[ Merged Dan's fixups for !MODULES and !SMP into this patch. ]
Signed-off-by: Sebastian Andrzej Siewior 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Andrew Morton 
Cc: Dan Murphy 
Cc: Linus Torvalds 
Cc: Paul E. McKenney 
Cc: Peter Zijlstra 
Link: http://lkml.kernel.org/r/20170227143736.pectaimkjkan5kow@linutronix.de
Signed-off-by: Ingo Molnar

percpu: acquire pcpu_lock when updating pcpu_nr_empty_pop_pages

2017-03-06T20:55:39+00:00

Update to pcpu_nr_empty_pop_pages in pcpu_alloc() is currently done
without holding pcpu_lock. This can lead to bad updates to the variable.
Add missing lock calls.

Fixes: b539b87fed37 ("percpu: implmeent pcpu_nr_empty_pop_pages and chunk->nr_populated")
Signed-off-by: Tahsin Erdogan 
Signed-off-by: Tejun Heo 
Cc: stable@vger.kernel.org # v3.18+