linux-stable.git/arch/mips/kernel/smp.c, branch linux-4.9.y

MIPS: smp: fill in sibling and core maps earlier

2022-03-23T08:00:33+00:00

[ Upstream commit f2703def339c793674010cc9f01bfe4980231808 ]

After enabling CONFIG_SCHED_CORE (landed during 5.14 cycle),
2-core 2-thread-per-core interAptiv (CPS-driven) started emitting
the following:

[    0.025698] CPU1 revision is: 0001a120 (MIPS interAptiv (multi))
[    0.048183] ------------[ cut here ]------------
[    0.048187] WARNING: CPU: 1 PID: 0 at kernel/sched/core.c:6025 sched_core_cpu_starting+0x198/0x240
[    0.048220] Modules linked in:
[    0.048233] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.17.0-rc3+ #35 b7b319f24073fd9a3c2aa7ad15fb7993eec0b26f
[    0.048247] Stack : 817f0000 00000004 327804c8 810eb050 00000000 00000004 00000000 c314fdd1
[    0.048278]         830cbd64 819c0000 81800000 817f0000 83070bf4 00000001 830cbd08 00000000
[    0.048307]         00000000 00000000 815fcbc4 00000000 00000000 00000000 00000000 00000000
[    0.048334]         00000000 00000000 00000000 00000000 817f0000 00000000 00000000 817f6f34
[    0.048361]         817f0000 818a3c00 817f0000 00000004 00000000 00000000 4dc33260 0018c933
[    0.048389]         ...
[    0.048396] Call Trace:
[    0.048399] [<8105a7bc>] show_stack+0x3c/0x140
[    0.048424] [<8131c2a0>] dump_stack_lvl+0x60/0x80
[    0.048440] [<8108b5c0>] __warn+0xc0/0xf4
[    0.048454] [<8108b658>] warn_slowpath_fmt+0x64/0x10c
[    0.048467] [<810bd418>] sched_core_cpu_starting+0x198/0x240
[    0.048483] [<810c6514>] sched_cpu_starting+0x14/0x80
[    0.048497] [<8108c0f8>] cpuhp_invoke_callback_range+0x78/0x140
[    0.048510] [<8108d914>] notify_cpu_starting+0x94/0x140
[    0.048523] [<8106593c>] start_secondary+0xbc/0x280
[    0.048539]
[    0.048543] ---[ end trace 0000000000000000 ]---
[    0.048636] Synchronize counters for CPU 1: done.

...for each but CPU 0/boot.
Basic debug printks right before the mentioned line say:

[    0.048170] CPU: 1, smt_mask:

So smt_mask, which is sibling mask obviously, is empty when entering
the function.
This is critical, as sched_core_cpu_starting() calculates
core-scheduling parameters only once per CPU start, and it's crucial
to have all the parameters filled in at that moment (at least it
uses cpu_smt_mask() which in fact is `&cpu_sibling_map[cpu]` on
MIPS).

A bit of debugging led me to that set_cpu_sibling_map() performing
the actual map calculation, was being invocated after
notify_cpu_start(), and exactly the latter function starts CPU HP
callback round (sched_core_cpu_starting() is basically a CPU HP
callback).
While the flow is same on ARM64 (maps after the notifier, although
before calling set_cpu_online()), x86 started calculating sibling
maps earlier than starting the CPU HP callbacks in Linux 4.14 (see
[0] for the reference). Neither me nor my brief tests couldn't find
any potential caveats in calculating the maps right after performing
delay calibration, but the WARN splat is now gone.
The very same debug prints now yield exactly what I expected from
them:

[    0.048433] CPU: 1, smt_mask: 0-1

[0] https://git.kernel.org/pub/scm/linux/kernel/git/mips/linux.git/commit/?id=76ce7cfe35ef

Signed-off-by: Alexander Lobakin 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Thomas Bogendoerfer 
Signed-off-by: Sasha Levin

MIPS: SMP: Fix deadlock & online race

2017-11-15T14:53:19+00:00

commit 9e8c399a88f0b87e41a894911475ed2a8f8dff9e upstream.

Commit 6f542ebeaee0 ("MIPS: Fix race on setting and getting
cpu_online_mask") effectively reverted commit 8f46cca1e6c06 ("MIPS: SMP:
Fix possibility of deadlock when bringing CPUs online") and thus has
reinstated the possibility of deadlock.

The commit was based on testing of kernel v4.4, where the CPU hotplug
core code issued a BUG() if the starting CPU is not marked online when
the boot CPU returns from __cpu_up. The commit fixes this race (in
v4.4), but re-introduces the deadlock situation.

As noted in the commit message, upstream differs in this area. Commit
8df3e07e7f21f ("cpu/hotplug: Let upcoming cpu bring itself fully up")
adds a completion event in the CPU hotplug core code, making this race
impossible. However, people were unhappy with relying on the core code
to do the right thing.

To address the issues both commits were trying to fix, add a second
completion event in the MIPS smp hotplug path. It removes the
possibility of a race, since the MIPS smp hotplug code now synchronises
both the boot and secondary CPUs before they return to the hotplug core
code. It also addresses the deadlock by ensuring that the secondary CPU
is not marked online before it's counters are synchronised.

This fix should also be backported to fix the race condition introduced
by the backport of commit 8f46cca1e6c06 ("MIPS: SMP: Fix possibility of
deadlock when bringing CPUs online"), through really that race only
existed before commit 8df3e07e7f21f ("cpu/hotplug: Let upcoming cpu
bring itself fully up").

Signed-off-by: Matt Redfearn 
Fixes: 6f542ebeaee0 ("MIPS: Fix race on setting and getting cpu_online_mask")
CC: Matija Glavinic Pecotic 
Patchwork: https://patchwork.linux-mips.org/patch/17376/
Signed-off-by: James Hogan 
[jhogan@kernel.org: Backported 4.1..4.9]
Signed-off-by: Greg Kroah-Hartman

MIPS: Fix race on setting and getting cpu_online_mask

2017-11-15T14:53:19+00:00

commit 6f542ebeaee0ee552a902ce3892220fc22c7ec8e upstream.

While testing cpu hoptlug (cpu down and up in loops) on kernel 4.4, it was
observed that occasionally check for cpu online will fail in kernel/cpu.c,
_cpu_up:

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree/kernel/cpu.c?h=v4.4.79#n485
 518        /* Arch-specific enabling code. */
 519        ret = __cpu_up(cpu, idle);
 520
 521        if (ret != 0)
 522                goto out_notify;
 523        BUG_ON(!cpu_online(cpu));

Reason is race between start_secondary and _cpu_up. cpu_callin_map is set
before cpu_online_mask. In __cpu_up, cpu_callin_map is waited for, but cpu
online mask is not, resulting in race in which secondary processor started
and set cpu_callin_map, but not yet set the online mask,resulting in above
BUG being hit.

Upstream differs in the area. cpu_online check is in bringup_wait_for_ap,
which is after cpu reached AP_ONLINE_IDLE,where secondary passed its start
function. Nonetheless, fix makes start_secondary safe and not depending on
other locks throughout the code. It protects as well against cpu_online
checks put in between sometimes in the future.

Fix this by moving completion after all flags are set.

Signed-off-by: Matija Glavinic Pecotic 
Cc: Alexander Sverdlin 
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/16925/
Signed-off-by: Ralf Baechle 
Signed-off-by: Greg Kroah-Hartman

MIPS: SMP: Use a completion event to signal CPU up

2017-11-15T14:53:19+00:00

commit a00eeede507c975087b7b8df8cf2c9f88ba285de upstream.

If a secondary CPU failed to start, for any reason, the CPU requesting
the secondary to start would get stuck in the loop waiting for the
secondary to be present in the cpu_callin_map.

Rather than that, use a completion event to signal that the secondary
CPU has started and is waiting to synchronise counters.

Since the CPU presence will no longer be marked in cpu_callin_map,
remove the redundant test from arch_cpu_idle_dead().

Signed-off-by: Matt Redfearn 
Cc: Maciej W. Rozycki 
Cc: Jiri Slaby 
Cc: Paul Gortmaker 
Cc: Chris Metcalf 
Cc: Thomas Gleixner 
Cc: Qais Yousef 
Cc: James Hogan 
Cc: Paul Burton 
Cc: Marcin Nowakowski 
Cc: Andrew Morton 
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/14502/
Signed-off-by: Ralf Baechle 
Signed-off-by: Greg Kroah-Hartman

MIPS: kernel: Audit and remove any unnecessary uses of module.h

2016-10-04T23:31:20+00:00

Historically a lot of these existed because we did not have
a distinction between what was modular code and what was providing
support to modules via EXPORT_SYMBOL and friends.  That changed
when we forked out support for the latter into the export.h file.

This means we should be able to reduce the usage of module.h
in code that is obj-y Makefile or bool Kconfig.  The advantage
in doing so is that module.h itself sources about 15 other headers;
adding significantly to what we feed cpp, and it can obscure what
headers we are effectively using.

Since module.h was the source for init.h (for __init) and for
export.h (for EXPORT_SYMBOL) we consider each obj-y/bool instance
for the presence of either and replace as needed.

In the case of the n32/o32 files, we have to get rid of a couple
no-op MODULE_ tags to facilitate the module.h removal.  They piggy
back off the fs/ elf binary support, which is also a bool Kconfig.

Signed-off-by: Paul Gortmaker 
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/14032/
Signed-off-by: Ralf Baechle

MIPS: smp.c: Introduce mechanism for freeing and allocating IPIs

2016-10-04T23:31:20+00:00

For the MIPS remote processor implementation, we need additional IPIs to
talk to the remote processor. Since MIPS GIC reserves exactly the right
number of IPI IRQs required by Linux for the number of VPs in the
system, this is not possible without releasing some recources.

This commit introduces mips_smp_ipi_allocate() which allocates IPIs to a
given cpumask. It is called as normal with the cpu_possible_mask at
bootup to initialise IPIs to all CPUs. mips_smp_ipi_free() may then be
used to free IPIs to a subset of those CPUs so that their hardware
resources can be reused.

Signed-off-by: Matt Redfearn 
Cc: Bjorn Andersson 
Cc: Ohad Ben-Cohen 
Cc: Thomas Gleixner 
Cc: Lisa Parratt 
Cc: James Hogan 
Cc: Qais Yousef 
Cc: Paul Burton 
Cc: linux-mips@linux-mips.org
Cc: linux-remoteproc@vger.kernel.org
Cc: lisa.parratt@imgtec.com
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/14285/
Signed-off-by: Ralf Baechle

MIPS: SMP: Wrap call to mips_cpc_lock_other in mips_cm_lock_other

2016-10-04T14:13:57+00:00

All calls to mips_cpc_lock_other should be wrapped in
mips_cm_lock_other. This only matters if the system has CM3 and is using
cpu idle, since otherwise a) the CPC lock is sufficent for CM < 3 and b)
any systems with CM > 3 have not been able to use cpu idle until now.

Signed-off-by: Matt Redfearn 
Reviewed-by: Paul Burton 
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Cc: Thomas Gleixner 
Cc: James Hogan 
Cc: Qais Yousef 
Patchwork: https://patchwork.linux-mips.org/patch/14227/
Signed-off-by: Ralf Baechle

MIPS: SMP: Fix possibility of deadlock when bringing CPUs online

2016-09-24T23:43:52+00:00

This patch fixes the possibility of a deadlock when bringing up
secondary CPUs.
The deadlock occurs because the set_cpu_online() is called before
synchronise_count_slave(). This can cause a deadlock if the boot CPU,
having scheduled another thread, attempts to send an IPI to the
secondary CPU, which it sees has been marked online. The secondary is
blocked in synchronise_count_slave() waiting for the boot CPU to enter
synchronise_count_master(), but the boot cpu is blocked in
smp_call_function_many() waiting for the secondary to respond to it's
IPI request.

Fix this by marking the CPU online in cpu_callin_map and synchronising
counters before declaring the CPU online and calculating the maps for
IPIs.

Signed-off-by: Matt Redfearn 
Reported-by: Justin Chen 
Tested-by: Justin Chen 
Cc: Florian Fainelli 
Cc: stable@vger.kernel.org # v4.1+
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/14302/
Signed-off-by: Ralf Baechle

MIPS: c-r4k: Exclude sibling CPUs in SMP calls

2016-07-29T08:19:29+00:00

When performing SMP calls to foreign cores, exclude sibling CPUs from
the provided map, as we already handle the local core on the current
CPU. This prevents an SMP call from for example core 0, VPE 1 to VPE 0
on the same core.

In the process the cpu_foreign_map cpumask is turned into an array of
cpumasks, so that each CPU has its own version of it which excludes
sibling CPUs. r4k_op_needs_ipi() is also updated to reflect that cache
management SMP calls are not needed when all CPUs are siblings (i.e.
there are no foreign CPUs according to the new cpu_foreign_map[]
semantics which exclude siblings).

Signed-off-by: James Hogan 
Cc: Paul Burton 
Cc: Leonid Yegoshin 
Cc: Felix Fietkau 
Cc: Jayachandran C. 
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13801/
Signed-off-by: Ralf Baechle

MIPS: SMP: Drop stop_this_cpu() cpu_foreign_map hack

2016-07-29T08:19:28+00:00

Commit cccf34e9411c ("MIPS: c-r4k: Fix cache flushing for MT cores")
added the cpu_foreign_map cpumask containing a single VPE from each
online core, and recalculated it when secondary CPUs are brought up.

stop_this_cpu() was also updated to recalculate cpu_foreign_map, but
with an additional hack before marking the CPU as offline to copy
cpu_online_mask into cpu_foreign_map and perform an SMP memory barrier.

This appears to have been intended to prevent cache management IPIs
being missed when the VPE representing the core in cpu_foreign_map is
taken offline while other VPEs remain online. Unfortunately there is
nothing in this hack to prevent r4k_on_each_cpu() from reading the old
cpu_foreign_map, and smp_call_function_many() from reading that new
cpu_online_mask with the core's representative VPE marked offline. It
then wouldn't send an IPI to any online VPEs of that core.

stop_this_cpu() is only actually called in panic and system shutdown /
halt / reboot situations, in which case all CPUs are going down and we
don't really need to care about cache management, so drop this hack.

Note that the __cpu_disable() case for CPU hotplug is handled in the
previous commit, and no synchronisation is needed there due to the use
of stop_machine() which prevents hotplug from taking place while any CPU
has disabled preemption (as r4k_on_each_cpu() does).

Signed-off-by: James Hogan 
Cc: Paul Burton 
Cc: Leonid Yegoshin 
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13796/
Signed-off-by: Ralf Baechle