To make use of per_cpu variables in interrupt code per_cpu_init() must
be done after setup_per_cpu_areas(). This is achieved by calling it
in smp_prepare_boot_cpu() via a new smp_ops method.

Signed-off-by: Thomas Bogendoerfer <tbogendoerfer@suse.de>
Signed-off-by: Paul Burton <paul.burton@mips.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: James Hogan <jhogan@kernel.org>
Cc: linux-mips@vger.kernel.org
Cc: linux-kernel@vger.kernel.org

Introduce support for using MemoryMapIDs (MMIDs) as an alternative to
Address Space IDs (ASIDs). The major difference between the two is that
MMIDs are global - ie. an MMID uniquely identifies an address space
across all coherent CPUs. In contrast ASIDs are non-global per-CPU IDs,
wherein each address space is allocated a separate ASID for each CPU
upon which it is used. This global namespace allows a new GINVT
instruction be used to globally invalidate TLB entries associated with a
particular MMID across all coherent CPUs in the system, removing the
need for IPIs to invalidate entries with separate ASIDs on each CPU.

The allocation scheme used here is largely borrowed from arm64 (see
arch/arm64/mm/context.c). In essence we maintain a bitmap to track
available MMIDs, and MMIDs in active use at the time of a rollover to a
new MMID version are preserved in the new version. The allocation scheme
requires efficient 64 bit atomics in order to perform reasonably, so
this support depends upon CONFIG_GENERIC_ATOMIC64=n (ie. currently it
will only be included in MIPS64 kernels).

The first, and currently only, available CPU with support for MMIDs is
the MIPS I6500. This CPU supports 16 bit MMIDs, and so for now we cap
our MMIDs to 16 bits wide in order to prevent the bitmap growing to
absurd sizes if any future CPU does implement 32 bit MMIDs as the
architecture manuals suggest is recommended.

When MMIDs are in use we also make use of GINVT instruction which is
available due to the global nature of MMIDs. By executing a sequence of
GINVT & SYNC 0x14 instructions we can avoid the overhead of an IPI to
each remote CPU in many cases. One complication is that GINVT will
invalidate wired entries (in all cases apart from type 0, which targets
the entire TLB). In order to avoid GINVT invalidating any wired TLB
entries we set up, we make sure to create those entries using a reserved
MMID (0) that we never associate with any address space.

Also of note is that KVM will require further work in order to support
MMIDs & GINVT, since KVM is involved in allocating IDs for guests & in
configuring the MMU. That work is not part of this patch, so for now
when MMIDs are in use KVM is disabled.

Signed-off-by: Paul Burton <paul.burton@mips.com>
Cc: linux-mips@vger.kernel.org

When we gain MMID support we'll be storing MMIDs as atomic64_t values
and accessing them via atomic64_* functions. This necessitates that we
don't use cpu_context() as the left hand side of an assignment, ie. as a
modifiable lvalue. In preparation for this introduce a new
set_cpu_context() function & replace all assignments with cpu_context()
on their left hand side with an equivalent call to set_cpu_context().

To enforce that cpu_context() should not be used for assignments, we
rewrite it as a static inline function.

Signed-off-by: Paul Burton <paul.burton@mips.com>
Cc: linux-mips@vger.kernel.org

All 3 variants of local_flush_tlb_mm() are now effectively simple calls
to drop_mmu_context(). Remove them and use drop_mmu_context() directly.

Signed-off-by: Paul Burton <paul.burton@mips.com>
Cc: linux-mips@vger.kernel.org

Pull MIPS updates from James Hogan:
 "These are the main MIPS changes for 4.15.

  Fixes:
   - ralink: Fix MT7620 PCI build issues (4.5)
   - Disable cmpxchg64() and HAVE_VIRT_CPU_ACCOUNTING_GEN for 32-bit SMP
     (4.1)
   - Fix MIPS64 FP save/restore on 32-bit kernels (4.0)
   - ptrace: Pick up ptrace/seccomp changed syscall numbers (3.19)
   - ralink: Fix MT7628 pinmux (3.19)
   - BCM47XX: Fix LED inversion on WRT54GSv1 (3.17)
   - Fix n32 core dumping as o32 since regset support (3.13)
   - ralink: Drop obsolete USB_ARCH_HAS_HCD select

  Build system:
   - Default to "generic" (multiplatform) system type instead of IP22
   - Use generic little endian MIPS32 r2 configuration as default
     defconfig instead of ip22_defconfig

  FPU emulation:
   - Fix exception generation for certain R6 FPU instructions

  SMP:
   - Allow __cpu_number_map to be larger than NR_CPUS for sparse CPU id
     spaces

  Miscellaneous:
   - Add iomem resource for kernel bss section for kexec/kdump
   - Atomics: Nudge writes on bit unlock
   - DT files: Standardise "ok" -> "okay"

  Minor cleanups:
   - Define virt_to_pfn()
   - Make thread_saved_pc static
   - Simplify 32-bit sign extension in __read_64bit_c0_split()
   - DMA: Use vma_pages() helper
   - FPU emulation: Replace unsigned with unsigned int
   - MM: Removed unused lastpfn
   - Alchemy: Make clk_ops const
   - Lasat: Use setup_timer() helper
   - ralink: Use BIT() in MT7620 PCI driver

  Platform support:

  BMIPS:
  - Enable HARDIRQS_SW_RESEND

  Broadcom BCM63XX:
  - Add clkdev lookup support
  - Update clk driver, UART driver, DTs to handle named refclk from DTs
  - Split apart various clocks to more closely match hardware
  - Add ethernet clocks

  Cavium Octeon:
  - Remove usage of cvmx_wait() in favour of __delay()

  ImgTec Pistachio:
  - DT: Drop deprecated dwmmc num-slots property

  Ingenic JZ4780:
  - Add NFS root to Ci20 defconfig
  - Add watchdog to Ci20 DT & defconfig, and allow building of watchdog
    driver with this SoC

  Generic (multiplatform):
  - Migrate xilfpga (MIPSfpga) platform to the generic platform

  Lantiq xway:
  - Fix ASC0/ASC1 clocks"

* tag 'mips_4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/jhogan/mips: (46 commits)
  MIPS: Add iomem resource for kernel bss section.
  MIPS: cmpxchg64() and HAVE_VIRT_CPU_ACCOUNTING_GEN don't work for 32-bit SMP
  MIPS: BMIPS: Enable HARDIRQS_SW_RESEND
  MIPS: pci: Make use of the BIT() macro inside the mt7620 driver
  MIPS: pci: Remove KERN_WARN instance inside the mt7620 driver
  MIPS: pci: Remove duplicate define in mt7620 driver
  MIPS: ralink: Fix typo in mt7628 pinmux function
  MIPS: ralink: Fix MT7628 pinmux
  MIPS: Fix odd fp register warnings with MIPS64r2
  watchdog: jz4780: Allow selection of jz4740-wdt driver
  MIPS/ptrace: Update syscall nr on register changes
  MIPS/ptrace: Pick up ptrace/seccomp changed syscalls
  MIPS: Fix an n32 core file generation regset support regression
  MIPS: Fix MIPS64 FP save/restore on 32-bit kernels
  MIPS: page.h: Define virt_to_pfn()
  MIPS: Xilfpga: Switch to using generic defconfigs
  MIPS: generic: Add support for MIPSfpga
  MIPS: Set defconfig target to a generic system for 32r2el
  MIPS: Kconfig: Set default MIPS system type as generic
  MIPS: DTS: Remove num-slots from Pistachio SoC
  ...

In systems where the CPU id space is sparse, this allows a smaller
NR_CPUS to be chosen, thus keeping internal data structures smaller.

Signed-off-by: David Daney <david.daney@cavium.com>
Signed-off-by: Carlos Munoz <cmunoz@caviumnetworks.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17388/
[jhogan@kernel.org: Add depends on SMP to fix
 "warning: symbol value '' invalid for MIPS_NR_CPU_NR_MAP"]
Signed-off-by: James Hogan <jhogan@kernel.org>

Commit 6f542ebeaee0 ("MIPS: Fix race on setting and getting
cpu_online_mask") effectively reverted commit 8f46cca1e6c06 ("MIPS: SMP:
Fix possibility of deadlock when bringing CPUs online") and thus has
reinstated the possibility of deadlock.

The commit was based on testing of kernel v4.4, where the CPU hotplug
core code issued a BUG() if the starting CPU is not marked online when
the boot CPU returns from __cpu_up. The commit fixes this race (in
v4.4), but re-introduces the deadlock situation.

As noted in the commit message, upstream differs in this area. Commit
8df3e07e7f21f ("cpu/hotplug: Let upcoming cpu bring itself fully up")
adds a completion event in the CPU hotplug core code, making this race
impossible. However, people were unhappy with relying on the core code
to do the right thing.

To address the issues both commits were trying to fix, add a second
completion event in the MIPS smp hotplug path. It removes the
possibility of a race, since the MIPS smp hotplug code now synchronises
both the boot and secondary CPUs before they return to the hotplug core
code. It also addresses the deadlock by ensuring that the secondary CPU
is not marked online before it's counters are synchronised.

This fix should also be backported to fix the race condition introduced
by the backport of commit 8f46cca1e6c06 ("MIPS: SMP: Fix possibility of
deadlock when bringing CPUs online"), through really that race only
existed before commit 8df3e07e7f21f ("cpu/hotplug: Let upcoming cpu
bring itself fully up").

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Fixes: 6f542ebeaee0 ("MIPS: Fix race on setting and getting cpu_online_mask")
CC: Matija Glavinic Pecotic <matija.glavinic-pecotic.ext@nokia.com>
Cc: <stable@vger.kernel.org> # v4.1+: 8f46cca1e6c0: "MIPS: SMP: Fix possibility of deadlock when bringing CPUs online"
Cc: <stable@vger.kernel.org> # v4.1+: a00eeede507c: "MIPS: SMP: Use a completion event to signal CPU up"
Cc: <stable@vger.kernel.org> # v4.1+: 6f542ebeaee0: "MIPS: Fix race on setting and getting cpu_online_mask"
Cc: <stable@vger.kernel.org> # v4.1+
Patchwork: https://patchwork.linux-mips.org/patch/17376/
Signed-off-by: James Hogan <jhogan@kernel.org>

Commit e83f7e02af50c ("MIPS: CPS: Have asm/mips-cps.h include CM & CPC
headers") adds a #error to arch/mips/include/asm/mips-cpc.h if it is
included directly. While this commit replaced almost all direct includes
of mips-cm.h and mips-cpc.h, 2 remain.

With some defconfigs, mips-cps.h is indirectly included before
mips-cpc.h, but in others this results in compilation errors:

In file included from arch/mips/generic/init.c:23:0:
./arch/mips/include/asm/mips-cpc.h:12:3: error: #error Please include
asm/mips-cps.h rather than asm/mips-cpc.h
 # error Please include asm/mips-cps.h rather than asm/mips-cpc.h

In file included from arch/mips/kernel/smp.c:23:0:
./arch/mips/include/asm/mips-cpc.h:12:3: error: #error Please include
asm/mips-cps.h rather than asm/mips-cpc.h
 # error Please include asm/mips-cps.h rather than asm/mips-cpc.h

In both cases, fix this by including mips-cps.h instead.

Fixes: e83f7e02af50c ("MIPS: CPS: Have asm/mips-cps.h include CM & CPC headers")
Signed-off-by: Matt Redfearn <matt.redfearn@mips.com>
Patchwork: https://patchwork.linux-mips.org/patch/17492/
Signed-off-by: James Hogan <jhogan@kernel.org>

Pull MIPS updates from Ralf Baechle:
 "This is the main pull request for 4.14 for MIPS; below a summary of
  the non-merge commits:

  CM:
   - Rename mips_cm_base to mips_gcr_base
   - Specify register size when generating accessors
   - Use BIT/GENMASK for register fields, order & drop shifts
   - Add cluster & block args to mips_cm_lock_other()

  CPC:
   - Use common CPS accessor generation macros
   - Use BIT/GENMASK for register fields, order & drop shifts
   - Introduce register modify (set/clear/change) accessors
   - Use change_*, set_* & clear_* where appropriate
   - Add CM/CPC 3.5 register definitions
   - Use GlobalNumber macros rather than magic numbers
   - Have asm/mips-cps.h include CM & CPC headers
   - Cluster support for topology functions
   - Detect CPUs in secondary clusters

  CPS:
   - Read GIC_VL_IDENT directly, not via irqchip driver

  DMA:
   - Consolidate coherent and non-coherent dma_alloc code
   - Don't use dma_cache_sync to implement fd_cacheflush

  FPU emulation / FP assist code:
   - Another series of 14 commits fixing corner cases such as NaN
     propgagation and other special input values.
   - Zero bits 32-63 of the result for a CLASS.D instruction.
   - Enhanced statics via debugfs
   - Do not use bools for arithmetic. GCC 7.1 moans about this.
   - Correct user fault_addr type

  Generic MIPS:
   - Enhancement of stack backtraces
   - Cleanup from non-existing options
   - Handle non word sized instructions when examining frame
   - Fix detection and decoding of ADDIUSP instruction
   - Fix decoding of SWSP16 instruction
   - Refactor handling of stack pointer in get_frame_info
   - Remove unreachable code from force_fcr31_sig()
   - Convert to using %pOF instead of full_name
   - Remove the R6000 support.
   - Move FP code from *_switch.S to *_fpu.S
   - Remove unused ST_OFF from r2300_switch.S
   - Allow platform to specify multiple its.S files
   - Add #includes to various files to ensure code builds reliable and
     without warning..
   - Remove __invalidate_kernel_vmap_range
   - Remove plat_timer_setup
   - Declare various variables & functions static
   - Abstract CPU core & VP(E) ID access through accessor functions
   - Store core & VP IDs in GlobalNumber-style variable
   - Unify checks for sibling CPUs
   - Add CPU cluster number accessors
   - Prevent direct use of generic_defconfig
   - Make CONFIG_MIPS_MT_SMP default y
   - Add __ioread64_copy
   - Remove unnecessary inclusions of linux/irqchip/mips-gic.h

  GIC:
   - Introduce asm/mips-gic.h with accessor functions
   - Use new GIC accessor functions in mips-gic-timer
   - Remove counter access functions from irq-mips-gic.c
   - Remove gic_read_local_vp_id() from irq-mips-gic.c
   - Simplify shared interrupt pending/mask reads in irq-mips-gic.c
   - Simplify gic_local_irq_domain_map() in irq-mips-gic.c
   - Drop gic_(re)set_mask() functions in irq-mips-gic.c
   - Remove gic_set_polarity(), gic_set_trigger(), gic_set_dual_edge(),
     gic_map_to_pin() and gic_map_to_vpe() from irq-mips-gic.c.
   - Convert remaining shared reg access, local int mask access and
     remaining local reg access to new accessors
   - Move GIC_LOCAL_INT_* to asm/mips-gic.h
   - Remove GIC_CPU_INT* macros from irq-mips-gic.c
   - Move various definitions to the driver
   - Remove gic_get_usm_range()
   - Remove __gic_irq_dispatch() forward declaration
   - Remove gic_init()
   - Use mips_gic_present() in place of gic_present and remove
     gic_present
   - Move gic_get_c0_*_int() to asm/mips-gic.h
   - Remove linux/irqchip/mips-gic.h
   - Inline __gic_init()
   - Inline gic_basic_init()
   - Make pcpu_masks a per-cpu variable
   - Use pcpu_masks to avoid reading GIC_SH_MASK*
   - Clean up mti, reserved-cpu-vectors handling
   - Use cpumask_first_and() in gic_set_affinity()
   - Let the core set struct irq_common_data affinity

  microMIPS:
   - Fix microMIPS stack unwinding on big endian systems

  MIPS-GIC:
   - SYNC after enabling GIC region

  NUMA:
   - Remove the unused parent_node() macro

  R6:
   - Constify r2_decoder_tables
   - Add accessor & bit definitions for GlobalNumber

  SMP:
   - Constify smp ops
   - Allow boot_secondary SMP op to return errors

  VDSO:
   - Drop gic_get_usm_range() usage
   - Avoid use of linux/irqchip/mips-gic.h

  Platform changes:

  Alchemy:
   - Add devboard machine type to cpuinfo
   - update cpu feature overrides
   - Threaded carddetect irqs for devboards

  AR7:
   - allow NULL clock for clk_get_rate

  BCM63xx:
   - Fix ENETDMA_6345_MAXBURST_REG offset
   - Allow NULL clock for clk_get_rate

  CI20:
   - Enable GPIO and RTC drivers in defconfig
   - Add ethernet and fixed-regulator nodes to DTS

  Generic platform:
   - Move Boston and NI 169445 FIT image source to their own files
   - Include asm/bootinfo.h for plat_fdt_relocated()
   - Include asm/time.h for get_c0_*_int()
   - Include asm/bootinfo.h for plat_fdt_relocated()
   - Include asm/time.h for get_c0_*_int()
   - Allow filtering enabled boards by requirements
   - Don't explicitly disable CONFIG_USB_SUPPORT
   - Bump default NR_CPUS to 16

  JZ4700:
   - Probe the jz4740-rtc driver from devicetree

  Lantiq:
   - Drop check of boot select from the spi-falcon driver.
   - Drop check of boot select from the lantiq-flash MTD driver.
   - Access boot cause register in the watchdog driver through regmap
   - Add device tree binding documentation for the watchdog driver
   - Add docs for the RCU DT bindings.
   - Convert the fpi bus driver to a platform_driver
   - Remove ltq_reset_cause() and ltq_boot_select(
   - Switch to a proper reset driver
   - Switch to a new drivers/soc GPHY driver
   - Add an USB PHY driver for the Lantiq SoCs using the RCU module
   - Use of_platform_default_populate instead of __dt_register_buses
   - Enable MFD_SYSCON to be able to use it for the RCU MFD
   - Replace ltq_boot_select() with dummy implementation.

  Loongson 2F:
   - Allow NULL clock for clk_get_rate

  Malta:
   - Use new GIC accessor functions

  NI 169445:
   - Add support for NI 169445 board.
   - Only include in 32r2el kernels

  Octeon:
   - Add support for watchdog of 78XX SOCs.
   - Add support for watchdog of CN68XX SOCs.
   - Expose support for mips32r1, mips32r2 and mips64r1
   - Enable more drivers in config file
   - Add support for accessing the boot vector.
   - Remove old boot vector code from watchdog driver
   - Define watchdog registers for 70xx, 73xx, 78xx, F75xx.
   - Make CSR functions node aware.
   - Allow access to CIU3 IRQ domains.
   - Misc cleanups in the watchdog driver

  Omega2+:
   - New board, add support and defconfig

  Pistachio:
   - Enable Root FS on NFS in defconfig

  Ralink:
   - Add Mediatek MT7628A SoC
   - Allow NULL clock for clk_get_rate
   - Explicitly request exclusive reset control in the pci-mt7620 PCI driver.

  SEAD3:
   - Only include in 32 bit kernels by default

  VoCore:
   - Add VoCore as a vendor t0 dt-bindings
   - Add defconfig file"

* '4.14-features' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus: (167 commits)
  MIPS: Refactor handling of stack pointer in get_frame_info
  MIPS: Stacktrace: Fix microMIPS stack unwinding on big endian systems
  MIPS: microMIPS: Fix decoding of swsp16 instruction
  MIPS: microMIPS: Fix decoding of addiusp instruction
  MIPS: microMIPS: Fix detection of addiusp instruction
  MIPS: Handle non word sized instructions when examining frame
  MIPS: ralink: allow NULL clock for clk_get_rate
  MIPS: Loongson 2F: allow NULL clock for clk_get_rate
  MIPS: BCM63XX: allow NULL clock for clk_get_rate
  MIPS: AR7: allow NULL clock for clk_get_rate
  MIPS: BCM63XX: fix ENETDMA_6345_MAXBURST_REG offset
  mips: Save all registers when saving the frame
  MIPS: Add DWARF unwinding to assembly
  MIPS: Make SAVE_SOME more standard
  MIPS: Fix issues in backtraces
  MIPS: jz4780: DTS: Probe the jz4740-rtc driver from devicetree
  MIPS: Ci20: Enable RTC driver
  watchdog: octeon-wdt: Add support for 78XX SOCs.
  watchdog: octeon-wdt: Add support for cn68XX SOCs.
  watchdog: octeon-wdt: File cleaning.
  ...

Allow the boot_secondary SMP op to return an error to __cpu_up(), which
will in turn return it to its caller.

This will allow SMP implementations to return errors quickly in cases
they they know have failed, rather than relying upon __cpu_up()
eventually timing out waiting for the cpu_running completion.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17014/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>