Multicore MIPSes without I/D hardware coherency suffered from a race
condition in the page fault handler. The page table entry was
published before any pending lazy D-cache flush was committed, hence
it allowed execution of stale page cache data by other VPEs in the
system.

To make the cache handling safe we need to perform flushing already in
the set_pte_at function. MIPSes without coherent I-caches can get a
small increase in flushes due to the unavailability of the execute
flag in set_pte_at.

[ralf@linux-mips.org: outlining set_pte_at() saves a good k in a test
build, so I moved its definition from pgtable.h to cache.c.]

Signed-off-by: Lars Persson <larper@axis.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/7511/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

The Hardware Page Table Walker aims to speed up TLB refill exceptions
by handling them in the hardware level instead of having a software
TLB refill handler. However, a TLB refill exception can still be
thrown in certain cases such as, synchronus exceptions, or address
translation or memory errors during the HTW operation. As a result of
which, HTW must not be considered a complete replacement for the TLB
refill software handler, but rather a fast-path for it.
For HTW to work, the PWBase register must contain the task's page
global directory address so the HTW will kick in on TLB refill
exceptions.

Due to HTW being a separate engine embedded deep in the CPU pipeline,
we need to restart the HTW everytime a PTE changes to avoid HTW
fetching a old entry from the page tables. It's also necessary to
restart the HTW on context switches to prevent it from fetching a
page from the previous process. Finally, since HTW is using the
entryhi register to write the translations to the TLB, it's necessary
to stop the HTW whenever the entryhi changes (eg for tlb probe
perations) and enable it back afterwards.

== Performance ==

The following trivial test was used to measure the performance of the
HTW. Using the same root filesystem, the following command was used
to measure the number of tlb refill handler executions with and
without (using 'nohtw' kernel parameter) HTW support.  The kernel was
modified to use a scratch register as a counter for the TLB refill
exceptions.

find /usr -type f -exec ls -lh {} \;

HTW Enabled:
TLB refill exceptions: 12306

HTW Disabled:
TLB refill exceptions: 17805

Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: Markos Chandras <markos.chandras@imgtec.com>
Patchwork: https://patchwork.linux-mips.org/patch/7336/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

This is identical to kmap_coherent apart from the cache coherency
attribute used for the TLB entry, so kmap_coherent is abstracted to
kmap_prot which is then called for both kmap_coherent &
kmap_noncoherent. This will be used by a subsequent patch.

Suggested-by: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Signed-off-by: Paul Burton <paul.burton@imgtec.com>

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

With the addition of transparent huge pages, pgtable.h uses struct page.
However, it is possible to include pgtable.h without anything defining
struct page.  So add the include to get it.

Signed-off-by: Corey Minyard <cminyard@mvista.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

Having received another series of whitespace patches I decided to do this
once and for all rather than dealing with this kind of patches trickling
in forever.

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

Pull MIPS updates from Ralf Baechle:
 "The MIPS bits for 3.8.  This also includes a bunch fixes that were
  sitting in the linux-mips.org git tree for a long time.  This pull
  request contains updates to several OCTEON drivers and the board
  support code for BCM47XX, BCM63XX, XLP, XLR, XLS, lantiq, Loongson1B,
  updates to the SSB bus support, MIPS kexec code and adds support for
  kdump.

  When pulling this, there are two expected merge conflicts in
  include/linux/bcma/bcma_driver_chipcommon.h which are trivial to
  resolve, just remove the conflict markers and keep both alternatives."

* 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus: (90 commits)
  MIPS: PMC-Sierra Yosemite: Remove support.
  VIDEO: Newport Fix console crashes
  MIPS: wrppmc: Fix build of PCI code.
  MIPS: IP22/IP28: Fix build of EISA code.
  MIPS: RB532: Fix build of prom code.
  MIPS: PowerTV: Fix build.
  MIPS: IP27: Correct fucked grammar in ops-bridge.c
  MIPS: Highmem: Fix build error if CONFIG_DEBUG_HIGHMEM is disabled
  MIPS: Fix potencial corruption
  MIPS: Fix for warning from FPU emulation code
  MIPS: Handle COP3 Unusable exception as COP1X for FP emulation
  MIPS: Fix poweroff failure when HOTPLUG_CPU configured.
  MIPS: MT: Fix build with CONFIG_UIDGID_STRICT_TYPE_CHECKS=y
  MIPS: Remove unused smvp.h
  MIPS/EDAC: Improve OCTEON EDAC support.
  MIPS: OCTEON: Add definitions for OCTEON memory contoller registers.
  MIPS: OCTEON: Add OCTEON family definitions to octeon-model.h
  ata: pata_octeon_cf: Use correct byte order for DMA in when built little-endian.
  MIPS/OCTEON/ata: Convert pata_octeon_cf.c to use device tree.
  MIPS: Remove usage of CEVT_R4K_LIB config option.
  ...

We have two different implementation of is_zero_pfn() and my_zero_pfn()
helpers: for architectures with and without zero page coloring.

Let's consolidate them in <asm-generic/pgtable.h>.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

Signed-off-by: Ralf Baechle <ralf@linux-mips.org>