linux.git/arch/powerpc/lib, branch v4.17

powerpc/64s: Add support for a store forwarding barrier at kernel entry/exit

2018-05-22T03:45:31+00:00

On some CPUs we can prevent a vulnerability related to store-to-load
forwarding by preventing store forwarding between privilege domains,
by inserting a barrier in kernel entry and exit paths.

This is known to be the case on at least Power7, Power8 and Power9
powerpc CPUs.

Barriers must be inserted generally before the first load after moving
to a higher privilege, and after the last store before moving to a
lower privilege, HV and PR privilege transitions must be protected.

Barriers are added as patch sections, with all kernel/hypervisor entry
points patched, and the exit points to lower privilge levels patched
similarly to the RFI flush patching.

Firmware advertisement is not implemented yet, so CPU flush types
are hard coded.

Thanks to Michal Suchánek for bug fixes and review.

Signed-off-by: Nicholas Piggin 
Signed-off-by: Mauricio Faria de Oliveira 
Signed-off-by: Michael Neuling 
Signed-off-by: Michal Suchánek 
Signed-off-by: Michael Ellerman 
Signed-off-by: Linus Torvalds

powerpc/lib: Fix off-by-one in alternate feature patching

2018-04-16T14:37:48+00:00

When we patch an alternate feature section, we have to adjust any
relative branches that branch out of the alternate section.

But currently we have a bug if we have a branch that points to past
the last instruction of the alternate section, eg:

  FTR_SECTION_ELSE
  1:     b       2f
         or      6,6,6
  2:
  ALT_FTR_SECTION_END(...)
         nop

This will result in a relative branch at 1 with a target that equals
the end of the alternate section.

That branch does not need adjusting when it's moved to the non-else
location. Currently we do adjust it, resulting in a branch that goes
off into the link-time location of the else section, which is junk.

The fix is to not patch branches that have a target == end of the
alternate section.

Fixes: d20fe50a7b3c ("KVM: PPC: Book3S HV: Branch inside feature section")
Fixes: 9b1a735de64c ("powerpc: Add logic to patch alternative feature sections")
Cc: stable@vger.kernel.org # v2.6.27+
Signed-off-by: Michael Ellerman

powerpc/64s: Set assembler machine type to POWER4

2018-03-31T13:47:49+00:00

Rather than override the machine type in .S code (which can hide wrong
or ambiguous code generation for the target), set the type to power4
for all assembly.

This also means we need to be careful not to build power4-only code
when we're not building for Book3S, such as the "power7" versions of
copyuser/page/memcpy.

Signed-off-by: Nicholas Piggin 
[mpe: Fix Book3E build, don't build the "power7" variants for non-Book3S]
Signed-off-by: Michael Ellerman

powerpc/rfi-flush: Differentiate enabled and patched flush types

2018-03-27T08:25:14+00:00

Currently the rfi-flush messages print 'Using  flush' for all
enabled_flush_types, but that is not necessarily true -- as now the
fallback flush is always enabled on pseries, but the fixup function
overwrites its nop/branch slot with other flush types, if available.

So, replace the 'Using  flush' messages with ' flush is
available'.

Also, print the patched flush types in the fixup function, so users
can know what is (not) being used (e.g., the slower, fallback flush,
or no flush type at all if flush is disabled via the debugfs switch).

Suggested-by: Michael Ellerman 
Signed-off-by: Mauricio Faria de Oliveira 
Signed-off-by: Michael Ellerman

powerpc/32: Move the inline keyword at the beginning of function declaration

2018-03-13T04:50:33+00:00

The inline keyword was not at the beginning of the function declaration.
Fix the following warning (treated as error in W=1):

  arch/powerpc/lib/sstep.c:283:1: error: ‘inline’ is not at beginning of declaration
   static int nokprobe_inline copy_mem_in(u8 *dest, unsigned long ea, int nb,
  arch/powerpc/lib/sstep.c:388:1: error: ‘inline’ is not at beginning of declaration
   static int nokprobe_inline copy_mem_out(u8 *dest, unsigned long ea, int nb,

Signed-off-by: Mathieu Malaterre 
Signed-off-by: Michael Ellerman

Merge branch 'fixes' into next

2018-01-21T12:21:14+00:00

Merge our fixes branch from the 4.15 cycle.

Unusually the fixes branch saw some significant features merged,
notably the RFI flush patches, so we want the code in next to be
tested against that, to avoid any surprises when the two are merged.

There's also some other work on the panic handling that was reverted
in fixes and we now want to do properly in next, which would conflict.

And we also fix a few other minor merge conflicts.

powerpc/lib/feature-fixups: use raw_patch_instruction()

2018-01-21T04:06:25+00:00

feature fixups need to use patch_instruction() early in the boot,
even before the code is relocated to its final address, requiring
patch_instruction() to use PTRRELOC() in order to address data.

But feature fixups applies on code before it is set to read only,
even for modules. Therefore, feature fixups can use
raw_patch_instruction() instead.

Signed-off-by: Christophe Leroy 
Signed-off-by: Michael Ellerman

powerpc/lib/code-patching: refactor patch_instruction()

2018-01-21T04:06:25+00:00

patch_instruction() uses almost the same sequence as
__patch_instruction()

This patch refactor it so that patch_instruction() uses
__patch_instruction() instead of duplicating code.

Signed-off-by: Christophe Leroy 
Acked-by: Balbir Singh 
Signed-off-by: Michael Ellerman

powerpc/64s: Add support for RFI flush of L1-D cache

2018-01-10T10:27:06+00:00

On some CPUs we can prevent the Meltdown vulnerability by flushing the
L1-D cache on exit from kernel to user mode, and from hypervisor to
guest.

This is known to be the case on at least Power7, Power8 and Power9. At
this time we do not know the status of the vulnerability on other CPUs
such as the 970 (Apple G5), pasemi CPUs (AmigaOne X1000) or Freescale
CPUs. As more information comes to light we can enable this, or other
mechanisms on those CPUs.

The vulnerability occurs when the load of an architecturally
inaccessible memory region (eg. userspace load of kernel memory) is
speculatively executed to the point where its result can influence the
address of a subsequent speculatively executed load.

In order for that to happen, the first load must hit in the L1,
because before the load is sent to the L2 the permission check is
performed. Therefore if no kernel addresses hit in the L1 the
vulnerability can not occur. We can ensure that is the case by
flushing the L1 whenever we return to userspace. Similarly for
hypervisor vs guest.

In order to flush the L1-D cache on exit, we add a section of nops at
each (h)rfi location that returns to a lower privileged context, and
patch that with some sequence. Newer firmwares are able to advertise
to us that there is a special nop instruction that flushes the L1-D.
If we do not see that advertised, we fall back to doing a displacement
flush in software.

For guest kernels we support migration between some CPU versions, and
different CPUs may use different flush instructions. So that we are
prepared to migrate to a machine with a different flush instruction
activated, we may have to patch more than one flush instruction at
boot if the hypervisor tells us to.

In the end this patch is mostly the work of Nicholas Piggin and
Michael Ellerman. However a cast of thousands contributed to analysis
of the issue, earlier versions of the patch, back ports testing etc.
Many thanks to all of them.

Tested-by: Jon Masters 
Signed-off-by: Nicholas Piggin 
Signed-off-by: Michael Ellerman

powerpc/modules: Don't try to restore r2 after a sibling call

2017-12-11T02:03:29+00:00

When attempting to load a livepatch module, I got the following error:

  module_64: patch_module: Expect noop after relocate, got 3c820000

The error was triggered by the following code in
unregister_netdevice_queue():

  14c:   00 00 00 48     b       14c 
                         14c: R_PPC64_REL24      net_set_todo
  150:   00 00 82 3c     addis   r4,r2,0

GCC didn't insert a nop after the branch to net_set_todo() because it's
a sibling call, so it never returns.  The nop isn't needed after the
branch in that case.

Signed-off-by: Josh Poimboeuf 
Acked-by: Naveen N. Rao 
Reviewed-and-tested-by: Kamalesh Babulal 
Signed-off-by: Michael Ellerman