linux-stable.git/arch/x86/kernel/cpu/amd.c, branch linux-4.1.y

x86/cpu/AMD: Use LFENCE_RDTSC in preference to MFENCE_RDTSC

2018-05-20T03:15:41+00:00

[ Upstream commit 9c6a73c75864ad9fa49e5fa6513e4c4071c0e29f ]

With LFENCE now a serializing instruction, use LFENCE_RDTSC in preference
to MFENCE_RDTSC.  However, since the kernel could be running under a
hypervisor that does not support writing that MSR, read the MSR back and
verify that the bit has been set successfully.  If the MSR can be read
and the bit is set, then set the LFENCE_RDTSC feature, otherwise set the
MFENCE_RDTSC feature.

Signed-off-by: Tom Lendacky 
Signed-off-by: Thomas Gleixner 
Reviewed-by: Reviewed-by: Borislav Petkov 
Cc: Peter Zijlstra 
Cc: Tim Chen 
Cc: Dave Hansen 
Cc: Borislav Petkov 
Cc: Dan Williams 
Cc: Linus Torvalds 
Cc: Greg Kroah-Hartman 
Cc: David Woodhouse 
Cc: Paul Turner 
Link: https://lkml.kernel.org/r/20180108220932.12580.52458.stgit@tlendack-t1.amdoffice.net
Signed-off-by: Sasha Levin

x86/cpu/AMD: Make LFENCE a serializing instruction

2018-05-20T03:15:41+00:00

[ Upstream commit e4d0e84e490790798691aaa0f2e598637f1867ec ]

To aid in speculation control, make LFENCE a serializing instruction
since it has less overhead than MFENCE.  This is done by setting bit 1
of MSR 0xc0011029 (DE_CFG).  Some families that support LFENCE do not
have this MSR.  For these families, the LFENCE instruction is already
serializing.

Signed-off-by: Tom Lendacky 
Signed-off-by: Thomas Gleixner 
Reviewed-by: Reviewed-by: Borislav Petkov 
Cc: Peter Zijlstra 
Cc: Tim Chen 
Cc: Dave Hansen 
Cc: Borislav Petkov 
Cc: Dan Williams 
Cc: Linus Torvalds 
Cc: Greg Kroah-Hartman 
Cc: David Woodhouse 
Cc: Paul Turner 
Link: https://lkml.kernel.org/r/20180108220921.12580.71694.stgit@tlendack-t1.amdoffice.net
Signed-off-by: Sasha Levin

x86/AMD: Apply erratum 665 on machines without a BIOS fix

2016-09-15T22:53:30+00:00

[ Upstream commit d1992996753132e2dafe955cccb2fb0714d3cfc4 ]

AMD F12h machines have an erratum which can cause DIV/IDIV to behave
unpredictably. The workaround is to set MSRC001_1029[31] but sometimes
there is no BIOS update containing that workaround so let's do it
ourselves unconditionally. It is simple enough.

[ Borislav: Wrote commit message. ]

Signed-off-by: Emanuel Czirai 
Signed-off-by: Borislav Petkov 
Cc: Yaowu Xu 
Cc: stable@vger.kernel.org
Link: http://lkml.kernel.org/r/20160902053550.18097-1-bp@alien8.de
Signed-off-by: Thomas Gleixner 
Signed-off-by: Sasha Levin

x86_64, asm: Work around AMD SYSRET SS descriptor attribute issue

2015-04-27T00:57:38+00:00

AMD CPUs don't reinitialize the SS descriptor on SYSRET, so SYSRET with
SS == 0 results in an invalid usermode state in which SS is apparently
equal to __USER_DS but causes #SS if used.

Work around the issue by setting SS to __KERNEL_DS __switch_to, thus
ensuring that SYSRET never happens with SS set to NULL.

This was exposed by a recent vDSO cleanup.

Fixes: e7d6eefaaa44 x86/vdso32/syscall.S: Do not load __USER32_DS to %ss
Signed-off-by: Andy Lutomirski 
Cc: Peter Anvin 
Cc: Borislav Petkov 
Cc: Denys Vlasenko 
Cc: Brian Gerst 
Signed-off-by: Linus Torvalds

Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2015-04-13T20:31:32+00:00

Pull x86 mm changes from Ingo Molnar:
 "The main changes in this cycle were:

   - reduce the x86/32 PAE per task PGD allocation overhead from 4K to
     0.032k (Fenghua Yu)

   - early_ioremap/memunmap() usage cleanups (Juergen Gross)

   - gbpages support cleanups (Luis R Rodriguez)

   - improve AMD Bulldozer (family 0x15) ASLR I$ aliasing workaround to
     increase randomization by 3 bits (per bootup) (Hector
     Marco-Gisbert)

   - misc fixlets"

* 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86/mm: Improve AMD Bulldozer ASLR workaround
  x86/mm/pat: Initialize __cachemode2pte_tbl[] and __pte2cachemode_tbl[] in a bit more readable fashion
  init.h: Clean up the __setup()/early_param() macros
  x86/mm: Simplify probe_page_size_mask()
  x86/mm: Further simplify 1 GB kernel linear mappings handling
  x86/mm: Use early_param_on_off() for direct_gbpages
  init.h: Add early_param_on_off()
  x86/mm: Simplify enabling direct_gbpages
  x86/mm: Use IS_ENABLED() for direct_gbpages
  x86/mm: Unexport set_memory_ro() and set_memory_rw()
  x86/mm, efi: Use early_ioremap() in arch/x86/platform/efi/efi-bgrt.c
  x86/mm: Use early_memunmap() instead of early_iounmap()
  x86/mm/pat: Ensure different messages in STRICT_DEVMEM and PAT cases
  x86/mm: Reduce PAE-mode per task pgd allocation overhead from 4K to 32 bytes

x86/mm: Improve AMD Bulldozer ASLR workaround

2015-03-31T08:01:17+00:00

The ASLR implementation needs to special-case AMD F15h processors by
clearing out bits [14:12] of the virtual address in order to avoid I$
cross invalidations and thus performance penalty for certain workloads.
For details, see:

  dfb09f9b7ab0 ("x86, amd: Avoid cache aliasing penalties on AMD family 15h")

This special case reduces the mmapped file's entropy by 3 bits.

The following output is the run on an AMD Opteron 62xx class CPU
processor under x86_64 Linux 4.0.0:

  $ for i in `seq 1 10`; do cat /proc/self/maps | grep "r-xp.*libc" ; done
  b7588000-b7736000 r-xp 00000000 00:01 4924       /lib/i386-linux-gnu/libc.so.6
  b7570000-b771e000 r-xp 00000000 00:01 4924       /lib/i386-linux-gnu/libc.so.6
  b75d0000-b777e000 r-xp 00000000 00:01 4924       /lib/i386-linux-gnu/libc.so.6
  b75b0000-b775e000 r-xp 00000000 00:01 4924       /lib/i386-linux-gnu/libc.so.6
  b7578000-b7726000 r-xp 00000000 00:01 4924       /lib/i386-linux-gnu/libc.so.6
  ...

Bits [12:14] are always 0, i.e. the address always ends in 0x8000 or
0x0000.

32-bit systems, as in the example above, are especially sensitive
to this issue because 32-bit randomness for VA space is 8 bits (see
mmap_rnd()). With the Bulldozer special case, this diminishes to only 32
different slots of mmap virtual addresses.

This patch randomizes per boot the three affected bits rather than
setting them to zero. Since all the shared pages have the same value
at bits [12..14], there is no cache aliasing problems. This value gets
generated during system boot and it is thus not known to a potential
remote attacker. Therefore, the impact from the Bulldozer workaround
gets diminished and ASLR randomness increased.

More details at:

  http://hmarco.org/bugs/AMD-Bulldozer-linux-ASLR-weakness-reducing-mmaped-files-by-eight.html

Original white paper by AMD dealing with the issue:

  http://developer.amd.com/wordpress/media/2012/10/SharedL1InstructionCacheonAMD15hCPU.pdf

Mentored-by: Ismael Ripoll 
Signed-off-by: Hector Marco-Gisbert 
Signed-off-by: Borislav Petkov 
Acked-by: Kees Cook 
Cc: Alexander Viro 
Cc: Andrew Morton 
Cc: H. Peter Anvin 
Cc: Jan-Simon 
Cc: Thomas Gleixner 
Cc: linux-fsdevel@vger.kernel.org
Link: http://lkml.kernel.org/r/1427456301-3764-1-git-send-email-hecmargi@upv.es
Signed-off-by: Ingo Molnar

x86/asm: Cleanup prefetch primitives

2015-02-23T12:44:17+00:00

This is based on a patch originally by hpa.

With the current improvements to the alternatives, we can simply use %P1
as a mem8 operand constraint and rely on the toolchain to generate the
proper instruction sizes. For example, on 32-bit, where we use an empty
old instruction we get:

  apply_alternatives: feat: 6*32+8, old: (c104648b, len: 4), repl: (c195566c, len: 4)
  c104648b: alt_insn: 90 90 90 90
  c195566c: rpl_insn: 0f 0d 4b 5c

  ...

  apply_alternatives: feat: 6*32+8, old: (c18e09b4, len: 3), repl: (c1955948, len: 3)
  c18e09b4: alt_insn: 90 90 90
  c1955948: rpl_insn: 0f 0d 08

  ...

  apply_alternatives: feat: 6*32+8, old: (c1190cf9, len: 7), repl: (c1955a79, len: 7)
  c1190cf9: alt_insn: 90 90 90 90 90 90 90
  c1955a79: rpl_insn: 0f 0d 0d a0 d4 85 c1

all with the proper padding done depending on the size of the
replacement instruction the compiler generates.

Signed-off-by: Borislav Petkov 
Cc: H. Peter Anvin

Merge branch 'perf/hw_breakpoints' into perf/core

2015-01-28T14:48:59+00:00

The new hw_breakpoint bits are now ready for v3.20, merge them
into the main branch, to avoid conflicts.

Conflicts:
	tools/perf/Documentation/perf-record.txt

Signed-off-by: Ingo Molnar

perf/x86/amd: AMD support for bp_len > HW_BREAKPOINT_LEN_8

2014-12-03T14:14:26+00:00

Implement hardware breakpoint address mask for AMD Family 16h and
above processors. CPUID feature bit indicates hardware support for
DRn_ADDR_MASK MSRs. These masks further qualify DRn/DR7 hardware
breakpoint addresses to allow matching of larger addresses ranges.

Valuable advice and pseudo code from Oleg Nesterov 

Signed-off-by: Jacob Shin 
Signed-off-by: Suravee Suthikulpanit 
Acked-by: Jiri Olsa 
Reviewed-by: Oleg Nesterov 
Cc: Arnaldo Carvalho de Melo 
Cc: Ingo Molnar 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: xiakaixu 
Signed-off-by: Frederic Weisbecker

x86, CPU, AMD: Move K8 TLB flush filter workaround to K8 code

2014-11-11T16:58:20+00:00

This belongs with the rest of the code in init_amd_k8() which gets
executed on family 0xf.

Signed-off-by: Borislav Petkov