linux-stable.git/arch/x86/lib, branch linux-3.16.y

x86/cpufeature: Carve out X86_FEATURE_*

2019-05-22T22:15:08+00:00

commit cd4d09ec6f6c12a2cc3db5b7d8876a325a53545b upstream.

Move them to a separate header and have the following
dependency:

  x86/cpufeatures.h <- x86/processor.h <- x86/cpufeature.h

This makes it easier to use the header in asm code and not
include the whole cpufeature.h and add guards for asm.

Suggested-by: H. Peter Anvin 
Signed-off-by: Borislav Petkov 
Cc: Andy Lutomirski 
Cc: Borislav Petkov 
Cc: Brian Gerst 
Cc: Denys Vlasenko 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Link: http://lkml.kernel.org/r/1453842730-28463-5-git-send-email-bp@alien8.de
Signed-off-by: Ingo Molnar 
[bwh: Backported to 3.16 to avoid a dependency loop:
 - Drop some inapplicable changes
 - Move all the previously backported feature and bug flags across
 - Also change  and lib/retpoline.S to use
   
 - Also include  in , as the vdso fails to
   build without that
 - Adjust context]
Signed-off-by: Ben Hutchings

x86/uaccess: Use __uaccess_begin_nospec() and uaccess_try_nospec

2018-03-19T18:59:17+00:00

commit 304ec1b050310548db33063e567123fae8fd0301 upstream.

Quoting Linus:

    I do think that it would be a good idea to very expressly document
    the fact that it's not that the user access itself is unsafe. I do
    agree that things like "get_user()" want to be protected, but not
    because of any direct bugs or problems with get_user() and friends,
    but simply because get_user() is an excellent source of a pointer
    that is obviously controlled from a potentially attacking user
    space. So it's a prime candidate for then finding _subsequent_
    accesses that can then be used to perturb the cache.

__uaccess_begin_nospec() covers __get_user() and copy_from_iter() where the
limit check is far away from the user pointer de-reference. In those cases
a barrier_nospec() prevents speculation with a potential pointer to
privileged memory. uaccess_try_nospec covers get_user_try.

Suggested-by: Linus Torvalds 
Suggested-by: Andi Kleen 
Signed-off-by: Dan Williams 
Signed-off-by: Thomas Gleixner 
Cc: linux-arch@vger.kernel.org
Cc: Kees Cook 
Cc: kernel-hardening@lists.openwall.com
Cc: gregkh@linuxfoundation.org
Cc: Al Viro 
Cc: alan@linux.intel.com
Link: https://lkml.kernel.org/r/151727416953.33451.10508284228526170604.stgit@dwillia2-desk3.amr.corp.intel.com
[bwh: Backported to 3.16:
 - Convert several more functions to use __uaccess_begin_nospec(), that
   are just wrappers in mainline
 - There's no 'case 8' in __copy_to_user_inatomic()
 - Adjust context]
Signed-off-by: Ben Hutchings

x86/usercopy: Replace open coded stac/clac with __uaccess_{begin, end}

2018-03-19T18:59:17+00:00

commit b5c4ae4f35325d520b230bab6eb3310613b72ac1 upstream.

In preparation for converting some __uaccess_begin() instances to
__uacess_begin_nospec(), make sure all 'from user' uaccess paths are
using the _begin(), _end() helpers rather than open-coded stac() and
clac().

No functional changes.

Suggested-by: Ingo Molnar 
Signed-off-by: Dan Williams 
Signed-off-by: Thomas Gleixner 
Cc: linux-arch@vger.kernel.org
Cc: Tom Lendacky 
Cc: Kees Cook 
Cc: kernel-hardening@lists.openwall.com
Cc: gregkh@linuxfoundation.org
Cc: Al Viro 
Cc: torvalds@linux-foundation.org
Cc: alan@linux.intel.com
Link: https://lkml.kernel.org/r/151727416438.33451.17309465232057176966.stgit@dwillia2-desk3.amr.corp.intel.com
[bwh: Backported to 3.16:
 - Convert several more functions to use __uaccess_begin_nospec(), that
   are just wrappers in mainline
 - Adjust context]
Signed-off-by: Ben Hutchings

x86/get_user: Use pointer masking to limit speculation

2018-03-19T18:59:13+00:00

commit c7f631cb07e7da06ac1d231ca178452339e32a94 upstream.

Quoting Linus:

    I do think that it would be a good idea to very expressly document
    the fact that it's not that the user access itself is unsafe. I do
    agree that things like "get_user()" want to be protected, but not
    because of any direct bugs or problems with get_user() and friends,
    but simply because get_user() is an excellent source of a pointer
    that is obviously controlled from a potentially attacking user
    space. So it's a prime candidate for then finding _subsequent_
    accesses that can then be used to perturb the cache.

Unlike the __get_user() case get_user() includes the address limit check
near the pointer de-reference. With that locality the speculation can be
mitigated with pointer narrowing rather than a barrier, i.e.
array_index_nospec(). Where the narrowing is performed by:

	cmp %limit, %ptr
	sbb %mask, %mask
	and %mask, %ptr

With respect to speculation the value of %ptr is either less than %limit
or NULL.

Co-developed-by: Linus Torvalds 
Signed-off-by: Dan Williams 
Signed-off-by: Thomas Gleixner 
Cc: linux-arch@vger.kernel.org
Cc: Kees Cook 
Cc: kernel-hardening@lists.openwall.com
Cc: gregkh@linuxfoundation.org
Cc: Al Viro 
Cc: Andy Lutomirski 
Cc: torvalds@linux-foundation.org
Cc: alan@linux.intel.com
Link: https://lkml.kernel.org/r/151727417469.33451.11804043010080838495.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Ben Hutchings

x86/retpoline: Remove the esp/rsp thunk

2018-03-19T18:59:10+00:00

commit 1df37383a8aeabb9b418698f0bcdffea01f4b1b2 upstream.

It doesn't make sense to have an indirect call thunk with esp/rsp as
retpoline code won't work correctly with the stack pointer register.
Removing it will help compiler writers to catch error in case such
a thunk call is emitted incorrectly.

Fixes: 76b043848fd2 ("x86/retpoline: Add initial retpoline support")
Suggested-by: Jeff Law 
Signed-off-by: Waiman Long 
Signed-off-by: Thomas Gleixner 
Acked-by: David Woodhouse 
Cc: Tom Lendacky 
Cc: Kees Cook 
Cc: Andi Kleen 
Cc: Tim Chen 
Cc: Peter Zijlstra 
Cc: Linus Torvalds 
Cc: Jiri Kosina 
Cc: Andy Lutomirski 
Cc: Dave Hansen 
Cc: Josh Poimboeuf 
Cc: Arjan van de Ven 
Cc: Greg Kroah-Hartman 
Cc: Paul Turner 
Link: https://lkml.kernel.org/r/1516658974-27852-1-git-send-email-longman@redhat.com
[bwh: Backported to 3.16: adjust filename]
Signed-off-by: Ben Hutchings

kprobes/x86: Blacklist indirect thunk functions for kprobes

2018-03-19T18:59:09+00:00

commit c1804a236894ecc942da7dc6c5abe209e56cba93 upstream.

Mark __x86_indirect_thunk_* functions as blacklist for kprobes
because those functions can be called from anywhere in the kernel
including blacklist functions of kprobes.

Signed-off-by: Masami Hiramatsu 
Signed-off-by: Thomas Gleixner 
Acked-by: David Woodhouse 
Cc: Andi Kleen 
Cc: Peter Zijlstra 
Cc: Ananth N Mavinakayanahalli 
Cc: Arjan van de Ven 
Cc: Greg Kroah-Hartman 
Link: https://lkml.kernel.org/r/151629209111.10241.5444852823378068683.stgit@devbox
[bwh: Backported to 3.16: exports are still done from C]
Signed-off-by: Ben Hutchings

retpoline: Introduce start/end markers of indirect thunk

2018-03-19T18:59:09+00:00

commit 736e80a4213e9bbce40a7c050337047128b472ac upstream.

Introduce start/end markers of __x86_indirect_thunk_* functions.
To make it easy, consolidate .text.__x86.indirect_thunk.* sections
to one .text.__x86.indirect_thunk section and put it in the
end of kernel text section and adds __indirect_thunk_start/end
so that other subsystem (e.g. kprobes) can identify it.

Signed-off-by: Masami Hiramatsu 
Signed-off-by: Thomas Gleixner 
Acked-by: David Woodhouse 
Cc: Andi Kleen 
Cc: Peter Zijlstra 
Cc: Ananth N Mavinakayanahalli 
Cc: Arjan van de Ven 
Cc: Greg Kroah-Hartman 
Link: https://lkml.kernel.org/r/151629206178.10241.6828804696410044771.stgit@devbox
[bwh: Backported to 3.16: adjust context]
Signed-off-by: Ben Hutchings

x86/retpoline/checksum32: Convert assembler indirect jumps

2018-03-19T18:59:08+00:00

commit 5096732f6f695001fa2d6f1335a2680b37912c69 upstream.

Convert all indirect jumps in 32bit checksum assembler code to use
non-speculative sequences when CONFIG_RETPOLINE is enabled.

Signed-off-by: David Woodhouse 
Signed-off-by: Thomas Gleixner 
Acked-by: Arjan van de Ven 
Acked-by: Ingo Molnar 
Cc: gnomes@lxorguk.ukuu.org.uk
Cc: Rik van Riel 
Cc: Andi Kleen 
Cc: Josh Poimboeuf 
Cc: thomas.lendacky@amd.com
Cc: Peter Zijlstra 
Cc: Linus Torvalds 
Cc: Jiri Kosina 
Cc: Andy Lutomirski 
Cc: Dave Hansen 
Cc: Kees Cook 
Cc: Tim Chen 
Cc: Greg Kroah-Hartman 
Cc: Paul Turner 
Link: https://lkml.kernel.org/r/1515707194-20531-11-git-send-email-dwmw@amazon.co.uk
[bwh: Backported to 3.16: adjust context]
Signed-off-by: Ben Hutchings

x86/retpoline: Add initial retpoline support

2018-03-19T18:59:06+00:00

commit 76b043848fd22dbf7f8bf3a1452f8c70d557b860 upstream.

Enable the use of -mindirect-branch=thunk-extern in newer GCC, and provide
the corresponding thunks. Provide assembler macros for invoking the thunks
in the same way that GCC does, from native and inline assembler.

This adds X86_FEATURE_RETPOLINE and sets it by default on all CPUs. In
some circumstances, IBRS microcode features may be used instead, and the
retpoline can be disabled.

On AMD CPUs if lfence is serialising, the retpoline can be dramatically
simplified to a simple "lfence; jmp *\reg". A future patch, after it has
been verified that lfence really is serialising in all circumstances, can
enable this by setting the X86_FEATURE_RETPOLINE_AMD feature bit in addition
to X86_FEATURE_RETPOLINE.

Do not align the retpoline in the altinstr section, because there is no
guarantee that it stays aligned when it's copied over the oldinstr during
alternative patching.

[ Andi Kleen: Rename the macros, add CONFIG_RETPOLINE option, export thunks]
[ tglx: Put actual function CALL/JMP in front of the macros, convert to
  	symbolic labels ]
[ dwmw2: Convert back to numeric labels, merge objtool fixes ]

Signed-off-by: David Woodhouse 
Signed-off-by: Thomas Gleixner 
Acked-by: Arjan van de Ven 
Acked-by: Ingo Molnar 
Cc: gnomes@lxorguk.ukuu.org.uk
Cc: Rik van Riel 
Cc: Andi Kleen 
Cc: Josh Poimboeuf 
Cc: thomas.lendacky@amd.com
Cc: Peter Zijlstra 
Cc: Linus Torvalds 
Cc: Jiri Kosina 
Cc: Andy Lutomirski 
Cc: Dave Hansen 
Cc: Kees Cook 
Cc: Tim Chen 
Cc: Greg Kroah-Hartman 
Cc: Paul Turner 
Link: https://lkml.kernel.org/r/1515707194-20531-4-git-send-email-dwmw@amazon.co.uk
[bwh: Backported to 3.16:
 - Add C source to export the thunk symbols
 - Drop ANNOTATE_NOSPEC_ALTERNATIVE since we don't have objtool
 - Use the first available feaure numbers
 - Adjust filename, context]
Signed-off-by: Ben Hutchings

x86/decoder: Add new TEST instruction pattern

2018-02-13T18:42:30+00:00

commit 12a78d43de767eaf8fb272facb7a7b6f2dc6a9df upstream.

The kbuild test robot reported this build warning:

  Warning: arch/x86/tools/test_get_len found difference at :ffffffff8103dd2c

  Warning: ffffffff8103dd82: f6 09 d8 testb $0xd8,(%rcx)
  Warning: objdump says 3 bytes, but insn_get_length() says 2
  Warning: decoded and checked 1569014 instructions with 1 warnings

This sequence seems to be a new instruction not in the opcode map in the Intel SDM.

The instruction sequence is "F6 09 d8", means Group3(F6), MOD(00)REG(001)RM(001), and 0xd8.
Intel SDM vol2 A.4 Table A-6 said the table index in the group is "Encoding of Bits 5,4,3 of
the ModR/M Byte (bits 2,1,0 in parenthesis)"

In that table, opcodes listed by the index REG bits as:

  000         001       010 011  100        101        110         111
 TEST Ib/Iz,(undefined),NOT,NEG,MUL AL/rAX,IMUL AL/rAX,DIV AL/rAX,IDIV AL/rAX

So, it seems TEST Ib is assigned to 001.

Add the new pattern.

Reported-by: kbuild test robot 
Signed-off-by: Masami Hiramatsu 
Cc: Greg Kroah-Hartman 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar 
Signed-off-by: Ben Hutchings