linux.git/arch/x86/include/asm/text-patching.h, branch v5.6

x86/kprobes: Fix ordering while text-patching

2019-11-27T06:44:24+00:00

Kprobes does something like:

register:
	arch_arm_kprobe()
	  text_poke(INT3)
          /* guarantees nothing, INT3 will become visible at some point, maybe */

        kprobe_optimizer()
	  /* guarantees the bytes after INT3 are unused */
	  synchronize_rcu_tasks();
	  text_poke_bp(JMP32);
	  /* implies IPI-sync, kprobe really is enabled */

unregister:
	__disarm_kprobe()
	  unoptimize_kprobe()
	    text_poke_bp(INT3 + tail);
	    /* implies IPI-sync, so tail is guaranteed visible */
          arch_disarm_kprobe()
            text_poke(old);
	    /* guarantees nothing, old will maybe become visible */

	synchronize_rcu()

        free-stuff

Now the problem is that on register, the synchronize_rcu_tasks() does
not imply sufficient to guarantee all CPUs have already observed INT3
(although in practice this is exceedingly unlikely not to have
happened) (similar to how MEMBARRIER_CMD_PRIVATE_EXPEDITED does not
imply MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE).

Worse, even if it did, we'd have to do 2 synchronize calls to provide
the guarantee we're looking for, the first to ensure INT3 is visible,
the second to guarantee nobody is then still using the instruction
bytes after INT3.

Similar on unregister; the synchronize_rcu() between
__unregister_kprobe_top() and __unregister_kprobe_bottom() does not
guarantee all CPUs are free of the INT3 (and observe the old text).

Therefore, sprinkle some IPI-sync love around. This guarantees that
all CPUs agree on the text and RCU once again provides the required
guaranteed.

Tested-by: Alexei Starovoitov 
Tested-by: Steven Rostedt (VMware) 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Mathieu Desnoyers 
Acked-by: Masami Hiramatsu 
Acked-by: Alexei Starovoitov 
Acked-by: Paul E. McKenney 
Cc: Andy Lutomirski 
Cc: Borislav Petkov 
Cc: Brian Gerst 
Cc: Denys Vlasenko 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Link: https://lkml.kernel.org/r/20191111132458.162172862@infradead.org
Signed-off-by: Ingo Molnar

x86/kprobes: Convert to text-patching.h

2019-11-27T06:44:24+00:00

Convert kprobes to the new text-poke naming.

Tested-by: Alexei Starovoitov 
Tested-by: Steven Rostedt (VMware) 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Masami Hiramatsu 
Acked-by: Alexei Starovoitov 
Cc: Andy Lutomirski 
Cc: Borislav Petkov 
Cc: Brian Gerst 
Cc: Denys Vlasenko 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Link: https://lkml.kernel.org/r/20191111132458.103959370@infradead.org
Signed-off-by: Ingo Molnar

x86/ftrace: Use text_gen_insn()

2019-11-27T06:44:24+00:00

Replace the ftrace_code_union with the generic text_gen_insn() helper,
which does exactly this.

Tested-by: Alexei Starovoitov 
Tested-by: Steven Rostedt (VMware) 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Alexei Starovoitov 
Cc: Andy Lutomirski 
Cc: Borislav Petkov 
Cc: H. Peter Anvin 
Cc: Josh Poimboeuf 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Link: https://lkml.kernel.org/r/20191111132457.932808000@infradead.org
Signed-off-by: Ingo Molnar

x86/alternative: Add text_opcode_size()

2019-11-27T06:44:24+00:00

Introduce a common helper to map *_INSN_OPCODE to *_INSN_SIZE.

Tested-by: Alexei Starovoitov 
Tested-by: Steven Rostedt (VMware) 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Alexei Starovoitov 
Cc: Andy Lutomirski 
Cc: Borislav Petkov 
Cc: H. Peter Anvin 
Cc: Josh Poimboeuf 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Link: https://lkml.kernel.org/r/20191111132457.875666061@infradead.org
Signed-off-by: Ingo Molnar

x86/alternatives: Add and use text_gen_insn() helper

2019-11-27T06:44:24+00:00

Provide a simple helper function to create common instruction
encodings.

Tested-by: Alexei Starovoitov 
Tested-by: Steven Rostedt (VMware) 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Alexei Starovoitov 
Cc: Andy Lutomirski 
Cc: Borislav Petkov 
Cc: Daniel Bristot de Oliveira 
Cc: H. Peter Anvin 
Cc: Josh Poimboeuf 
Cc: Linus Torvalds 
Cc: Masami Hiramatsu 
Cc: Steven Rostedt 
Cc: Thomas Gleixner 
Link: https://lkml.kernel.org/r/20191111132457.703538332@infradead.org
Signed-off-by: Ingo Molnar

x86/alternatives, jump_label: Provide better text_poke() batching interface

2019-11-27T06:44:24+00:00

Adding another text_poke_bp_batch() user made me realize the interface
is all sorts of wrong. The text poke vector should be internal to the
implementation.

This then results in a trivial interface:

  text_poke_queue()  - which has the 'normal' text_poke_bp() interface
  text_poke_finish() - which takes no arguments and flushes any
                       pending text_poke()s.

Tested-by: Alexei Starovoitov 
Tested-by: Steven Rostedt (VMware) 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Masami Hiramatsu 
Reviewed-by: Daniel Bristot de Oliveira 
Acked-by: Alexei Starovoitov 
Cc: Andy Lutomirski 
Cc: Borislav Petkov 
Cc: H. Peter Anvin 
Cc: Josh Poimboeuf 
Cc: Linus Torvalds 
Cc: Steven Rostedt 
Cc: Thomas Gleixner 
Link: https://lkml.kernel.org/r/20191111132457.646280715@infradead.org
Signed-off-by: Ingo Molnar

x86/alternatives: Update int3_emulate_push() comment

2019-11-27T06:44:24+00:00

Update the comment now that we've merged x86_32 support.

Tested-by: Alexei Starovoitov 
Tested-by: Steven Rostedt (VMware) 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Alexei Starovoitov 
Cc: Andy Lutomirski 
Cc: Borislav Petkov 
Cc: H. Peter Anvin 
Cc: Josh Poimboeuf 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Link: https://lkml.kernel.org/r/20191111132457.588386013@infradead.org
Signed-off-by: Ingo Molnar

x86/alternatives: Teach text_poke_bp() to emulate instructions

2019-11-15T22:07:01+00:00

In preparation for static_call and variable size jump_label support,
teach text_poke_bp() to emulate instructions, namely:

  JMP32, JMP8, CALL, NOP2, NOP_ATOMIC5, INT3

The current text_poke_bp() takes a @handler argument which is used as
a jump target when the temporary INT3 is hit by a different CPU.

When patching CALL instructions, this doesn't work because we'd miss
the PUSH of the return address. Instead, teach poke_int3_handler() to
emulate an instruction, typically the instruction we're patching in.

This fits almost all text_poke_bp() users, except
arch_unoptimize_kprobe() which restores random text, and for that site
we have to build an explicit emulate instruction.

Tested-by: Alexei Starovoitov 
Tested-by: Steven Rostedt (VMware) 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Masami Hiramatsu 
Reviewed-by: Daniel Bristot de Oliveira 
Acked-by: Alexei Starovoitov 
Cc: Andy Lutomirski 
Cc: Borislav Petkov 
Cc: H. Peter Anvin 
Cc: Josh Poimboeuf 
Cc: Linus Torvalds 
Cc: Steven Rostedt 
Cc: Thomas Gleixner 
Link: https://lkml.kernel.org/r/20191111132457.529086974@infradead.org
Signed-off-by: Ingo Molnar 
(cherry picked from commit 8c7eebc10687af45ac8e40ad1bac0cf7893dba9f)
Signed-off-by: Alexei Starovoitov

x86: Correct misc typos

2019-09-02T12:02:59+00:00

Correct spelling typos in comments in different files under arch/x86/.

 [ bp: Merge into a single patch, massage. ]

Signed-off-by: Marco Ammon 
Signed-off-by: Borislav Petkov 
Cc: Daniel Bristot de Oliveira 
Cc: "H. Peter Anvin" 
Cc: Ingo Molnar 
Cc: Juergen Gross 
Cc: Masami Hiramatsu 
Cc: Nadav Amit 
Cc: Peter Zijlstra 
Cc: Pu Wen 
Cc: Rick Edgecombe 
Cc: "Steven Rostedt (VMware)" 
Cc: Thomas Gleixner 
Cc: trivial@kernel.org
Cc: x86-ml 
Link: https://lkml.kernel.org/r/20190902102436.27396-1-marco.ammon@fau.de

Merge branch 'x86-asm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2019-07-08T23:59:34+00:00

Pull x86 asm updates from Ingo Molnar:
 "Most of the changes relate to Peter Zijlstra's cleanup of ptregs
  handling, in particular the i386 part is now much simplified and
  standardized - no more partial ptregs stack frames via the esp/ss
  oddity. This simplifies ftrace, kprobes, the unwinder, ptrace, kdump
  and kgdb.

  There's also a CR4 hardening enhancements by Kees Cook, to make the
  generic platform functions such as native_write_cr4() less useful as
  ROP gadgets that disable SMEP/SMAP. Also protect the WP bit of CR0
  against similar attacks.

  The rest is smaller cleanups/fixes"

* 'x86-asm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86/alternatives: Add int3_emulate_call() selftest
  x86/stackframe/32: Allow int3_emulate_push()
  x86/stackframe/32: Provide consistent pt_regs
  x86/stackframe, x86/ftrace: Add pt_regs frame annotations
  x86/stackframe, x86/kprobes: Fix frame pointer annotations
  x86/stackframe: Move ENCODE_FRAME_POINTER to asm/frame.h
  x86/entry/32: Clean up return from interrupt preemption path
  x86/asm: Pin sensitive CR0 bits
  x86/asm: Pin sensitive CR4 bits
  Documentation/x86: Fix path to entry_32.S
  x86/asm: Remove unused TASK_TI_flags from asm-offsets.c