linux-stable.git/kernel/kprobes.c, branch v5.13.4

kprobe/static_call: Restore missing static_call_text_reserved()

2021-07-20T14:00:41+00:00

[ Upstream commit fa68bd09fc62240a383c0c601d3349c47db10c34 ]

Restore two hunks from commit:

  6333e8f73b83 ("static_call: Avoid kprobes on inline static_call()s")

that went walkabout in a Git merge commit.

Fixes: 76d4acf22b48 ("Merge tag 'perf-kprobes-2020-12-14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip")
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Masami Hiramatsu 
Link: https://lore.kernel.org/r/20210628113045.167127609@infradead.org
Signed-off-by: Ingo Molnar 
Signed-off-by: Sasha Levin

kprobes: Fix to delay the kprobes jump optimization

2021-02-19T19:57:12+00:00

Commit 36dadef23fcc ("kprobes: Init kprobes in early_initcall")
moved the kprobe setup in early_initcall(), which includes kprobe
jump optimization.
The kprobes jump optimizer involves synchronize_rcu_tasks() which
depends on the ksoftirqd and rcu_spawn_tasks_*(). However, since
those are setup in core_initcall(), kprobes jump optimizer can not
run at the early_initcall().

To avoid this issue, make the kprobe optimization disabled in the
early_initcall() and enables it in subsys_initcall().

Note that non-optimized kprobes is still available after
early_initcall(). Only jump optimization is delayed.

Link: https://lkml.kernel.org/r/161365856280.719838.12423085451287256713.stgit@devnote2

Fixes: 36dadef23fcc ("kprobes: Init kprobes in early_initcall")
Cc: Ingo Molnar 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: RCU 
Cc: Michael Ellerman 
Cc: Andrew Morton 
Cc: Daniel Axtens 
Cc: Frederic Weisbecker 
Cc: Neeraj Upadhyay 
Cc: Joel Fernandes 
Cc: Michal Hocko 
Cc: "Theodore Y . Ts'o" 
Cc: Oleksiy Avramchenko 
Cc: stable@vger.kernel.org
Reported-by: Paul E. McKenney 
Reported-by: Sebastian Andrzej Siewior 
Reported-by: Uladzislau Rezki 
Acked-by: Paul E. McKenney 
Signed-off-by: Masami Hiramatsu 
Signed-off-by: Steven Rostedt (VMware)

kprobes: Warn if the kprobe is reregistered

2021-02-09T17:44:32+00:00

Warn if the kprobe is reregistered, since there must be
a software bug (actively used resource must not be re-registered)
and caller must be fixed.

Link: https://lkml.kernel.org/r/161236436734.194052.4058506306336814476.stgit@devnote2

Acked-by: Naveen N. Rao 
Acked-by: Ananth N Mavinakayanahalli 
Signed-off-by: Masami Hiramatsu 
Signed-off-by: Steven Rostedt (VMware)

kretprobe: Avoid re-registration of the same kretprobe earlier

2021-01-29T22:29:16+00:00

Our system encountered a re-init error when re-registering same kretprobe,
where the kretprobe_instance in rp->free_instances is illegally accessed
after re-init.

Implementation to avoid re-registration has been introduced for kprobe
before, but lags for register_kretprobe(). We must check if kprobe has
been re-registered before re-initializing kretprobe, otherwise it will
destroy the data struct of kretprobe registered, which can lead to memory
leak, system crash, also some unexpected behaviors.

We use check_kprobe_rereg() to check if kprobe has been re-registered
before running register_kretprobe()'s body, for giving a warning message
and terminate registration process.

Link: https://lkml.kernel.org/r/20210128124427.2031088-1-bobo.shaobowang@huawei.com

Cc: stable@vger.kernel.org
Fixes: 1f0ab40976460 ("kprobes: Prevent re-registration of the same kprobe")
[ The above commit should have been done for kretprobes too ]
Acked-by: Naveen N. Rao 
Acked-by: Ananth N Mavinakayanahalli 
Acked-by: Masami Hiramatsu 
Signed-off-by: Wang ShaoBo 
Signed-off-by: Cheng Jian 
Signed-off-by: Steven Rostedt (VMware)

tracing/kprobe: Fix to support kretprobe events on unloaded modules

2021-01-29T20:39:48+00:00

Fix kprobe_on_func_entry() returns error code instead of false so that
register_kretprobe() can return an appropriate error code.

append_trace_kprobe() expects the kprobe registration returns -ENOENT
when the target symbol is not found, and it checks whether the target
module is unloaded or not. If the target module doesn't exist, it
defers to probe the target symbol until the module is loaded.

However, since register_kretprobe() returns -EINVAL instead of -ENOENT
in that case, it always fail on putting the kretprobe event on unloaded
modules. e.g.

Kprobe event:
/sys/kernel/debug/tracing # echo p xfs:xfs_end_io >> kprobe_events
[   16.515574] trace_kprobe: This probe might be able to register after target module is loaded. Continue.

Kretprobe event: (p -> r)
/sys/kernel/debug/tracing # echo r xfs:xfs_end_io >> kprobe_events
sh: write error: Invalid argument
/sys/kernel/debug/tracing # cat error_log
[   41.122514] trace_kprobe: error: Failed to register probe event
  Command: r xfs:xfs_end_io
             ^

To fix this bug, change kprobe_on_func_entry() to detect symbol lookup
failure and return -ENOENT in that case. Otherwise it returns -EINVAL
or 0 (succeeded, given address is on the entry).

Link: https://lkml.kernel.org/r/161176187132.1067016.8118042342894378981.stgit@devnote2

Cc: stable@vger.kernel.org
Fixes: 59158ec4aef7 ("tracing/kprobes: Check the probe on unloaded module correctly")
Reported-by: Jianlin Lv 
Signed-off-by: Masami Hiramatsu 
Signed-off-by: Steven Rostedt (VMware)

Merge branch 'linus' into perf/kprobes

2020-11-07T12:18:49+00:00

Merge recent kprobes updates into perf/kprobes that came from -mm.

Signed-off-by: Ingo Molnar

kprobes: Replace rp->free_instance with freelist

2020-10-12T16:27:28+00:00

Gets rid of rp->lock, and as a result kretprobes are now fully
lockless.

Signed-off-by: Peter Zijlstra (Intel) 
Signed-off-by: Masami Hiramatsu 
Signed-off-by: Ingo Molnar 
Link: https://lore.kernel.org/r/159870623583.1229682.17472357584134058687.stgit@devnote2

kprobes: Remove kretprobe hash

2020-10-12T16:27:27+00:00

The kretprobe hash is mostly superfluous, replace it with a per-task
variable.

This gets rid of the task hash and it's related locking.

Note that this may change the kprobes module-exported API for kretprobe
handlers. If any out-of-tree kretprobe user uses ri->rp, use
get_kretprobe(ri) instead.

Signed-off-by: Peter Zijlstra (Intel) 
Signed-off-by: Masami Hiramatsu 
Signed-off-by: Ingo Molnar 
Link: https://lore.kernel.org/r/159870620431.1229682.16325792502413731312.stgit@devnote2

kprobes: Init kprobes in early_initcall

2020-09-22T01:06:04+00:00

Init kprobes feature in early_initcall as same as jump_label and
dynamic_debug does, so that we can use kprobes events in earlier
boot stage.

Link: https://lkml.kernel.org/r/159974151897.478751.8342374158615496628.stgit@devnote2

Signed-off-by: Masami Hiramatsu 
Signed-off-by: Steven Rostedt (VMware)

kprobes: tracing/kprobes: Fix to kill kprobes on initmem after boot

2020-09-18T18:27:24+00:00

Since kprobe_event= cmdline option allows user to put kprobes on the
functions in initmem, kprobe has to make such probes gone after boot.
Currently the probes on the init functions in modules will be handled
by module callback, but the kernel init text isn't handled.
Without this, kprobes may access non-exist text area to disable or
remove it.

Link: https://lkml.kernel.org/r/159972810544.428528.1839307531600646955.stgit@devnote2

Fixes: 970988e19eb0 ("tracing/kprobe: Add kprobe_event= boot parameter")
Cc: Jonathan Corbet 
Cc: Shuah Khan 
Cc: Randy Dunlap 
Cc: Ingo Molnar 
Cc: stable@vger.kernel.org
Signed-off-by: Masami Hiramatsu 
Signed-off-by: Steven Rostedt (VMware)