linux.git/kernel/trace/fgraph.c, branch v6.11-rc2

function_graph: Add READ_ONCE() when accessing fgraph_array[]

2024-06-14T16:37:08+00:00

In function_graph_enter() there's a loop that looks at fgraph_array[]
elements which are fgraph_ops. It first tests if it is a fgraph_stub op,
and if so skips it, as that's just there as a place holder. Then it checks
the fgraph_ops filters to see if the ops wants to trace the current
function.

But if the compiler reloads the fgraph_array[] after the check against
fgraph_stub, it could race with the fgraph_array[] being updated with the
fgraph_stub. That would cause the stub to be processed. But the stub has a
null "func_hash" field which will cause a NULL pointer dereference.

Add a READ_ONCE() so that the gops that is compared against the
fgraph_stub is also the gops that is processed later.

Link: https://lore.kernel.org/all/CA+G9fYsSVJQZH=nM=1cjTc94PgSnMF9y65BnOv6XSoCG_b6wmw@mail.gmail.com/
Link: https://lore.kernel.org/linux-trace-kernel/20240613095223.1f07e3a4@rorschach.local.home

Cc: Mark Rutland 
Cc: Mathieu Desnoyers 
Fixes: cc60ee813b503 ("function_graph: Use static_call and branch to optimize entry function")
Reported-by: Naresh Kamboju 
Signed-off-by: Steven Rostedt (Google) 
Acked-by: Masami Hiramatsu (Google)

function_graph: Everyone uses HAVE_FUNCTION_GRAPH_RET_ADDR_PTR, remove it

2024-06-11T15:18:24+00:00

All architectures that implement function graph also implements
HAVE_FUNCTION_GRAPH_RET_ADDR_PTR. Remove it, as it is no longer a
differentiator.

Link: https://lore.kernel.org/linux-trace-kernel/20240611031737.982047614@goodmis.org

Cc: Masami Hiramatsu 
Cc: Mark Rutland 
Cc: Mathieu Desnoyers 
Cc: Jonathan Corbet 
Cc: Catalin Marinas 
Cc: Will Deacon 
Cc: Guo Ren 
Cc: Huacai Chen 
Cc: WANG Xuerui 
Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Cc: Christophe Leroy 
Cc: "Naveen N. Rao" 
Cc: Paul Walmsley 
Cc: Palmer Dabbelt 
Cc: Albert Ou 
Cc: Heiko Carstens 
Cc: Vasily Gorbik 
Cc: Alexander Gordeev 
Cc: Christian Borntraeger 
Cc: Sven Schnelle 
Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: Borislav Petkov 
Cc: Dave Hansen 
Cc: "H. Peter Anvin" 
Signed-off-by: Steven Rostedt (Google)

function_graph: Fix up ftrace_graph_ret_addr()

2024-06-11T15:18:24+00:00

Yang Li sent a patch to fix the kerneldoc of ftrace_graph_ret_addr().
While reviewing it, I realized that the comments in the entire function
header needed a rewrite. When doing that, I realized that @idx parameter
was being ignored. Every time this was called by the unwinder, it would
start the loop at the top of the shadow stack and look for the matching
stack pointer. When it found it, it would return it. When the unwinder
asked for the next function, it would search from the beginning again.

In reality, it should start from where it left off. That was the reason
for the @idx parameter in the first place. The first time the unwinder
calls this function, the @idx pointer would contain zero. That would mean
to start from the top of the stack. The function was supposed to update
the @idx with the index where it found the return address, so that the
next time the unwinder calls this function it doesn't have to search
through the previous addresses it found (making it O(n^2)!).

This speeds up the unwinder's use of ftrace_graph_ret_addr() by an order
of magnitude.

Link: https://lore.kernel.org/linux-trace-kernel/20240610181746.656e3759@gandalf.local.home/
Link: https://lore.kernel.org/linux-trace-kernel/20240611031737.821995106@goodmis.org

Cc: Masami Hiramatsu 
Cc: Mark Rutland 
Cc: Mathieu Desnoyers 
Cc: Jonathan Corbet 
Cc: Catalin Marinas 
Cc: Will Deacon 
Cc: Guo Ren 
Cc: Huacai Chen 
Cc: WANG Xuerui 
Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Cc: Christophe Leroy 
Cc: "Naveen N. Rao" 
Cc: Paul Walmsley 
Cc: Palmer Dabbelt 
Cc: Albert Ou 
Cc: Heiko Carstens 
Cc: Vasily Gorbik 
Cc: Alexander Gordeev 
Cc: Christian Borntraeger 
Cc: Sven Schnelle 
Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: Borislav Petkov 
Cc: Dave Hansen 
Cc: "H. Peter Anvin" 
Reported-by: Yang Li 
Fixes: 7aa1eaef9f428 ("function_graph: Allow multiple users to attach to function graph")
Signed-off-by: Steven Rostedt (Google)

function_graph: Make fgraph_update_pid_func() a stub for !DYNAMIC_FTRACE

2024-06-10T22:08:23+00:00

When CONFIG_DYNAMIC_FTRACE is not set, the function
fgraph_update_pid_func() doesn't do anything. Currently, most of its logic
is within a "#ifdef CONFIG_DYNAMIC_FTRACE" block, but its variables were
declared outside that, and when DYNAMIC_FTRACE is not set, it produces
unused variable warnings.

Instead, just place it (and the helper function fgraph_pid_func()) within
the #ifdef block and have the header file use a empty stub function for
when DYNAMIC_FTRACE is not defined.

Link: https://lore.kernel.org/linux-trace-kernel/20240607094833.6a787d73@rorschach.local.home

Cc: Masami Hiramatsu 
Cc: Mark Rutland 
Cc: Mathieu Desnoyers 
Reported-by: kernel test robot 
Closes: https://lore.kernel.org/oe-kbuild-all/202406071806.BRjaC5FF-lkp@intel.com/
Signed-off-by: Steven Rostedt (Google)

fgraph: Remove some unused functions

2024-06-06T19:22:19+00:00

These functions are defined in the fgraph.c file, but not
called elsewhere, so delete these unused functions.

kernel/trace/fgraph.c:273:1: warning: unused function 'set_bitmap_bits'.
kernel/trace/fgraph.c:259:19: warning: unused function 'get_fgraph_type'.

Link: https://lore.kernel.org/linux-trace-kernel/20240606021053.27783-1-jiapeng.chong@linux.alibaba.com

Reported-by: Abaci Robot 
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=9289
Signed-off-by: Jiapeng Chong 
Signed-off-by: Steven Rostedt (Google)

function_graph: Do not update pid func if CONFIG_DYNAMIC_FTRACE not enabled

2024-06-06T19:22:18+00:00

The ftrace subops is only defined if CONFIG_DYNAMIC_FTRACE is enabled. If
it is not, function tracing is extremely limited, and the subops in the
ftrace_ops structure is not defined (and will fail to compile). If
DYNAMIC_FTRACE is not enabled, then function graph filtering will not
work (as it shouldn't).

Link: https://lore.kernel.org/linux-trace-kernel/20240605202709.096020676@goodmis.org

Cc: Mark Rutland 
Cc: Mathieu Desnoyers 
Cc: Andrew Morton 
Fixes: df3ec5da6a1e7 ("function_graph: Add pid tracing back to function graph tracer")
Reported-by: kernel test robot 
Closes: https://lore.kernel.org/oe-kbuild-all/202406051855.9VIYXbTB-lkp@intel.com/
Acked-by: Masami Hiramatsu (Google) 
Signed-off-by: Steven Rostedt (Google)

function_graph: Make fgraph_do_direct static key static

2024-06-06T19:22:18+00:00

The static branch key "fgraph_do_direct" was not declared static but is
only used in one file. Change it to a static variable.

Link: https://lore.kernel.org/linux-trace-kernel/20240605202708.936515302@goodmis.org

Cc: Mark Rutland 
Cc: Mathieu Desnoyers 
Cc: Andrew Morton 
Fixes: cc60ee813b503 ("function_graph: Use static_call and branch to optimize entry function")
Reported-by: kernel test robot 
Closes: https://lore.kernel.org/oe-kbuild-all/202406051711.dS1sQZ9n-lkp@intel.com/
Acked-by: Masami Hiramatsu (Google) 
Signed-off-by: Steven Rostedt (Google)

function_graph: Use static_call and branch to optimize return function

2024-06-04T14:38:23+00:00

In most cases function graph is used by a single user. Instead of calling
a loop to call function graph callbacks in this case, call the function
return callback directly.

Use the static_key that is set when the function graph tracer has less
than 2 callbacks registered. It will do the direct call in that case, and
will do the loop over all callers when there are 2 or more callbacks
registered.

Link: https://lore.kernel.org/linux-trace-kernel/20240603190824.921460797@goodmis.org

Cc: Masami Hiramatsu 
Cc: Mark Rutland 
Cc: Mathieu Desnoyers 
Cc: Andrew Morton 
Cc: Alexei Starovoitov 
Cc: Florent Revest 
Cc: Martin KaFai Lau 
Cc: bpf 
Cc: Sven Schnelle 
Cc: Alexei Starovoitov 
Cc: Jiri Olsa 
Cc: Arnaldo Carvalho de Melo 
Cc: Daniel Borkmann 
Cc: Alan Maguire 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Guo Ren 
Reviewed-by: Masami Hiramatsu (Google) 
Signed-off-by: Steven Rostedt (Google)

function_graph: Use static_call and branch to optimize entry function

2024-06-04T14:38:15+00:00

In most cases function graph is used by a single user. Instead of calling
a loop to call function graph callbacks in this case, call the function
entry callback directly.

Add a static_key that will be used to set the function graph logic to
either do the loop (when more than one callback is registered) or to call
the callback directly if there is only one registered callback.

Link: https://lore.kernel.org/linux-trace-kernel/20240603190824.766858241@goodmis.org

Cc: Masami Hiramatsu 
Cc: Mark Rutland 
Cc: Mathieu Desnoyers 
Cc: Andrew Morton 
Cc: Alexei Starovoitov 
Cc: Florent Revest 
Cc: Martin KaFai Lau 
Cc: bpf 
Cc: Sven Schnelle 
Cc: Alexei Starovoitov 
Cc: Jiri Olsa 
Cc: Arnaldo Carvalho de Melo 
Cc: Daniel Borkmann 
Cc: Alan Maguire 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Guo Ren 
Reviewed-by: Masami Hiramatsu (Google) 
Signed-off-by: Steven Rostedt (Google)

function_graph: Use bitmask to loop on fgraph entry

2024-06-04T14:38:10+00:00

Instead of looping through all the elements of fgraph_array[] to see if
there's an gops attached to one and then calling its gops->func(). Create
a fgraph_array_bitmask that sets bits when an index in the array is
reserved (via the simple lru algorithm). Then only the bits set in this
bitmask needs to be looked at where only elements in the array that have
ops registered need to be looked at.

Note, we do not care about races. If a bit is set before the gops is
assigned, it only wastes time looking at the element and ignoring it (as
it did before this bitmask is added).

Link: https://lore.kernel.org/linux-trace-kernel/20240603190824.604448781@goodmis.org

Cc: Mark Rutland 
Cc: Mathieu Desnoyers 
Cc: Andrew Morton 
Cc: Alexei Starovoitov 
Cc: Florent Revest 
Cc: Martin KaFai Lau 
Cc: bpf 
Cc: Sven Schnelle 
Cc: Alexei Starovoitov 
Cc: Jiri Olsa 
Cc: Arnaldo Carvalho de Melo 
Cc: Daniel Borkmann 
Cc: Alan Maguire 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Guo Ren 
Reviewed-by: Masami Hiramatsu (Google) 
Signed-off-by: Steven Rostedt (Google)