linux-stable.git/tools/tracing/rtla/src, branch v6.15

rtla: Always set all tracer options

2025-03-26T14:36:39+00:00

rtla currently only sets tracer options that are explicitly set by the
user, with the exception of OSNOISE_WORKLOAD.

This leads to improper behavior in case rtla is run with those options
not set to the default value. rtla does reset them to the original
value upon exiting, but that does not protect it from starting with
non-default values set either by an improperly exited rtla or by another
user of the tracers.

For example, after running this command:

$ echo 1 > /sys/kernel/tracing/osnoise/stop_tracing_us

all runs of rtla will stop at the 1us threshold, even if not requested
by the user:

$ rtla osnoise hist
Index   CPU-000   CPU-001
1             8         5
2             5         9
3             1         2
4             6         1
5             2         1
6             0         1
8             1         1
12            0         1
14            1         0
15            1         0
over:         0         0
count:       25        21
min:          1         1
avg:       3.68      3.05
max:         15        12
rtla osnoise hit stop tracing

Fix the problem by setting the default value for all tracer options if
the user has not provided their own value.

For most of the options, it's enough to just drop the if clause checking
for the value being set. For cpus, "all" is used as the default value,
and for osnoise default period and runtime, default values of
the osnoise_data variable in trace_osnoise.c are used.

Cc: Luis Goncalves 
Link: https://lore.kernel.org/20250320092500.101385-5-tglozar@redhat.com
Fixes: 1eceb2fc2ca5 ("rtla/osnoise: Add osnoise top mode")
Fixes: 829a6c0b5698 ("rtla/osnoise: Add the hist mode")
Fixes: a828cd18bc4a ("rtla: Add timerlat tool and timelart top mode")
Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode")
Signed-off-by: Tomas Glozar 
Reviewed-by: John Kacur 
Signed-off-by: Steven Rostedt (Google)

rtla/osnoise: Set OSNOISE_WORKLOAD to true

2025-03-26T14:36:39+00:00

If running rtla osnoise with NO_OSNOISE_WORKLOAD, it reports no samples:

$ echo NO_OSNOISE_WORKLOAD > /sys/kernel/tracing/osnoise/options
$ rtla osnoise hist -d 10s
Index
over: 0
count: 0
min: 0
avg: 0
max: 0

This situation can also happen when running rtla-osnoise after an
improperly exited rtla-timerlat run.

Set OSNOISE_WORKLOAD in rtla-osnoise, too, similarly to what we
already did for timerlat in commit 217f0b1e990e ("rtla/timerlat_top: Set
OSNOISE_WORKLOAD for kernel threads") and commit d8d866171a41
("rtla/timerlat_hist: Set OSNOISE_WORKLOAD for kernel threads").

Note that there is no user workload mode for rtla-osnoise yet, so
OSNOISE_WORKLOAD is always set to true.

Cc: Luis Goncalves 
Link: https://lore.kernel.org/20250320092500.101385-4-tglozar@redhat.com
Fixes: 1eceb2fc2ca5 ("rtla/osnoise: Add osnoise top mode")
Fixes: 829a6c0b5698 ("rtla/osnoise: Add the hist mode")
Signed-off-by: Tomas Glozar 
Reviewed-by: John Kacur 
Signed-off-by: Steven Rostedt (Google)

rtla: Unify apply_config between top and hist

2025-03-26T14:36:39+00:00

The functions osnoise_top_apply_config and osnoise_hist_apply_config, as
well as timerlat_top_apply_config and timerlat_hist_apply_config, are
mostly the same.

Move common part from them into separate functions osnoise_apply_config
and timerlat_apply_config.

For rtla-timerlat, also unify params->user_hist and params->user_top
into one field called params->user_data, and move several fields used
only by timerlat-top into the top-only section of struct
timerlat_params.

Cc: Luis Goncalves 
Link: https://lore.kernel.org/20250320092500.101385-3-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Reviewed-by: John Kacur 
Signed-off-by: Steven Rostedt (Google)

rtla/osnoise: Unify params struct

2025-03-26T14:36:39+00:00

Instead of having separate structs osnoise_top_params and
osnoise_hist_params, use one struct osnoise_params for both.

This allows code using the structs to be shared between osnoise-top and
osnoise-hist.

Cc: Luis Goncalves 
Link: https://lore.kernel.org/20250320092500.101385-2-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Reviewed-by: John Kacur 
Signed-off-by: Steven Rostedt (Google)

rtla: Fix segfault in save_trace_to_file call

2025-03-26T14:35:20+00:00

Running rtla with exit on threshold, but without saving trace leads to a
segmenetation fault:

$ rtla timerlat hist -T 10
...
Max timerlat IRQ latency from idle: 4.29 us in cpu 0
Segmentation fault

This is caused by null pointer deference in the call of
save_trace_to_file, which attempts to dereference an uninitialized
osnoise_tool variable:

save_trace_to_file(record->trace.inst, params->trace_output);
                   ^ this is uninitialized if params->trace_output is
                     not set

Fix this by not attempting to dereference "record" if it is NULL and
passing NULL instead. As a safety measure, the first field is also
checked for NULL inside save_trace_to_file.

Cc: John Kacur 
Cc: Luis Goncalves 
Cc: Costa Shulyupin 
Link: https://lore.kernel.org/20250313141034.299117-1-tglozar@redhat.com
Fixes: dc4d4e7c72d1 ("rtla: Refactor save_trace_to_file")
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla: Refactor save_trace_to_file

2025-03-04T19:17:19+00:00

The functions osnoise_hist_main(), osnoise_top_main(),
timerlat_hist_main(), and timerlat_top_main() are lengthy and contain
duplicated code.

Refactor by consolidating the duplicate lines into the
save_trace_to_file() function.

Cc: Daniel Bristot de Oliveira 
Cc: John Kacur 
Cc: "Luis Claudio R. Goncalves" 
Cc: Eder Zulian 
Cc: Dan Carpenter 
Cc: Gabriele Monaco 
Link: https://lore.kernel.org/20250219115138.406075-1-costa.shul@redhat.com
Signed-off-by: Costa Shulyupin 
Reviewed-by: Tomas Glozar 
Tested-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla/timerlat_top: Use BPF to collect samples

2025-03-04T17:35:18+00:00

Collect samples using BPF program instead of pulling them from tracefs.

If the osnoise:timerlat_sample tracepoint is unavailable or the BPF
program fails to load for whatever reason, rtla falls back to the old
implementation.

The collection of samples using the BPF program is fully self-contained
and requires no activity of the userspace part of rtla during the
measurement. Thus, rtla only pulls the summary from the BPF map and
displays it every second, improving the performance.

In --aa-only mode, the BPF program does not collect any data and only
signalizes the end of tracing to userspace. An optimization that re-used
the main trace instance for auto-analysis in aa-only mode was dropped, as
rtla no longer turns tracing on in the main trace instance, making it
useless for auto-analysis.

Cc: John Kacur 
Cc: Luis Goncalves 
Cc: Gabriele Monaco 
Cc: Clark Williams 
Link: https://lore.kernel.org/20250218145859.27762-8-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla/timerlat_top: Move divisor to update

2025-03-04T17:35:18+00:00

Unlike timerlat-hist, timerlat-top applies the output divisor used to
set ns/us mode when printing results instead of applying it when
collecting the samples.

Move the application of the divisor from timerlat_top_print into
timerlat_top_update to make it consistent with timerlat-hist.

Cc: John Kacur 
Cc: Luis Goncalves 
Cc: Gabriele Monaco 
Cc: Clark Williams 
Link: https://lore.kernel.org/20250218145859.27762-7-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla/timerlat_hist: Use BPF to collect samples

2025-03-04T17:35:18+00:00

Collect samples using BPF program instead of pulling them from tracefs.

If the osnoise:timerlat_sample tracepoint is unavailable or the BPF
program fails to load for whatever reason, rtla falls back to the old
implementation.

The collection of samples using the BPF program is fully self-contained
and requires no activity of the userspace part of rtla during the
measurement. Thus, instead of waking up every second to collect samples,
rtla simply sleeps until woken up by a signal or threshold overflow.

Cc: John Kacur 
Cc: Luis Goncalves 
Cc: Gabriele Monaco 
Cc: Clark Williams 
Link: https://lore.kernel.org/20250218145859.27762-6-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla/timerlat: Add BPF skeleton to collect samples

2025-03-04T17:35:18+00:00

Add BPF program that attaches to the osnoise:timerlat_sample tracepoint
and collects both the summary and the histogram (if requested) into BPF
maps (one map of each kind per context).

The program is designed to be used for both timerlat-top and
timerlat-hist. If using with timerlat-top, the "entries" parameter is
set to zero, which prevents the BPF program from recording histogram
entries. In that case, the maps for histograms do not have to be
created, as the BPF verifier will identify the code using them as
unreachable.

An IRQ or thread latency threshold might be supplied to stop recording
if hit, similar to the timerlat tracer threshold, which stops ftrace
tracing if hit. A BPF ringbuffer is used to signal threshold overflow to
userspace. In aa-only mode, this is the only function of the BPF
program.

Cc: John Kacur 
Cc: Luis Goncalves 
Cc: Gabriele Monaco 
Cc: Clark Williams 
Link: https://lore.kernel.org/20250218145859.27762-5-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)