linux.git/tools/tracing, branch v6.16

Merge tag 'trace-tools-v6.16' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace

2025-05-30T03:59:52+00:00

Pull tracing tools updates from Steven Rostedt:

 - Set distinctive value for failed tests

   When running "make check" that performs tests on rtla the failure is
   checked by examining the output. Instead have the tool return an
   error status if it exceeds the threadhold.

 - Define __NR_sched_setattr for LoongArch

   Define __NR_sched_setattr to allow this to build for LoongArch.

 - Define _GNU_SOURCE for timerlat_bpf.c

   Due to modifications of struct sched_attr in utils.h when _GNU_SOURCE
   is not defined, this can cause errors for timerlat_bpf_init() and
   breakage in BPF sample collection mode.

* tag 'trace-tools-v6.16' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
  rtla: Define _GNU_SOURCE in timerlat_bpf.c
  rtla: Define __NR_sched_setattr for LoongArch
  rtla: Set distinctive exit value for failed tests

rtla: Define _GNU_SOURCE in timerlat_bpf.c

2025-05-07T21:25:46+00:00

Newer versions of glibc include a definition of struct sched_attr in
bits/sched.h (included through sched.h which is included by rtla).
Commit 0eecee340672 ("tools/rtla: fix collision with glibc
sched_attr/sched_set_attr") has modified the definition of struct
sched_attr in utils.h, so that it is only applied with older versions of
glibc that do not define it, in order to prevent build failure.

The definition in bits/sched.h depends on _GNU_SOURCE.
timerlat_bpf.c does not define _GNU_SOURCE, making it fall back to the
definition in utils.h. The latter has two fields less, leading to
shifted offsets of struct timerlat_params in timerlat_bpf_init.

Because of the shift, timerlat_bpf_init incorrectly reads
params->entries as 0 for timerlat-hist and disables the creation of
histogram maps, causing breakage in BPF sample collection mode:

$ rtla timerlat hist -d 1s
Error pulling BPF data

Fix the issue by also defining _GNU_SOURCE in timerlat_bpf.c.

Cc: John Kacur 
Cc: Luis Goncalves 
Link: https://lore.kernel.org/20250430144651.621766-1-tglozar@redhat.com
Fixes: e34293ddcebd ("rtla/timerlat: Add BPF skeleton to collect samples")
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla: Define __NR_sched_setattr for LoongArch

2025-05-07T20:35:31+00:00

When executing "make -C tools/tracing/rtla" on LoongArch, there exists
the following error:

  src/utils.c:237:24: error: '__NR_sched_setattr' undeclared

Just define __NR_sched_setattr for LoongArch if not exist.

Link: https://lore.kernel.org/20250422074917.25771-1-yangtiezhu@loongson.cn
Reported-by: Haiyong Sun 
Signed-off-by: Tiezhu Yang 
Signed-off-by: Steven Rostedt (Google)

rtla: Set distinctive exit value for failed tests

2025-05-07T19:36:19+00:00

A test is considered failed when a sample trace exceeds the threshold.
Failed tests return the same exit code as passed tests, requiring test
frameworks to determine the result by searching for "hit stop tracing"
in the output.

Assign a distinct exit code for failed tests to enable the use of shell
expressions and seamless integration with testing frameworks without the
need to parse output.

Add enum type for return value.

Update `make check`.

Cc: Daniel Bristot de Oliveira 
Cc: John Kacur 
Cc: "Luis Claudio R. Goncalves" 
Cc: Eder Zulian 
Cc: Dan Carpenter 
Cc: Jan Stancek 
Link: https://lore.kernel.org/20250417185757.2194541-1-costa.shul@redhat.com
Signed-off-by: Costa Shulyupin 
Reviewed-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

Documentation/rtla: Include BPF sample collection

2025-04-14T16:42:55+00:00

Add dependencies needed to build rtla with BPF sample collection support
to README, and document both ways of sample collection in the manpages.

Signed-off-by: Tomas Glozar 
Acked-by: Steven Rostedt (Google) 
Reviewed-by: Luis Claudio R. Goncalves 
Signed-off-by: Jonathan Corbet 
Link: https://lore.kernel.org/r/20250311114936.148012-5-tglozar@redhat.com

Merge tag 'trace-tools-v6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace

2025-03-28T00:03:01+00:00

Pull tracing tooling updates from Steven Rostedt:

 - Allow RTLA to collect data via BPF

   The current implementation of rtla uses libtracefs and libtraceevent
   to pull sample events generated by the timerlat tracer from the trace
   buffer. rtla then processes the sample by updating the histogram and
   summary (current, maximum, minimum, and sum values) as well as checks
   if tracing has been stopped due to threshold overflow.

   In use cases where a large number of samples is being generated, that
   is, with measurements running on many CPUs and with a low interval,
   this sample processing design causes a significant CPU load on the
   rtla side. Furthermore, with >100 CPUs and 100us interval, rtla was
   reported as not being able to keep up with the samples and dropping
   most of them, leading to it being unusable.

   Change the way the timerlat trace processes samples by attaching a
   BPF program to the trace event using the BPF skeleton feature of
   bpftool. Unlike the current implementation, the BPF implementation
   does not check whether tracing is stopped (in BPF mode, tracing is
   always off to improve performance), but waits for a write to a BPF
   ringbuffer instead. This allows rtla to exit immediately when a
   threshold is violated, without waiting for the next iteration of the
   while loop.

   If the requirements for the BPF implementation are not met, either at
   build time or at run time, the current implementation is used as
   fallback. Which implementation is being used can be seen when running
   rtla timerlat with "-D" option. rtla can be forced to run in non-BPF
   mode by setting the RTLA_NO_BPF option to 1, for debugging purposes.

 - Fix LD_FLAGS from being dropped in build

 - Refactor code to remove duplication of save_trace_to_file

 - Always set options and do not rely on default settings

   Do not rely on the default kernel settings of the tracers when
   starting. They could have been changed by the user which gives
   inconsistent results. Always set the options that rtla expects.

 - Add creation of ctags and TAGS for traversing code

* tag 'trace-tools-v6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
  rtla: Add the ability to create ctags and etags
  rtla/tests: Test setting default options
  rtla/tests: Reset osnoise options before check
  rtla: Always set all tracer options
  rtla/osnoise: Set OSNOISE_WORKLOAD to true
  rtla: Unify apply_config between top and hist
  rtla/osnoise: Unify params struct
  rtla: Fix segfault in save_trace_to_file call
  tools/build: Use SYSTEM_BPFTOOL for system bpftool
  rtla: Refactor save_trace_to_file
  tools/rv: Keep user LDFLAGS in build
  rtla/timerlat: Test BPF mode
  rtla/timerlat_top: Use BPF to collect samples
  rtla/timerlat_top: Move divisor to update
  rtla/timerlat_hist: Use BPF to collect samples
  rtla/timerlat: Add BPF skeleton to collect samples
  rtla: Add optional dependency on BPF tooling
  tools/build: Add bpftool-skeletons feature test
  rtla/timerlat: Unify params struct

rtla: Add the ability to create ctags and etags

2025-03-26T14:39:40+00:00

- Add the ability to create and remove ctags and etags, using the following
make tags
make TAGS
make tags_clean

- fix a comment in Makefile.rtla with the correct spelling and don't
  imply that the ability to create an rtla tarball will be removed

Cc: Tomas Glozar 
Cc: "Luis Claudio R . Goncalves" 
Link: https://lore.kernel.org/20250321175053.29048-1-jkacur@redhat.com
Signed-off-by: John Kacur 
Signed-off-by: Steven Rostedt (Google)

rtla/tests: Test setting default options

2025-03-26T14:36:39+00:00

Add function to test engine to test with pre-set osnoise options, and
use it to test whether osnoise period (as an example) is set correctly.

The test works by pre-setting a high period of 10 minutes and stop on
threshold. Thus, it is easy to check whether rtla is properly resetting
the period to default: if it is, the test will complete on time, since
the first sample will overflow the threshold. If not, it will time out.

Cc: Luis Goncalves 
Link: https://lore.kernel.org/20250320092500.101385-7-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Reviewed-by: John Kacur 
Signed-off-by: Steven Rostedt (Google)

rtla/tests: Reset osnoise options before check

2025-03-26T14:36:39+00:00

Remove any dangling tracing instances from previous improperly exited
runs of rtla, and reset osnoise options to default before running a test
case.

This ensures that the test results are deterministic. Specific test
cases checked that rtla behaves correctly even when the tracer state is
not clean will be added later.

Cc: John Kacur 
Cc: Luis Goncalves 
Link: https://lore.kernel.org/20250320092500.101385-6-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla: Always set all tracer options

2025-03-26T14:36:39+00:00

rtla currently only sets tracer options that are explicitly set by the
user, with the exception of OSNOISE_WORKLOAD.

This leads to improper behavior in case rtla is run with those options
not set to the default value. rtla does reset them to the original
value upon exiting, but that does not protect it from starting with
non-default values set either by an improperly exited rtla or by another
user of the tracers.

For example, after running this command:

$ echo 1 > /sys/kernel/tracing/osnoise/stop_tracing_us

all runs of rtla will stop at the 1us threshold, even if not requested
by the user:

$ rtla osnoise hist
Index   CPU-000   CPU-001
1             8         5
2             5         9
3             1         2
4             6         1
5             2         1
6             0         1
8             1         1
12            0         1
14            1         0
15            1         0
over:         0         0
count:       25        21
min:          1         1
avg:       3.68      3.05
max:         15        12
rtla osnoise hit stop tracing

Fix the problem by setting the default value for all tracer options if
the user has not provided their own value.

For most of the options, it's enough to just drop the if clause checking
for the value being set. For cpus, "all" is used as the default value,
and for osnoise default period and runtime, default values of
the osnoise_data variable in trace_osnoise.c are used.

Cc: Luis Goncalves 
Link: https://lore.kernel.org/20250320092500.101385-5-tglozar@redhat.com
Fixes: 1eceb2fc2ca5 ("rtla/osnoise: Add osnoise top mode")
Fixes: 829a6c0b5698 ("rtla/osnoise: Add the hist mode")
Fixes: a828cd18bc4a ("rtla: Add timerlat tool and timelart top mode")
Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode")
Signed-off-by: Tomas Glozar 
Reviewed-by: John Kacur 
Signed-off-by: Steven Rostedt (Google)