linux-stable.git/tools/tracing, branch v6.14

Merge tag 'perf-tools-fixes-for-v6.14-2-2025-02-25' of git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools

2025-02-25T21:32:32+00:00

Pull perf tools fixes from Arnaldo Carvalho de Melo:

 - Fix tools/ quiet build Makefile infrastructure that was broken when
   working on tools/perf/ without testing on other tools/ living
   utilities.

* tag 'perf-tools-fixes-for-v6.14-2-2025-02-25' of git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools:
  tools: Remove redundant quiet setup
  tools: Unify top-level quiet infrastructure

tools: Remove redundant quiet setup

2025-02-18T19:27:43+00:00

Q is exported from Makefile.include so it is not necessary to manually
set it.

Reviewed-by: Jiri Olsa 
Signed-off-by: Charlie Jenkins 
Acked-by: Andrii Nakryiko 
Acked-by: Quentin Monnet 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Alexei Starovoitov 
Cc: Benjamin Tissoires 
Cc: Daniel Borkmann 
Cc: Daniel Lezcano 
Cc: Eduard Zingerman 
Cc: Hao Luo 
Cc: Ian Rogers 
Cc: Ingo Molnar 
Cc: Jiri Kosina 
Cc: John Fastabend 
Cc: Josh Poimboeuf 
Cc: KP Singh 
Cc: Lukasz Luba 
Cc: Mark Rutland 
Cc: Martin KaFai Lau 
Cc: Mykola Lysenko 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Rafael J. Wysocki 
Cc: Shuah Khan 
Cc: Song Liu 
Cc: Stanislav Fomichev 
Cc: Steven Rostedt (VMware) 
Cc: Yonghong Song 
Cc: Zhang Rui 
Link: https://lore.kernel.org/r/20250213-quiet_tools-v3-2-07de4482a581@rivosinc.com
Signed-off-by: Arnaldo Carvalho de Melo

Merge tag 'trace-tools-v6.14' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace

2025-01-26T22:25:58+00:00

Pull rv and tools/rtla updates from Steven Rostedt:

 - Add a test suite to test the tool

   Add a small test suite that can be used to test rtla's basic features
   to at least have something to test when applying changes.

 - Automate manual steps in monitor creation

   While creating a new monitor in RV, besides generating code from
   dot2k, there are a few manual steps which can be tedious and error
   prone, like adding the tracepoints, makefile lines and kconfig, or
   selecting events that start the monitor in the initial state.

   Updates were made to try and automate as much as possible among those
   steps to make creating a new RV monitor much quicker. It is still
   requires to select proper tracepoints, this step is harder to
   automate in a general way and, in several cases, would still need
   user intervention.

 - Have rtla timerlat hist and top set OSNOISE_WORKLOAD flag

   Have both rtla-timerlat-hist and rtla-timerlat-top set
   OSNOISE_WORKLOAD to the proper value ("on" when running with -k,
   "off" when running with -u) every time the option is available
   instead of setting it only when running with -u.

   This prevents rtla timerlat -k from giving no results when
   NO_OSNOISE_WORKLOAD is set, either manually or by an abnormally
   exited earlier run of rtla timerlat -u.

 - Stop rtla timerlat on signal properly when overloaded

   There is an issue where if rtla is run on machines with a high number
   of CPUs (100+), timerlat can generate more samples than rtla is able
   to process via tracefs_iterate_raw_events. This is especially common
   when the interval is set to 100us (rteval and cyclictest default) as
   opposed to the rtla default of 1000us, but also happens with the rtla
   default.

   Currently, this leads to rtla hanging and having to be terminated
   with SIGTERM. SIGINT setting stop_tracing is not enough, since more
   and more events are coming and tracefs_iterate_raw_events never
   exits.

   To fix this: Stop the timerlat tracer on SIGINT/SIGALRM to ensure no
   more events are generated when rtla is supposed to exit.

   Also on receiving SIGINT/SIGALRM twice, abort iteration immediately
   with tracefs_iterate_stop, making rtla exit right away instead of
   waiting for all events to be processed.

 - Account for missed events

   Due to tracefs buffer overflow, it can happen that rtla misses
   events, making the tracing results inaccurate.

   Count both the number of missed events and the total number of
   processed events, and display missed events as well as their
   percentage. The numbers are displayed for both osnoise and timerlat,
   even though for the earlier, missed events are generally not
   expected.

   For hist, the number is displayed at the end of the run; for top, it
   is displayed on each printing of the top table.

 - Changes to make osnoise more robust

   There was a dependency in the code that the first field of the
   osnoise_tool structure was the trace field. If that that ever
   changed, then the code work break. Change the code to encapsulate
   this dependency where the code that uses the structure does not have
   this dependency.

* tag 'trace-tools-v6.14' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (22 commits)
  rtla: Report missed event count
  rtla: Add function to report missed events
  rtla: Count all processed events
  rtla: Count missed trace events
  tools/rtla: Add osnoise_trace_is_off()
  rtla/timerlat_top: Set OSNOISE_WORKLOAD for kernel threads
  rtla/timerlat_hist: Set OSNOISE_WORKLOAD for kernel threads
  rtla/osnoise: Distinguish missing workload option
  rtla/timerlat_top: Abort event processing on second signal
  rtla/timerlat_hist: Abort event processing on second signal
  rtla/timerlat_top: Stop timerlat tracer on signal
  rtla/timerlat_hist: Stop timerlat tracer on signal
  rtla: Add trace_instance_stop
  tools/rtla: Add basic test suite
  verification/dot2k: Implement event type detection
  verification/dot2k: Auto patch current kernel source
  verification/dot2k: Simplify manual steps in monitor creation
  rv: Simplify manual steps in monitor creation
  verification/dot2k: Add support for name and description options
  verification/dot2k: More robust template variables
  ...

rtla: Report missed event count

2025-01-24T18:47:07+00:00

Print how many events were missed by trace buffer overflow in the main
instance at the end of the run (for hist) or during the run (for top).

Cc: John Kacur 
Cc: Luis Goncalves 
Link: https://lore.kernel.org/20250123142339.990300-5-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Tested-by: Gabriele Monaco 
Signed-off-by: Steven Rostedt (Google)

rtla: Add function to report missed events

2025-01-24T18:46:43+00:00

Add osnoise_report_missed_events to be used to report the number
of missed events either during or after an osnoise or timerlat run.
Also, display the percentage of missed events compared to the total
number of received events.

If an unknown number of missed events was reported during the run, the
entire number of missed events is reported as unknown.

Cc: John Kacur 
Cc: Luis Goncalves 
Cc: Gabriele Monaco 
Link: https://lore.kernel.org/20250123142339.990300-4-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla: Count all processed events

2025-01-24T18:46:24+00:00

Add a field processed_events to struct trace_instance and increment it
in collect_registered_events, regardless of whether a handler is
registered for the event.

The purpose is to calculate the percentage of events that were missed
due to tracefs buffer overflow.

Cc: John Kacur 
Cc: Luis Goncalves 
Cc: Gabriele Monaco 
Link: https://lore.kernel.org/20250123142339.990300-3-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla: Count missed trace events

2025-01-24T18:46:16+00:00

Add function collect_missed_events to trace.c to act as a callback for
tracefs_follow_missed_events, summing the number of total missed events
into a new field missing_events of struct trace_instance.

In case record->missed_events is negative, trace->missed_events is set
to UINT64_MAX to signify an unknown number of events was missed.

The callback is activated on initialization of the trace instance.

Cc: John Kacur 
Cc: Luis Goncalves 
Cc: Gabriele Monaco 
Link: https://lore.kernel.org/20250123142339.990300-2-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

tools/rtla: Add osnoise_trace_is_off()

2025-01-24T18:46:11+00:00

All of the users of trace_is_off() passes in &record->trace as the second
parameter, where record is a pointer to a struct osnoise_tool. This record
could be NULL and there is a hidden dependency that the trace field is the
first field to allow &record->trace to work with a NULL record pointer.

In order to make this code a bit more robust, as record shouldn't be
dereferenced if it is NULL, even if the code does work, create a new
function called osnoise_trace_is_off() that takes the pointer to a
struct osnoise_tool as its second parameter. This way it can properly test
if it is NULL before it dereferences it.

The old function trace_is_off() is removed and the function
osnoise_trace_is_off() is added into osnoise.c which is what the
struct osnoise_tool is associated with.

Cc: John Kacur 
Cc: "Luis Claudio R. Goncalves" 
Cc: Eder Zulian 
Cc: Dan Carpenter 
Cc: Tomas Glozar 
Cc: Gabriele Monaco 
Link: https://lore.kernel.org/20250115180055.2136815-1-costa.shul@redhat.com
Signed-off-by: Costa Shulyupin 
Signed-off-by: Steven Rostedt (Google)

rtla/timerlat_top: Set OSNOISE_WORKLOAD for kernel threads

2025-01-24T18:46:03+00:00

When using rtla timerlat with userspace threads (-u or -U), rtla
disables the OSNOISE_WORKLOAD option in
/sys/kernel/tracing/osnoise/options. This option is not re-enabled in a
subsequent run with kernel-space threads, leading to rtla collecting no
results if the previous run exited abnormally:

$ rtla timerlat top -u
^\Quit (core dumped)
$ rtla timerlat top -k -d 1s
                                     Timer Latency
  0 00:00:01   |          IRQ Timer Latency (us)        |         Thread Timer Latency (us)
CPU COUNT      |      cur       min       avg       max |      cur       min       avg       max

The issue persists until OSNOISE_WORKLOAD is set manually by running:
$ echo OSNOISE_WORKLOAD > /sys/kernel/tracing/osnoise/options

Set OSNOISE_WORKLOAD when running rtla with kernel-space threads if
available to fix the issue.

Cc: stable@vger.kernel.org
Cc: John Kacur 
Cc: Luis Goncalves 
Link: https://lore.kernel.org/20250107144823.239782-4-tglozar@redhat.com
Fixes: cdca4f4e5e8e ("rtla/timerlat_top: Add timerlat user-space support")
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla/timerlat_hist: Set OSNOISE_WORKLOAD for kernel threads

2025-01-24T18:45:55+00:00

When using rtla timerlat with userspace threads (-u or -U), rtla
disables the OSNOISE_WORKLOAD option in
/sys/kernel/tracing/osnoise/options. This option is not re-enabled in a
subsequent run with kernel-space threads, leading to rtla collecting no
results if the previous run exited abnormally:

$ rtla timerlat hist -u
^\Quit (core dumped)
$ rtla timerlat hist -k -d 1s
Index
over:
count:
min:
avg:
max:
ALL:        IRQ       Thr       Usr
count:        0         0         0
min:          -         -         -
avg:          -         -         -
max:          -         -         -

The issue persists until OSNOISE_WORKLOAD is set manually by running:
$ echo OSNOISE_WORKLOAD > /sys/kernel/tracing/osnoise/options

Set OSNOISE_WORKLOAD when running rtla with kernel-space threads if
available to fix the issue.

Cc: stable@vger.kernel.org
Cc: John Kacur 
Cc: Luis Goncalves 
Link: https://lore.kernel.org/20250107144823.239782-3-tglozar@redhat.com
Fixes: ed774f7481fa ("rtla/timerlat_hist: Add timerlat user-space support")
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)