linux.git/tools/tracing/rtla/src/timerlat_hist.c, branch for-next

rtla/timerlat: Fix histogram ALL for zero samples

2024-12-27T16:21:46+00:00

rtla timerlat hist currently computers the minimum, maximum and average
latency even in cases when there are zero samples. This leads to
nonsensical values being calculated for maximum and minimum, and to
divide by zero for average.

A similar bug is fixed by 01b05fc0e5f3 ("rtla/timerlat: Fix histogram
report when a cpu count is 0") but the bug still remains for printing
the sum over all CPUs in timerlat_print_stats_all.

The issue can be reproduced with this command:

$ rtla timerlat hist -U -d 1s
Index
over:
count:
min:
avg:
max:
Floating point exception (core dumped)

(There are always no samples with -U unless the user workload is
created.)

Fix the bug by omitting max/min/avg when sample count is zero,
displaying a dash instead, just like we already do for the individual
CPUs. The logic is moved into a new function called
format_summary_value, which is used for both the individual CPUs
and for the overall summary.

Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/20241127134130.51171-1-tglozar@redhat.com
Fixes: 1462501c7a8 ("rtla/timerlat: Add a summary for hist mode")
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla/timerlat: Do not set params->user_workload with -U

2024-11-19T13:56:59+00:00

Since commit fb9e90a67ee9 ("rtla/timerlat: Make user-space threads
the default"), rtla-timerlat has been defaulting to
params->user_workload if neither that or params->kernel_workload is set.
This has unintentionally made -U, which sets only params->user_hist/top
but not params->user_workload, to behave like -u unless -k is set,
preventing the user from running a custom workload.

Example:
$ rtla timerlat hist -U -c 0 &
[1] 7413
$ python sample/timerlat_load.py 0
Error opening timerlat fd, did you run timerlat -U?
$ ps | grep timerlatu
7415 pts/4    00:00:00 timerlatu/0

Fix the issue by checking for params->user_top/hist instead of
params->user_workload when setting default thread mode.

Link: https://lore.kernel.org/20241021123140.14652-1-tglozar@redhat.com
Fixes: fb9e90a67ee9 ("rtla/timerlat: Make user-space threads the default")
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla/timerlat: Add --deepest-idle-state for hist

2024-10-17T21:13:15+00:00

Support limiting deepest idle state also for timerlat-hist.

Link: https://lore.kernel.org/20241017140914.3200454-6-tglozar@redhat.com
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla/timerlat: Make timerlat_hist_cpu->*_count unsigned long long

2024-10-11T17:09:57+00:00

Do the same fix as in previous commit also for timerlat-hist.

Link: https://lore.kernel.org/20241011121015.2868751-2-tglozar@redhat.com
Reported-by: Attila Fazekas 
Signed-off-by: Tomas Glozar 
Signed-off-by: Steven Rostedt (Google)

rtla: Fix consistency in getopt_long for timerlat_hist

2024-10-04T23:33:38+00:00

Commit e9a4062e1527 ("rtla: Add --trace-buffer-size option") adds a new
long option to rtla utilities, but among all affected files,
timerlat_hist misses a trailing `:` in the corresponding short option
inside the getopt string (e.g. `\3:`). This patch propagates the `:`.

Although this change is not functionally required, it improves
consistency and slightly reduces the likelihood a future change would
introduce a problem.

Cc: John Kacur 
Cc: Luis Goncalves 
Cc: Tomas Glozar 
Link: https://lore.kernel.org/20240926143417.54039-1-gmonaco@redhat.com
Signed-off-by: Gabriele Monaco 
Signed-off-by: Steven Rostedt (Google)

rtla: Fix -t\--trace[=file]

2024-05-16T14:49:10+00:00

The -t option has an optional argument.
The usual case is for a short option to be specified without an '='
and for the long version to be specified with an '='

Various forms of this do not work as expected.
For example:
rtla timerlat hist -T50 -tfile.txt
will result in a truncated file name of "ile.txt"

Another example is that the long form without the '=' will result in the
default file name instead of the requested file name.

This patch properly parses the optional argument with and without '='
and with and without spaces for the short form.

This patch was also tested using -t and --trace without providing a file
name both as the last requested option and with a following long and
short option.

For example:

  rtla timerlat hist -T50 -t -u
  rtla timerlat hist -T50 --trace -u

This fix is applied to both timerlat top and hist
and to osnoise top and hist.

Here is the full testing for rtla timerlat hist.
Before applying the patch

  rtla timerlat hist -T50 -t=file.txt
    Works as expected, "file.txt"

  rtla timerlat hist -T50 -tfile.txt
    Truncated file name "ile.txt"

  rtla timerlat hist -T50 -t file.txt
    Default file name instead of file.txt

  rtla timerlat hist -T50 --trace=file.txt
    Truncated file name "ile.txt"

  rtla timerlat hist -T50 --trace file.txt
    Default file name "timerlat_trace.txt" instead of "file.txt"

After applying the patch:

  rtla timerlat hist -T50 -t=file.txt
    Works as expected, "file.txt"

  rtla timerlat hist -T50 -tfile.txt
    Works as expected, "file.txt"

  rtla timerlat hist -T50 -t file.txt
    Works as expected, "file.txt"

  rtla timerlat hist -T50 --trace=file.txt
    Works as expected, "file.txt"

  rtla timerlat hist -T50 --trace file.txt
    Works as expected, "file.txt"

In addition the following tests were performed to make sure that
the default file name worked as expected including with trailing
options.

  rtla timerlat hist -T50 -t
    Works as expected "timerlat_trace.txt"

  rtla timerlat hist -T50 --trace
    Works as expected "timerlat_trace.txt"

  rtla timerlat hist -T50 -t -u
    Works as expected "timerlat_trace.txt"

  rtla timerlat hist -T50 --trace -u
    Works as expected "timerlat_trace.txt"

Link: https://lkml.kernel.org/r/20240515183024.59985-1-jkacur@redhat.com

Cc: Daniel Bristot de Oliveria 
Signed-off-by: John Kacur 
Signed-off-by: Daniel Bristot de Oliveira

rtla/timerlat: Fix histogram report when a cpu count is 0

2024-05-16T14:45:40+00:00

On short runs it is possible to get no samples on a cpu, like this:

  # rtla timerlat hist -u -T50

  Index   IRQ-001   Thr-001   Usr-001   IRQ-002   Thr-002   Usr-002
  2             1         0         0         0         0         0
  33            0         1         0         0         0         0
  36            0         0         1         0         0         0
  49            0         0         0         1         0         0
  52            0         0         0         0         1         0
  over:         0         0         0         0         0         0
  count:        1         1         1         1         1         0
  min:          2        33        36        49        52 18446744073709551615
  avg:          2        33        36        49        52         -
  max:          2        33        36        49        52         0
  rtla timerlat hit stop tracing
    IRQ handler delay:		(exit from idle)	    48.21 us (91.09 %)
    IRQ latency:						    49.11 us
    Timerlat IRQ duration:				     2.17 us (4.09 %)
    Blocking thread:					     1.01 us (1.90 %)
  	               swapper/2:0        		     1.01 us
  ------------------------------------------------------------------------
    Thread latency:					    52.93 us (100%)

  Max timerlat IRQ latency from idle: 49.11 us in cpu 2

Note, the value 18446744073709551615 is the same as ~0.

Fix this by reporting no results for the min, avg and max if the count
is 0.

Link: https://lkml.kernel.org/r/20240510190318.44295-1-jkacur@redhat.com

Cc: stable@vger.kernel.org
Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode")
Suggested-by: Daniel Bristot de Oliveria 
Signed-off-by: John Kacur 
Signed-off-by: Daniel Bristot de Oliveira

rtla: Add --trace-buffer-size option

2024-05-16T14:44:34+00:00

Add the option allow the users to set a different buffer size for the
trace. For example, in large systems, the user might be interested on
reducing the trace buffer to avoid large tracing files.

The buffer size is specified in kB, and it is only affecting
the tracing instance.

The function trace_set_buffer_size() appears on libtracefs v1.6,
so increase the minimum required version on Makefile.config.

Link: https://lkml.kernel.org/r/e7c9ca5b3865f28e131a49ec3b984fadf2d056c6.1715860611.git.bristot@kernel.org

Cc: Jonathan Corbet 
Cc: Juri Lelli 
Cc: John Kacur 
Signed-off-by: Daniel Bristot de Oliveira

rtla/timerlat: Make user-space threads the default

2024-05-15T13:13:57+00:00

After ther -u addition, most of the known users are setting it. And
it makes sense, as it adds more information, and inherits the default
setup for the threads - e.g., cgroups configs.

Thus, if the user-space interface is available, enable -u. Otherwise,
use the in-kernel thread.

Add the -k option to allow the user to request kernel-threads.

Link: https://lkml.kernel.org/r/9241d3089de4091b124f780ed832a0e6646cadaa.1713968967.git.bristot@kernel.org

Cc: Jonathan Corbet 
Cc: Juri Lelli 
Signed-off-by: Daniel Bristot de Oliveira

rtla: Add the --warm-up option

2024-05-15T13:13:57+00:00

On many cases, the results right after the startup are different
from the rest of the execution, biasing the results. For example,
on osnoise, the scheduler might take some time to adapt to the new
busy-loop workload.

Add the --warm-up  option, adding a warm-up phase (in
seconds) where the workload is set, but the results are discarded.

Link: https://lkml.kernel.org/r/e682d5ce5af90f123bd13220f63d5c3d118a92be.1713968967.git.bristot@kernel.org

Cc: Jonathan Corbet 
Cc: Juri Lelli 
Signed-off-by: Daniel Bristot de Oliveira