linux-stable.git/tools/tracing/rtla/src/timerlat_hist.c, branch v6.9.2

tools/rtla: Add -U/--user-load option to timerlat

2024-03-20T04:39:06+00:00

The timerlat tracer provides an interface for any application to wait
for the timerlat's periodic wakeup. Currently, rtla timerlat uses it
to dispatch its user-space workload (-u option).

But as the tracer interface is generic, rtla timerlat can also be used
to monitor any workload that uses it. For example, a user might
place their own workload to wait on the tracer interface, and
monitor the results with rtla timerlat.

Add the -U option to rtla timerlat top and hist. With this option, rtla
timerlat will not dispatch its workload but only setting up the
system, waiting for a user to dispatch its workload.

The sample code in this patch is an example of python application
that loops in the timerlat tracer fd.

To use it, dispatch:

 # rtla timerlat -U

In a terminal, then run the python program on another terminal,
specifying the CPU to run it. For example, setting on CPU 1:

 #./timerlat_load.py 1

Then rtla timerlat will start printing the statistics of the
./timerlat_load.py app.

An interesting point is that the "Ret user Timer Latency" value
is the overall response time of the load. The sample load does
a memory copy to exemplify that.

The stop tracing options on rtla timerlat works in this setup
as well, including auto analysis.

Link: https://lkml.kernel.org/r/36e6bcf18fe15c7601048fd4c65aeb193c502cc8.1707229706.git.bristot@kernel.org

Cc: Jonathan Corbet 
Cc: Masami Hiramatsu 
Signed-off-by: Daniel Bristot de Oliveira

tools/rtla: Exit with EXIT_SUCCESS when help is invoked

2024-02-12T09:59:09+00:00

Fix rtla so that the following commands exit with 0 when help is invoked

rtla osnoise top -h
rtla osnoise hist -h
rtla timerlat top -h
rtla timerlat hist -h

Link: https://lore.kernel.org/linux-trace-devel/20240203001607.69703-1-jkacur@redhat.com

Cc: stable@vger.kernel.org
Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode")
Signed-off-by: John Kacur 
Signed-off-by: Daniel Bristot de Oliveira

tools/rtla: Fix uninitialized bucket/data->bucket_size warning

2024-02-12T08:56:22+00:00

When compiling rtla with clang, I am getting the following warnings:

$ make HOSTCC=clang CC=clang LLVM_IAS=1

[..]
clang -O -g -DVERSION=\"6.8.0-rc3\" -flto=auto -fexceptions
	-fstack-protector-strong -fasynchronous-unwind-tables
	-fstack-clash-protection  -Wall -Werror=format-security
	-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS
	$(pkg-config --cflags libtracefs)
	-c -o src/osnoise_hist.o src/osnoise_hist.c
src/osnoise_hist.c:138:6: warning: variable 'bucket' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
  138 |         if (data->bucket_size)
      |             ^~~~~~~~~~~~~~~~~
src/osnoise_hist.c:149:6: note: uninitialized use occurs here
  149 |         if (bucket < entries)
      |             ^~~~~~
src/osnoise_hist.c:138:2: note: remove the 'if' if its condition is always true
  138 |         if (data->bucket_size)
      |         ^~~~~~~~~~~~~~~~~~~~~~
  139 |                 bucket = duration / data->bucket_size;
src/osnoise_hist.c:132:12: note: initialize the variable 'bucket' to silence this warning
  132 |         int bucket;
      |                   ^
      |                    = 0
1 warning generated.

[...]

clang -O -g -DVERSION=\"6.8.0-rc3\" -flto=auto -fexceptions
	-fstack-protector-strong -fasynchronous-unwind-tables
	-fstack-clash-protection  -Wall -Werror=format-security
	-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS
	$(pkg-config --cflags libtracefs)
	-c -o src/timerlat_hist.o src/timerlat_hist.c
src/timerlat_hist.c:181:6: warning: variable 'bucket' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
  181 |         if (data->bucket_size)
      |             ^~~~~~~~~~~~~~~~~
src/timerlat_hist.c:204:6: note: uninitialized use occurs here
  204 |         if (bucket < entries)
      |             ^~~~~~
src/timerlat_hist.c:181:2: note: remove the 'if' if its condition is always true
  181 |         if (data->bucket_size)
      |         ^~~~~~~~~~~~~~~~~~~~~~
  182 |                 bucket = latency / data->bucket_size;
src/timerlat_hist.c:175:12: note: initialize the variable 'bucket' to silence this warning
  175 |         int bucket;
      |                   ^
      |                    = 0
1 warning generated.

This is a legit warning, but data->bucket_size is always > 0 (see
timerlat_hist_parse_args()), so the if is not necessary.

Remove the unneeded if (data->bucket_size) to avoid the warning.

Link: https://lkml.kernel.org/r/6e1b1665cd99042ae705b3e0fc410858c4c42346.1707217097.git.bristot@kernel.org

Cc: stable@vger.kernel.org
Cc: Masami Hiramatsu 
Cc: Nathan Chancellor 
Cc: Nick Desaulniers 
Cc: Bill Wendling 
Cc: Justin Stitt 
Cc: Donald Zickus 
Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode")
Fixes: 829a6c0b5698 ("rtla/osnoise: Add the hist mode")
Signed-off-by: Daniel Bristot de Oliveira

rtla/timerlat_hist: Add timerlat user-space support

2023-06-13T20:41:14+00:00

Add the support for running timerlat threads in user-space. In this
mode, enabled with -u/--user-threads, timerlat dispatches user-space
processes that will loop in the timerlat_fd, measuring the overhead
for going to user-space and then returning to the kernel - in addition
to the existing measurements.

Here is one example of the tool's output with -u enabled:

  $ sudo timerlat  hist -u -c 1-3 -d 600
  # RTLA timerlat histogram
  # Time unit is microseconds (us)
  # Duration:   0 00:10:01
  Index   IRQ-001   Thr-001   Usr-001   IRQ-002   Thr-002   Usr-002   IRQ-003   Thr-003   Usr-003
  0        477555         0         0    425287         0         0    474357         0         0
  1        122385      7998         0    174616      1921         0    125412      3138         0
  2            47    587376    492150        89    594717    447830       147    593463    454872
  3            11      2549    101930         7      2682    145580        64      2530    138680
  4             3      1954      2833         1       463      4917        11       548      4656
  5             0        60      1037         0       138      1117         6       179      1130
  6             0        26      1837         0        38       277         1        76       339
  7             0        15       143         0        28       147         2        37       156
  8             0        10        23         0        11        75         0        12        80
  9             0         7        17         0         0        26         0        11        42
  10            0         2        11         0         0        18         0         2        20
  11            0         0         7         0         1         8         0         2        12
  12            0         0         6         0         1         4         0         2         8
  13            0         1         3         0         0         0         0         0         1
  14            0         1         0         0         0         1         0         0         2
  15            0         1         0         0         0         0         0         0         2
  16            0         1         2         0         0         0         0         0         0
  17            0         0         1         0         0         0         0         0         0
  19            0         0         1         0         0         0         0         0         0
  over:         0         0         0         0         0         0         0         0         0
  count:   600001    600001    600001    600000    600000    600000    600000    600000    600000
  min:          0         1         2         0         1         2         0         1         2
  avg:          0         1         2         0         2         2         0         2         2
  max:          4        16        19         4        12        14         7        12        15

The tuning setup like -p or -C work for the user-space threads as well.

Link: https://lkml.kernel.org/r/b6a042d55003c4a67ff7dce28d96044b7044f00d.1686066600.git.bristot@kernel.org

Cc: William White 
Cc: Jonathan Corbet 
Tested-by: Juri Lelli 
Signed-off-by: Daniel Bristot de Oliveira 
Signed-off-by: Steven Rostedt (Google)

rtla: Start the tracers after creating all instances

2023-06-13T20:37:05+00:00

Group all start tracing after finishing creating all instances.

The tracing instance starts first for the case of hitting a stop
tracing while enabling other instances. The trace instance is the
one with most valuable information.

Link: https://lkml.kernel.org/r/67da7a703a56f75d7cd46568525145a65501a7e8.1686066600.git.bristot@kernel.org

Cc: William White 
Cc: Jonathan Corbet 
Tested-by: Juri Lelli 
Signed-off-by: Daniel Bristot de Oliveira 
Signed-off-by: Steven Rostedt (Google)

rtla/timerlat_hist: Add auto-analysis support

2023-06-13T20:35:46+00:00

Add auto-analysis to timerlat hist, including the --no-aa option to
reduce overhead and --dump-task. --aa-only was not added as it is
already on timerlat top.

Link: https://lkml.kernel.org/r/c2693f47ee83e659a7723fed8035f5d2534f528e.1686066600.git.bristot@kernel.org

Cc: William White 
Cc: Jonathan Corbet 
Tested-by: Juri Lelli 
Signed-off-by: Daniel Bristot de Oliveira 
Signed-off-by: Steven Rostedt (Google)

rtla: Automatically move rtla to a house-keeping cpu

2023-06-13T20:30:13+00:00

When the user sets -c  try to move rtla out of the ,
even without an -H option. This is useful to avoid having rtla
interfering with the workload.

This works by removing  from rtla's current affinity.

If rtla fails to move itself away it is not that of a problem as this
is an automatic measure.

Link: https://lkml.kernel.org/r/c54304d90c777310fb85a3e658d1449173759aab.1686066600.git.bristot@kernel.org

Cc: William White 
Cc: Jonathan Corbet 
Tested-by: Juri Lelli 
Signed-off-by: Daniel Bristot de Oliveira 
Signed-off-by: Steven Rostedt (Google)

rtla: Change monitored_cpus from char * to cpu_set_t

2023-06-13T20:28:56+00:00

Use a cpumask instead of a char *, reducing memory footprint and code.

No functional change, and in preparation for auto house-keeping.

Link: https://lkml.kernel.org/r/54c46293261d13cb1042d0314486539eeb45fe5d.1686066600.git.bristot@kernel.org

Cc: William White 
Cc: Jonathan Corbet 
Tested-by: Juri Lelli 
Signed-off-by: Daniel Bristot de Oliveira 
Signed-off-by: Steven Rostedt (Google)

rtla: Add --house-keeping option

2023-06-13T20:26:03+00:00

To avoid having rtla interfering with the measurement threads, add an
option for the user to set the CPUs in which rtla should run. For
instance:

  # rtla timerlat top -H 0 -c 1-7

Will place rtla in the CPU 0, while running the measurement threads in
the CPU 1-7.

Link: https://lkml.kernel.org/r/6a6c78a579a96ba8b02ae67ee1e0ba2cb5e03c4a.1686066600.git.bristot@kernel.org

Cc: William White 
Cc: Jonathan Corbet 
Tested-by: Juri Lelli 
Suggested-by: Juri Lelli 
Signed-off-by: Daniel Bristot de Oliveira 
Signed-off-by: Steven Rostedt (Google)

rtla: Add -C cgroup support

2023-06-13T20:25:13+00:00

The -C option sets a cgroup to the tracer's threads. If the -C option is
passed without arguments, the tracer's thread will inherit rtla's
cgroup. Otherwise, the threads will be placed on the cgroup passed
to the option.

Link: https://lkml.kernel.org/r/cb051477331d292f17c08bf1d66f0e0384bbe5a5.1686066600.git.bristot@kernel.org

Cc: William White 
Cc: Jonathan Corbet 
Tested-by: Juri Lelli 
Signed-off-by: Daniel Bristot de Oliveira 
Signed-off-by: Steven Rostedt (Google)