linux.git/tools/lib/subcmd, branch v6.5-rc2

Merge tag 'perf-tools-for-v6.5-1-2023-06-28' of git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next

2023-06-30T18:35:41+00:00

Pull perf tools updates from Namhyung Kim:
"Internal cleanup:

- Refactor PMU data management to handle hybrid systems in a generic
way.

Do more work in the lexer so that legacy event types parse more
easily. A side-effect of this is that if a PMU is specified,
scanning sysfs is avoided improving start-up time.

- Fix hybrid metrics, for example, the TopdownL1 works for both
performance and efficiency cores on Intel machines. To support
this, sort and regroup events after parsing.

- Add reference count checking for the 'thread' data structure.

- Lots of fixes for memory leaks in various places thanks to the ASAN
and Ian's refcount checker.

- Reduce the binary size by replacing static variables with local or
dynamically allocated memory.

- Introduce shared_mutex for annotate data to reduce memory
footprint.

- Make filesystem access library functions more thread safe.

Test:

- Organize cpu_map tests into a single suite.

- Add metric value validation test to check if the values are within
correct value ranges.

- Add perf stat stdio output test to check if event and metric names
match.

- Add perf data converter JSON output test.

- Fix a lot of issues reported by shellcheck(1). This is a
preparation to enable shellcheck by default.

- Make the large x86 new instructions test optional at build time
using EXTRA_TESTS=1.

- Add a test for libpfm4 events.

perf script:

- Add 'dsoff' outpuf field to display offset from the DSO.

$ perf script -F comm,pid,event,ip,dsoff
ls 2695501 cycles: 152cc73ef4b5 (/usr/lib/x86_64-linux-gnu/ld-2.31.so+0x1c4b5)
ls 2695501 cycles: ffffffff99045b3e ([kernel.kallsyms])
ls 2695501 cycles: ffffffff9968e107 ([kernel.kallsyms])
ls 2695501 cycles: ffffffffc1f54afb ([kernel.kallsyms])
ls 2695501 cycles: ffffffff9968382f ([kernel.kallsyms])
ls 2695501 cycles: ffffffff99e00094 ([kernel.kallsyms])
ls 2695501 cycles: 152cc718a8d0 (/usr/lib/x86_64-linux-gnu/libselinux.so.1+0x68d0)
ls 2695501 cycles: ffffffff992a6db0 ([kernel.kallsyms])

- Adjust width for large PID/TID values.

perf report:

- Robustify reading addr2line output for srcline by checking sentinel
output before the actual data and by using timeout of 1 second.

- Allow config terms (like 'name=ABC') with breakpoint events.

$ perf record -e mem:0x55feb98dd169:x/name=breakpoint/ -p 19646 -- sleep 1

perf annotate:

- Handle x86 instruction suffix like 'l' in 'movl' generally.

- Parse instruction operands properly even with a whitespace. This is
needed for llvm-objdump output.

- Support RISC-V binutils lookup using the triplet prefixes.

- Add '<' and '>' key to navigate to prev/next symbols in TUI.

- Fix instruction association and parsing for LoongArch.

perf stat:

- Add --per-cache aggregation option, optionally specify a cache
level like `--per-cache=L2`.

$ sudo perf stat --per-cache -a -e ls_dmnd_fills_from_sys.ext_cache_remote --\
taskset -c 0-15,64-79,128-143,192-207\
perf bench sched messaging -p -t -l 100000 -g 8

# Running 'sched/messaging' benchmark:
# 20 sender and receiver threads per group
# 8 groups == 320 threads run

Total time: 7.648 [sec]

Performance counter stats for 'system wide':

S0-D0-L3-ID0 16 17,145,912 ls_dmnd_fills_from_sys.ext_cache_remote
S0-D0-L3-ID8 16 14,977,628 ls_dmnd_fills_from_sys.ext_cache_remote
S0-D0-L3-ID16 16 262,539 ls_dmnd_fills_from_sys.ext_cache_remote
S0-D0-L3-ID24 16 3,140 ls_dmnd_fills_from_sys.ext_cache_remote
S0-D0-L3-ID32 16 27,403 ls_dmnd_fills_from_sys.ext_cache_remote
S0-D0-L3-ID40 16 17,026 ls_dmnd_fills_from_sys.ext_cache_remote
S0-D0-L3-ID48 16 7,292 ls_dmnd_fills_from_sys.ext_cache_remote
S0-D0-L3-ID56 16 2,464 ls_dmnd_fills_from_sys.ext_cache_remote
S1-D1-L3-ID64 16 22,489,306 ls_dmnd_fills_from_sys.ext_cache_remote
S1-D1-L3-ID72 16 21,455,257 ls_dmnd_fills_from_sys.ext_cache_remote
S1-D1-L3-ID80 16 11,619 ls_dmnd_fills_from_sys.ext_cache_remote
S1-D1-L3-ID88 16 30,978 ls_dmnd_fills_from_sys.ext_cache_remote
S1-D1-L3-ID96 16 37,628 ls_dmnd_fills_from_sys.ext_cache_remote
S1-D1-L3-ID104 16 13,594 ls_dmnd_fills_from_sys.ext_cache_remote
S1-D1-L3-ID112 16 10,164 ls_dmnd_fills_from_sys.ext_cache_remote
S1-D1-L3-ID120 16 11,259 ls_dmnd_fills_from_sys.ext_cache_remote

7.779171484 seconds time elapsed

- Change default (no event/metric) formatting for default metrics so
that events are hidden and the metric and group appear.

Performance counter stats for 'ls /':

1.85 msec task-clock # 0.594 CPUs utilized
0 context-switches # 0.000 /sec
0 cpu-migrations # 0.000 /sec
97 page-faults # 52.517 K/sec
2,187,173 cycles # 1.184 GHz
2,474,459 instructions # 1.13 insn per cycle
531,584 branches # 287.805 M/sec
13,626 branch-misses # 2.56% of all branches
TopdownL1 # 23.5 % tma_backend_bound
# 11.5 % tma_bad_speculation
# 39.1 % tma_frontend_bound
# 25.9 % tma_retiring

- Allow --cputype option to have any PMU name (not just hybrid).

- Fix output value not to added when it runs multiple times with -r
option.

perf list:

- Show metricgroup description from JSON file called
metricgroups.json.

- Allow 'pfm' argument to list only libpfm4 events and check each
event is supported before showing it.

JSON vendor events:

- Avoid event grouping using "NO_GROUP_EVENTS" constraints. The
topdown events are correctly grouped even if no group exists.

- Add "Default" metric group to print it in the default output. And
use "DefaultMetricgroupName" to indicate the real metric group
name.

- Add AmpereOne core PMU events.

Misc:

- Define man page date correctly.

- Track exception level properly on ARM CoreSight ETM.

- Allow anonymous struct, union or enum when retrieving type names
from DWARF.

- Fix incorrect filename when calling `perf inject --jit`.

- Handle PLT size correctly on LoongArch"

* tag 'perf-tools-for-v6.5-1-2023-06-28' of git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next: (269 commits)
perf test: Skip metrics w/o event name in stat STD output linter
perf test: Reorder event name checks in stat STD output linter
perf pmu: Remove a hard coded cpu PMU assumption
perf pmus: Add notion of default PMU for JSON events
perf unwind: Fix map reference counts
perf test: Set PERF_EXEC_PATH for script execution
perf script: Initialize buffer for regs_map()
perf tests: Fix test_arm_callgraph_fp variable expansion
perf symbol: Add LoongArch case in get_plt_sizes()
perf test: Remove x permission from lib/stat_output.sh
perf test: Rerun failed metrics with longer workload
perf test: Add skip list for metrics known would fail
perf test: Add metric value validation test
perf jit: Fix incorrect file name in DWARF line table
perf annotate: Fix instruction association and parsing for LoongArch
perf annotation: Switch lock from a mutex to a sharded_mutex
perf sharded_mutex: Introduce sharded_mutex
tools: Fix incorrect calculation of object size by sizeof
perf subcmd: Fix missing check for return value of malloc() in add_cmdname()
perf parse-events: Remove unneeded semicolon
...

perf subcmd: Fix missing check for return value of malloc() in add_cmdname()

2023-06-20T23:55:35+00:00

The malloc() function may return NULL when it fails,
which may cause null pointer deference in add_cmdname(),
add Null check for return value of malloc().

Found by our static analysis tool.

Signed-off-by: Chenyuan Mi 
Acked-by: Ian Rogers 
Cc: Leo Yan 
Cc: Arnaldo Carvalho de Melo 
Cc: linux-kernel@vger.kernel.org
Link: https://lore.kernel.org/r/20230614150118.115208-1-cymi20@fudan.edu.cn
Signed-off-by: Namhyung Kim

lib subcmd: Avoid memory leak in exclude_cmds

2023-06-12T21:18:13+00:00

exclude_cmds will shorten the cmds names array, before doing so free the
removed entry.

Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: André Almeida 
Cc: Darren Hart 
Cc: Davidlohr Bueso 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Link: https://lore.kernel.org/r/20230611233610.953456-1-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

libsubcmd: Avoid two path statics, removing 8192 bytes from .bss

2023-05-28T13:25:55+00:00

Use a single stack allocated buffer and avoid 8,192 bytes in .bss.

Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Andi Kleen 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: K Prateek Nayak 
Cc: Kan Liang 
Cc: Leo Yan 
Cc: Mark Rutland 
Cc: Masami Hiramatsu 
Cc: Namhyung Kim 
Cc: Paolo Bonzini 
Cc: Peter Zijlstra 
Cc: Ravi Bangoria 
Cc: Ross Zwisler 
Cc: Sean Christopherson 
Cc: Steven Rostedt (VMware) 
Cc: Tiezhu Yang 
Cc: Yang Jihong 
Link: https://lore.kernel.org/r/20230526183401.2326121-17-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

tools/lib/subcmd: Replace NORETURN usage with __noreturn

2023-05-16T13:31:54+00:00

NORETURN is redundant with __noreturn, just use the latter.

Reviewed-by: Miroslav Benes 
Link: https://lore.kernel.org/r/c7c83d1e6b3d2b0c3e65dd3790c22c772d3b2527.1681853186.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf

tools lib subcmd: Add dependency test to install_headers

2022-12-14T14:16:12+00:00

Compute the headers to be installed from their source headers and make
each have its own build target to install it. Using dependencies
avoids headers being reinstalled and getting a new timestamp which
then causes files that depend on the header to be rebuilt.

Signed-off-by: Ian Rogers 
Cc: Alexander Shishkin 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Josh Poimboeuf 
Cc: Mark Rutland 
Cc: Masahiro Yamada 
Cc: Namhyung Kim 
Cc: Nathan Chancellor 
Cc: Nick Desaulniers 
Cc: Nicolas Schier 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Cc: Tom Rix 
Cc: bpf@vger.kernel.org
Cc: llvm@lists.linux.dev
Link: https://lore.kernel.org/r/20221202045743.2639466-4-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

tools lib subcmd: Make install_headers clearer

2022-11-20T14:32:23+00:00

Add libsubcmd to the name so that this install_headers build appears
different to similar targets in different libraries.

Signed-off-by: Ian Rogers 
Cc: Alexander Shishkin 
Cc: Alexei Starovoitov 
Cc: Andrii Nakryiko 
Cc: Daniel Borkmann 
Cc: Hao Luo 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: John Fastabend 
Cc: KP Singh 
Cc: Mark Rutland 
Cc: Martin KaFai Lau 
Cc: Masahiro Yamada 
Cc: Namhyung Kim 
Cc: Nicolas Schier 
Cc: Peter Zijlstra 
Cc: Song Liu 
Cc: Stanislav Fomichev 
Cc: Stephane Eranian 
Cc: Yonghong Song 
Cc: bpf@vger.kernel.org
Link: https://lore.kernel.org/r/20221117004356.279422-6-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

tools lib subcmd: Add install target

2022-11-16T15:17:15+00:00

This allows libsubcmd to be installed as a dependency.

Signed-off-by: Ian Rogers 
Acked-by: Namhyung Kim 
Cc: Alexander Shishkin 
Cc: Andrii Nakryiko 
Cc: Jiri Olsa 
Cc: Mark Rutland 
Cc: Masahiro Yamada 
Cc: Nick Desaulniers 
Cc: Nicolas Schier 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Cc: bpf@vger.kernel.org
Link: http://lore.kernel.org/lkml/20221109184914.1357295-3-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf subcmd: Set environment variable "PREFIX"

2022-10-04T11:55:23+00:00

Set environment variable "PREFIX", it will be used by invoked shell
script, e.g. the shell script uses it to find lib paths.

Signed-off-by: Leo Yan 
Cc: Alexander Shishkin 
Cc: Ian Rogers 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Link: https://lore.kernel.org/r/20220925025835.70364-2-leo.yan@linaro.org
Signed-off-by: Arnaldo Carvalho de Melo

libsubcmd: Fix OPTION_GROUP sorting

2022-04-22T10:32:01+00:00

The OPTION_GROUP option type is a way of grouping certain options
together in the printed usage text.  It happens to be completely broken,
thanks to the fact that the subcmd option sorting just sorts everything,
without regard for grouping.  Luckily, nobody uses this option anyway,
though that will change shortly.

Fix it by sorting each group individually.

Signed-off-by: Josh Poimboeuf 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Miroslav Benes 
Link: https://lkml.kernel.org/r/e167ea3a11e2a9800eb062c1fd0f13e9cd05140c.1650300597.git.jpoimboe@redhat.com