linux.git/tools/perf/util/header.c, branch v6.5

perf header: Avoid out-of-bounds read

2023-06-12T18:57:54+00:00

intel-pt tests were failing:

  -- Test virtual LBR ---
  Linux
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.126 MB /tmp/perf-test-intel-pt-sh.FW57CXnCqQ/test-perf.data ]
  Failed with virtual lbr
  ...
  ```

  The root cause is an out-of-bounds read in header (where maxbrstack.py
  is from test_intel_pt.sh):
  ```
  $ perf --no-pager script --itrace=L -s maxbrstack.py
  =================================================================
  ==3907930==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6020000095a8 at pc 0x563c26c840bb bp 0x7fff43582710 sp 0x7fff43582708
  READ of size 4 at 0x6020000095a8 thread T0
      #0 0x563c26c840ba in process_group_desc util/header.c:2847
      #1 0x563c26c8bc78 in perf_file_section__process util/header.c:4037
      #2 0x563c26c8aa9b in perf_header__process_sections util/header.c:3813
      #3 0x563c26c8d028 in perf_session__read_header util/header.c:4286
      #4 0x563c26cbab29 in perf_session__open util/session.c:113
      #5 0x563c26cbb3d0 in __perf_session__new util/session.c:221
      #6 0x563c26aacb14 in perf_session__new util/session.h:73
      #7 0x563c26acf7f1 in cmd_script tools/perf/builtin-script.c:4212
      #8 0x563c26bb58ff in run_builtin tools/perf/perf.c:323
      #9 0x563c26bb5e70 in handle_internal_command tools/perf/perf.c:377
      #10 0x563c26bb6238 in run_argv tools/perf/perf.c:421
      #11 0x563c26bb67a0 in main tools/perf/perf.c:537
      #12 0x7f34bde46189 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
      #13 0x7f34bde46244 in __libc_start_main_impl ../csu/libc-start.c:381
      #14 0x563c26a33390 in _start (/tmp/perf/perf+0x1eb390)

  0x6020000095a8 is located 8 bytes to the right of 16-byte region [0x602000009590,0x6020000095a0)
  allocated by thread T0 here:
      #0 0x7f34beeb83b7 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77
      #1 0x563c26c83df8 in process_group_desc util/header.c:2824
      #2 0x563c26c8bc78 in perf_file_section__process util/header.c:4037
      #3 0x563c26c8aa9b in perf_header__process_sections util/header.c:3813
      #4 0x563c26c8d028 in perf_session__read_header util/header.c:4286
      #5 0x563c26cbab29 in perf_session__open util/session.c:113
      #6 0x563c26cbb3d0 in __perf_session__new util/session.c:221
      #7 0x563c26aacb14 in perf_session__new util/session.h:73
      #8 0x563c26acf7f1 in cmd_script tools/perf/builtin-script.c:4212
      #9 0x563c26bb58ff in run_builtin tools/perf/perf.c:323
      #10 0x563c26bb5e70 in handle_internal_command tools/perf/perf.c:377
      #11 0x563c26bb6238 in run_argv tools/perf/perf.c:421
      #12 0x563c26bb67a0 in main tools/perf/perf.c:537
      #13 0x7f34bde46189 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
  ```

Avoid the out-of-bounds read checking for the leader. Leave the 'nr'
check intact as nr will be 0 or the counting down and evsel be a group
member.

Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Ali Saidi 
Cc: Andi Kleen 
Cc: Athira Rajeev 
Cc: Brian Robbins 
Cc: Changbin Du 
Cc: Dmitrii Dolgov <9erthalion6@gmail.com>
Cc: Fangrui Song 
Cc: German Gomez 
Cc: Ian Rogers 
Cc: Ingo Molnar 
Cc: Ivan Babrou 
Cc: James Clark 
Cc: Jing Zhang 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: K Prateek Nayak 
Cc: Kan Liang 
Cc: Leo Yan 
Cc: Liam Howlett 
Cc: Mark Rutland 
Cc: Miguel Ojeda 
Cc: Mike Leach 
Cc: Namhyung Kim 
Cc: Naveen N. Rao 
Cc: Peter Zijlstra 
Cc: Ravi Bangoria 
Cc: Sean Christopherson 
Cc: Steinar H. Gunderson 
Cc: Suzuki Poulouse 
Cc: Wenyu Liu 
Cc: Will Deacon 
Cc: Yang Jihong 
Cc: Ye Xingchen 
Cc: Yuan Can 
Cc: linux-arm-kernel@lists.infradead.org
Link: https://lore.kernel.org/lkml/20230608232823.4027869-24-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf header: Ensure bitmaps are freed

2023-06-12T18:57:53+00:00

memory_node bitmaps need a bitmap_free to avoid memory leaks. Caught
by leak sanitizer.

Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Ali Saidi 
Cc: Andi Kleen 
Cc: Athira Rajeev 
Cc: Brian Robbins 
Cc: Changbin Du 
Cc: Dmitrii Dolgov <9erthalion6@gmail.com>
Cc: Fangrui Song 
Cc: German Gomez 
Cc: Ingo Molnar 
Cc: Ivan Babrou 
Cc: James Clark 
Cc: Jing Zhang 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: K Prateek Nayak 
Cc: Kan Liang 
Cc: Leo Yan 
Cc: Liam Howlett 
Cc: Mark Rutland 
Cc: Miguel Ojeda 
Cc: Mike Leach 
Cc: Namhyung Kim 
Cc: Naveen N. Rao 
Cc: Peter Zijlstra 
Cc: Ravi Bangoria 
Cc: Sean Christopherson 
Cc: Steinar H. Gunderson 
Cc: Suzuki Poulouse 
Cc: Wenyu Liu 
Cc: Will Deacon 
Cc: Yang Jihong 
Cc: Ye Xingchen 
Cc: Yuan Can 
Cc: coresight@lists.linaro.org
Cc: linux-arm-kernel@lists.infradead.org
Link: https://lore.kernel.org/r/20230608232823.4027869-11-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf header: Make nodes dynamic in write_mem_topology()

2023-05-28T13:20:38+00:00

Avoid a large static array, dynamically allocate the nodes avoiding a
hard coded limited as well.

Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Andi Kleen 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: K Prateek Nayak 
Cc: Kan Liang 
Cc: Leo Yan 
Cc: Mark Rutland 
Cc: Masami Hiramatsu 
Cc: Namhyung Kim 
Cc: Paolo Bonzini 
Cc: Peter Zijlstra 
Cc: Ravi Bangoria 
Cc: Ross Zwisler 
Cc: Sean Christopherson 
Cc: Steven Rostedt (VMware) 
Cc: Tiezhu Yang 
Cc: Yang Jihong 
Link: https://lore.kernel.org/r/20230526183401.2326121-2-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf pmus: Remove perf_pmus__has_hybrid

2023-05-27T12:42:38+00:00

perf_pmus__has_hybrid was used to detect when there was >1 core PMU,
this can be achieved with perf_pmus__num_core_pmus that doesn't depend
upon is_pmu_hybrid and PMU name comparisons. When modifying the
function calls take the opportunity to improve comments,
enable/simplify tests that were previously failing for hybrid but now
pass and to simplify generic code.

Reviewed-by: Kan Liang 
Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Ali Saidi 
Cc: Athira Rajeev 
Cc: Dmitrii Dolgov <9erthalion6@gmail.com>
Cc: Huacai Chen 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jing Zhang 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: Kajol Jain 
Cc: Kang Minchul 
Cc: Leo Yan 
Cc: Madhavan Srinivasan 
Cc: Mark Rutland 
Cc: Mike Leach 
Cc: Ming Wang 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Ravi Bangoria 
Cc: Rob Herring 
Cc: Sandipan Das 
Cc: Sean Christopherson 
Cc: Suzuki Poulouse 
Cc: Thomas Richter 
Cc: Will Deacon 
Cc: Xing Zhengjun 
Cc: coresight@lists.linaro.org
Cc: linux-arm-kernel@lists.infradead.org
Link: https://lore.kernel.org/r/20230527072210.2900565-34-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf pmus: Allow just core PMU scanning

2023-05-27T12:42:00+00:00

Scanning all PMUs is expensive as all PMUs sysfs entries are loaded,
benchmarking shows more than 4x the cost:

```
$ perf bench internals pmu-scan -i 1000
Computing performance of sysfs PMU event scan for 1000 times
  Average core PMU scanning took: 989.231 usec (+- 1.535 usec)
  Average PMU scanning took: 4309.425 usec (+- 74.322 usec)
```

Add new perf_pmus__scan_core routine that scans just core
PMUs. Replace perf_pmus__scan calls with perf_pmus__scan_core when
non-core PMUs are being ignored.

Reviewed-by: Kan Liang 
Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Ali Saidi 
Cc: Athira Rajeev 
Cc: Dmitrii Dolgov <9erthalion6@gmail.com>
Cc: Huacai Chen 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jing Zhang 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: Kajol Jain 
Cc: Kang Minchul 
Cc: Leo Yan 
Cc: Madhavan Srinivasan 
Cc: Mark Rutland 
Cc: Mike Leach 
Cc: Ming Wang 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Ravi Bangoria 
Cc: Rob Herring 
Cc: Sandipan Das 
Cc: Sean Christopherson 
Cc: Suzuki Poulouse 
Cc: Thomas Richter 
Cc: Will Deacon 
Cc: Xing Zhengjun 
Cc: coresight@lists.linaro.org
Cc: linux-arm-kernel@lists.infradead.org
Link: https://lore.kernel.org/r/20230527072210.2900565-30-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf pmu: Separate pmu and pmus

2023-05-27T12:41:39+00:00

Separate and hide the pmus list in pmus.[ch]. Move pmus functionality
out of pmu.[ch] into pmus.[ch] renaming pmus functions which were
prefixed perf_pmu__ to perf_pmus__.

Reviewed-by: Kan Liang 
Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Ali Saidi 
Cc: Athira Rajeev 
Cc: Dmitrii Dolgov <9erthalion6@gmail.com>
Cc: Huacai Chen 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jing Zhang 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: Kajol Jain 
Cc: Kang Minchul 
Cc: Leo Yan 
Cc: Madhavan Srinivasan 
Cc: Mark Rutland 
Cc: Mike Leach 
Cc: Ming Wang 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Ravi Bangoria 
Cc: Rob Herring 
Cc: Sandipan Das 
Cc: Sean Christopherson 
Cc: Suzuki Poulouse 
Cc: Thomas Richter 
Cc: Will Deacon 
Cc: Xing Zhengjun 
Cc: coresight@lists.linaro.org
Cc: linux-arm-kernel@lists.infradead.org
Link: https://lore.kernel.org/r/20230527072210.2900565-28-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf header: Avoid hybrid PMU list in write_pmu_caps

2023-05-27T12:40:48+00:00

Avoid perf_pmu__for_each_hybrid_pmu by iterating all PMUs are dumping
the core ones. This will eventually allow removal of the hybrid PMU
list.

Reviewed-by: Kan Liang 
Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Ali Saidi 
Cc: Athira Rajeev 
Cc: Dmitrii Dolgov <9erthalion6@gmail.com>
Cc: Huacai Chen 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jing Zhang 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: Kajol Jain 
Cc: Kang Minchul 
Cc: Leo Yan 
Cc: Madhavan Srinivasan 
Cc: Mark Rutland 
Cc: Mike Leach 
Cc: Ming Wang 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Ravi Bangoria 
Cc: Rob Herring 
Cc: Sandipan Das 
Cc: Sean Christopherson 
Cc: Suzuki Poulouse 
Cc: Thomas Richter 
Cc: Will Deacon 
Cc: Xing Zhengjun 
Cc: coresight@lists.linaro.org
Cc: linux-arm-kernel@lists.infradead.org
Link: https://lore.kernel.org/r/20230527072210.2900565-21-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf: Extract building cache level for a CPU into separate function

2023-05-23T15:20:14+00:00

build_caches() builds the complete cache topology of the system by
iterating over all CPU, building and comparing cache levels of each CPU,
keeping only the unique ones at the end.

Extract the unit that build the cache levels for a single CPU into a
separate function. Expose this function, and the MAX_CACHE_LVL value to
be used elsewhere in perf too.

Signed-off-by: K Prateek Nayak 
Acked-by: Ian Rogers 
Cc: Alexander Shishkin 
Cc: Ananth Narayan 
Cc: Gautham Shenoy 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Ravi Bangoria 
Cc: Sandipan Das 
Cc: Stephane Eranian 
Cc: Wen Pu 
Link: https://lore.kernel.org/r/20230517172745.5833-2-kprateek.nayak@amd.com
Signed-off-by: Arnaldo Carvalho de Melo

perf evlist: Remove nr_groups

2023-03-13T20:42:27+00:00

Maintaining the number of groups during event parsing is problematic
and since changing to sort/regroup events can only be computed by a
linear pass over the evlist. As the value is generally only used in
tests, rather than hold it in a variable compute it by passing over
the evlist when necessary.

This change highlights that libpfm's counting of groups with a single
entry disagreed with regular event parsing. The libpfm tests are
updated accordingly.

Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Florian Fischer 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: Kajol Jain 
Cc: Kan Liang 
Cc: Kim Phillips 
Cc: Leo Yan 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Ravi Bangoria 
Cc: Sean Christopherson 
Cc: Steinar H. Gunderson 
Cc: Stephane Eranian 
Cc: Suzuki Poulouse 
Cc: Xing Zhengjun 
Link: https://lore.kernel.org/r/20230312021543.3060328-2-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf build: Use libtraceevent from the system

2022-12-14T14:16:12+00:00

Remove the LIBTRACEEVENT_DYNAMIC and LIBTRACEFS_DYNAMIC make command
line variables.

If libtraceevent isn't installed or NO_LIBTRACEEVENT=1 is passed to the
build, don't compile in libtraceevent and libtracefs support.

This also disables CONFIG_TRACE that controls "perf trace".

CONFIG_LIBTRACEEVENT is used to control enablement in Build/Makefiles,
HAVE_LIBTRACEEVENT is used in C code.

Without HAVE_LIBTRACEEVENT tracepoints are disabled and as such the
commands kmem, kwork, lock, sched and timechart are removed.  The
majority of commands continue to work including "perf test".

Committer notes:

Fixed up a tools/perf/util/Build reject and added:

  #include 

to tools/perf/util/scripting-engines/trace-event-perl.c.

Committer testing:

  $ rpm -qi libtraceevent-devel
  Name        : libtraceevent-devel
  Version     : 1.5.3
  Release     : 2.fc36
  Architecture: x86_64
  Install Date: Mon 25 Jul 2022 03:20:19 PM -03
  Group       : Unspecified
  Size        : 27728
  License     : LGPLv2+ and GPLv2+
  Signature   : RSA/SHA256, Fri 15 Apr 2022 02:11:58 PM -03, Key ID 999f7cbf38ab71f4
  Source RPM  : libtraceevent-1.5.3-2.fc36.src.rpm
  Build Date  : Fri 15 Apr 2022 10:57:01 AM -03
  Build Host  : buildvm-x86-05.iad2.fedoraproject.org
  Packager    : Fedora Project
  Vendor      : Fedora Project
  URL         : https://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git/
  Bug URL     : https://bugz.fedoraproject.org/libtraceevent
  Summary     : Development headers of libtraceevent
  Description :
  Development headers of libtraceevent-libs
  $

Default build:

  $ ldd ~/bin/perf | grep tracee
  	libtraceevent.so.1 => /lib64/libtraceevent.so.1 (0x00007f1dcaf8f000)
  $

  # perf trace -e sched:* --max-events 10
       0.000 migration/0/17 sched:sched_migrate_task(comm: "", pid: 1603763 (perf), prio: 120, dest_cpu: 1)
       0.005 migration/0/17 sched:sched_wake_idle_without_ipi(cpu: 1)
       0.011 migration/0/17 sched:sched_switch(prev_comm: "", prev_pid: 17 (migration/0), prev_state: 1, next_comm: "", next_prio: 120)
       1.173 :0/0 sched:sched_wakeup(comm: "", pid: 3138 (gnome-terminal-), prio: 120)
       1.180 :0/0 sched:sched_switch(prev_comm: "", prev_prio: 120, next_comm: "", next_pid: 3138 (gnome-terminal-), next_prio: 120)
       0.156 migration/1/21 sched:sched_migrate_task(comm: "", pid: 1603763 (perf), prio: 120, orig_cpu: 1, dest_cpu: 2)
       0.160 migration/1/21 sched:sched_wake_idle_without_ipi(cpu: 2)
       0.166 migration/1/21 sched:sched_switch(prev_comm: "", prev_pid: 21 (migration/1), prev_state: 1, next_comm: "", next_prio: 120)
       1.183 :0/0 sched:sched_wakeup(comm: "", pid: 1602985 (kworker/u16:0-f), prio: 120, target_cpu: 1)
       1.186 :0/0 sched:sched_switch(prev_comm: "", prev_prio: 120, next_comm: "", next_pid: 1602985 (kworker/u16:0-f), next_prio: 120)
  #

Had to tweak tools/perf/util/setup.py to make sure the python binding
shared object links with libtraceevent if -DHAVE_LIBTRACEEVENT is
present in CFLAGS.

Building with NO_LIBTRACEEVENT=1 uncovered some more build failures:

- Make building of data-convert-bt.c to CONFIG_LIBTRACEEVENT=y

- perf-$(CONFIG_LIBTRACEEVENT) += scripts/

- bpf_kwork.o needs also to be dependent on CONFIG_LIBTRACEEVENT=y

- The python binding needed some fixups and util/trace-event.c can't be
  built and linked with the python binding shared object, so remove it
  in tools/perf/util/setup.py and exclude it from the list of
  dependencies in the python/perf.so Makefile.perf target.

Building without libtraceevent-devel installed uncovered more build
failures:

- The python binding tools/perf/util/python.c was assuming that
  traceevent/parse-events.h was always available, which was the case
  when we defaulted to using the in-kernel tools/lib/traceevent/ files,
  now we need to enclose it under ifdef HAVE_LIBTRACEEVENT, just like
  the other parts of it that deal with tracepoints.

- We have to ifdef the rules in the Build files with
  CONFIG_LIBTRACEEVENT=y to build builtin-trace.c and
  tools/perf/trace/beauty/ as we only ifdef setting CONFIG_TRACE=y when
  setting NO_LIBTRACEEVENT=1 in the make command line, not when we don't
  detect libtraceevent-devel installed in the system. Simplification here
  to avoid these two ways of disabling builtin-trace.c and not having
  CONFIG_TRACE=y when libtraceevent-devel isn't installed is the clean
  way.

From Athira:


tools/perf/arch/powerpc/util/Build
-perf-y += kvm-stat.o
+perf-$(CONFIG_LIBTRACEEVENT) += kvm-stat.o


Then, ditto for arm64 and s390, detected by container cross build tests.

- s/390 uses test__checkevent_tracepoint() that is now only available if
  HAVE_LIBTRACEEVENT is defined, enclose the callsite with ifder HAVE_LIBTRACEEVENT.

Also from Athira:


With this change, I could successfully compile in these environment:
- Without libtraceevent-devel installed
- With libtraceevent-devel installed
- With “make NO_LIBTRACEEVENT=1”


Then, finally rename CONFIG_TRACEEVENT to CONFIG_LIBTRACEEVENT for
consistency with other libraries detected in tools/perf/.

Signed-off-by: Ian Rogers 
Tested-by: Arnaldo Carvalho de Melo 
Tested-by: Athira Rajeev 
Cc: Alexander Shishkin 
Cc: Jiri Olsa 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Nick Desaulniers 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Cc: bpf@vger.kernel.org
Link: http://lore.kernel.org/lkml/20221205225940.3079667-3-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo