linux-stable.git/tools/perf/Makefile.perf, branch v6.0

perf tools: Don't install data files with x permissions

2022-09-08T18:55:56+00:00

install(1), by default, installs with rwxr-xr-x permissions. Modify
perf's Makefile to pass '-m 644' when installing:

  * Documentation/tips.txt
  * examples/bpf/*
  * perf-completion.sh
  * perf_dlfilter.h header
  * scripts/perl/Perf-Trace-Util/lib/Perf/Trace/*
  * scripts/perl/*.pl
  * tests/attr/*
  * tests/attr.py
  * tests/shell/lib/*.sh
  * trace/strace/groups/*

All those are supposed to be non-executable. Either they are not scripts
at all, or they don't have shebang.

Signed-off-by: 
Cc: Adrian Hunter 
Cc: Andi Kleen 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Kan Liang 
Cc: Leo Yan 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Link: https://lore.kernel.org/r/20220908060426.9619-1-jslaby@suse.cz
Signed-off-by: Arnaldo Carvalho de Melo

perf test: JSON format checking

2022-08-10T13:44:01+00:00

Add field checking tests for perf stat JSON output.

Sanity checks the expected number of fields are present, that the
expected keys are present and they have the correct values.

Committer notes:

Had to fix this:

  -               $(INSTALL) tests/shell/lib/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib' \
  +               $(INSTALL) tests/shell/lib/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \

Committer testing:

  [root@quaco ~]# perf test json
   90: perf stat JSON output linter                                    : Ok
  [root@quaco ~]# set -o vi
  [root@quaco ~]# perf test -v json
   90: perf stat JSON output linter                                    :
  --- start ---
  test child forked, pid 560794
  Checking json output: no args [Success]
  Checking json output: system wide [Success]
  Checking json output: system wide Checking json output: system wide no aggregation [Success]
  Checking json output: interval [Success]
  Checking json output: event [Success]
  Checking json output: per core [Success]
  Checking json output: per thread [Success]
  Checking json output: per die [Success]
  Checking json output: per node [Success]
  Checking json output: per socket [Success]
  test child finished with 0
  ---- end ----
  perf stat JSON output linter: Ok
  [root@quaco ~]#

Signed-off-by: Claire Jensen 
Acked-by: Namhyung Kim 
Tested-by: Arnaldo Carvalho de Melo 
Cc: Alexander Shishkin 
Cc: Alyssa Ross 
Cc: Claire Jensen 
Cc: Florian Fischer 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jiri Olsa 
Cc: Kan Liang 
Cc: Like Xu 
Cc: Mark Rutland 
Cc: Peter Zijlstra 
Cc: Sandipan Das 
Cc: Stephane Eranian 
Cc: Xing Zhengjun 
Link: https://lore.kernel.org/r/20220805200105.2020995-3-irogers@google.com
Signed-off-by: Ian Rogers 
Signed-off-by: Arnaldo Carvalho de Melo

perf lock: Use BPF for lock contention analysis

2022-08-01T12:28:38+00:00

Add -b/--use-bpf option to use BPF to collect lock contention stats.
For simplicity it now runs system-wide and requires C-c to stop.
Upcoming changes will add the usual filtering.

  $ sudo perf lock con -b
  ^C
   contended   total wait     max wait     avg wait         type   caller

          42    192.67 us     13.64 us      4.59 us     spinlock   queue_work_on+0x20
          23     85.54 us     10.28 us      3.72 us     spinlock   worker_thread+0x14a
           6     13.92 us      6.51 us      2.32 us        mutex   kernfs_iop_permission+0x30
           3     11.59 us     10.04 us      3.86 us        mutex   kernfs_dop_revalidate+0x3c
           1      7.52 us      7.52 us      7.52 us     spinlock   kthread+0x115
           1      7.24 us      7.24 us      7.24 us     rwlock:W   sys_epoll_wait+0x148
           2      7.08 us      3.99 us      3.54 us     spinlock   delayed_work_timer_fn+0x1b
           1      6.41 us      6.41 us      6.41 us     spinlock   idle_balance+0xa06
           2      2.50 us      1.83 us      1.25 us        mutex   kernfs_iop_lookup+0x2f
           1      1.71 us      1.71 us      1.71 us        mutex   kernfs_iop_getattr+0x2c

Signed-off-by: Namhyung Kim 
Cc: Blake Jones 
Cc: Boqun Feng 
Cc: Davidlohr Bueso 
Cc: Ian Rogers 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Cc: Waiman Long 
Cc: Will Deacon 
Link: https://lore.kernel.org/r/20220729200756.666106-3-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo

perf kwork: Implement BPF trace

2022-07-26T19:31:54+00:00

'perf record' generates perf.data, which generates extra interrupts
for hard disk, amount of data to be collected increases with time.

Using eBPF trace can process the data in kernel, which solves the
preceding two problems.

Add -b/--use-bpf option for latency and report to support
tracing kwork events using eBPF:

1. Create bpf prog and attach to tracepoints,
2. Start tracing after command is entered,
3. After user hit "ctrl+c", stop tracing and report,
4. Support CPU and name filtering.

This commit implements the framework code and
does not add specific event support.

Test cases:

  # perf kwork rep -h

   Usage: perf kwork report []

      -b, --use-bpf         Use BPF to measure kwork runtime
      -C, --cpu        list of cpus to profile
      -i, --input     input file name
      -n, --name      event name to profile
      -s, --sort 
                            sort by key(s): runtime, max, count
      -S, --with-summary    Show summary with statistics
          --time       Time span for analysis (start,stop)

  # perf kwork lat -h

   Usage: perf kwork latency []

      -b, --use-bpf         Use BPF to measure kwork latency
      -C, --cpu        list of cpus to profile
      -i, --input     input file name
      -n, --name      event name to profile
      -s, --sort 
                            sort by key(s): avg, max, count
          --time       Time span for analysis (start,stop)

  # perf kwork lat -b
  Unsupported bpf trace class irq

  # perf kwork rep -b
  Unsupported bpf trace class irq

Signed-off-by: Yang Jihong 
Cc: Alexander Shishkin 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Paul Clarke 
Cc: Peter Zijlstra 
Link: https://lore.kernel.org/r/20220709015033.38326-15-yangjihong1@huawei.com
[ Simplify work_findnew() ]
Signed-off-by: Arnaldo Carvalho de Melo

perf jevents: Remove jevents.c

2022-06-29T21:39:10+00:00

Remove files and build rules.

Remove test for comparing with jevents.py as there is no longer a binary
to compare with.

Signed-off-by: Ian Rogers 
Tested-by: John Garry 
Cc: Alexander Shishkin 
Cc: Ananth Narayan 
Cc: Andi Kleen 
Cc: Andrew Kilroy 
Cc: Caleb Biggers 
Cc: Felix Fietkau 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jiri Olsa 
Cc: Kajol Jain 
Cc: Kan Liang 
Cc: Kshipra Bopardikar 
Cc: Like Xu 
Cc: Mark Rutland 
Cc: Mathieu Poirier 
Cc: Namhyung Kim 
Cc: Nick Forrington 
Cc: Paul Clarke 
Cc: Perry Taylor 
Cc: Peter Zijlstra 
Cc: Qi Liu 
Cc: Ravi Bangoria 
Cc: Sandipan Das 
Cc: Santosh Shukla 
Cc: Stephane Eranian 
Cc: Will Deacon 
Cc: Xing Zhengjun 
Link: https://lore.kernel.org/r/20220629182505.406269-5-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf jevents: Switch build to use jevents.py

2022-06-29T21:36:14+00:00

Generate pmu-events.c using jevents.py rather than the binary built from
jevents.c.

Add a new config variable NO_JEVENTS that is set when there is no
architecture json or an appropriate python interpreter isn't present.

When NO_JEVENTS is defined the file pmu-events/empty-pmu-events.c is
copied and used as the pmu-events.c file.

Signed-off-by: Ian Rogers 
Tested-by: John Garry 
Cc: Alexander Shishkin 
Cc: Ananth Narayan 
Cc: Andi Kleen 
Cc: Andrew Kilroy 
Cc: Caleb Biggers 
Cc: Felix Fietkau 
Cc: Ian Rogers 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jiri Olsa 
Cc: Kajol Jain 
Cc: Kan Liang 
Cc: Kshipra Bopardikar 
Cc: Like Xu 
Cc: Mark Rutland 
Cc: Mathieu Poirier 
Cc: Namhyung Kim 
Cc: Nick Forrington 
Cc: Paul Clarke 
Cc: Perry Taylor 
Cc: Peter Zijlstra 
Cc: Qi Liu 
Cc: Ravi Bangoria 
Cc: Sandipan Das 
Cc: Santosh Shukla 
Cc: Stephane Eranian 
Cc: Will Deacon 
Cc: Xing Zhengjun 
Link: https://lore.kernel.org/r/20220629182505.406269-4-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf jevents: Add python converter script

2022-06-29T21:34:31+00:00

jevents.c is large, has a dependency on an old forked version of jsmn,
and is challenging to work upon. A lot of jevents.c's complexity comes
from needing to write json and csv parsing from first principles. In
contrast python has this functionality in standard libraries and is
already a build pre-requisite for tools like asciidoc (that builds all
of the perf man pages).

Introduce jevents.py that produces identical output to jevents.c. Add a
test that runs both converter tools and validates there are no output
differences. The test can be invoked with a phony build target like:

  $ make -C tools/perf jevents-py-test

The python code deliberately tries to replicate the behavior of
jevents.c so that the output matches and transitioning tools shouldn't
introduce regressions. In some cases the code isn't as elegant as hoped,
but fixing this can be done as follow up.

Committer testing:

  $ make -C tools/perf jevents-py-test
  make: Entering directory '/var/home/acme/git/perf/tools/perf'
    BUILD:   Doing 'make -j32' parallel build
    HOSTCC  fixdep.o
    HOSTLD  fixdep-in.o
    LINK    fixdep

  Auto-detecting system features:
  ...                         dwarf: [ on  ]
  ...            dwarf_getlocations: [ on  ]
  ...                         glibc: [ on  ]
  ...                        libbfd: [ on  ]
  ...                libbfd-buildid: [ on  ]
  ...                        libcap: [ on  ]
  ...                        libelf: [ on  ]
  ...                       libnuma: [ on  ]
  ...        numa_num_possible_cpus: [ on  ]
  ...                       libperl: [ on  ]
  ...                     libpython: [ on  ]
  ...                     libcrypto: [ OFF ]
  ...                     libunwind: [ on  ]
  ...            libdw-dwarf-unwind: [ on  ]
  ...                          zlib: [ on  ]
  ...                          lzma: [ on  ]
  ...                     get_cpuid: [ on  ]
  ...                           bpf: [ on  ]
  ...                        libaio: [ on  ]
  ...                       libzstd: [ on  ]
  ...        disassembler-four-args: [ on  ]

    HOSTCC  pmu-events/json.o
    HOSTCC  pmu-events/jsmn.o
    HOSTCC  pmu-events/jevents.o
    HOSTLD  pmu-events/jevents-in.o
    LINK    pmu-events/jevents
  Checking architecture: arm64
  Generating using jevents.c
  Generating using jevents.py
  Diffing
  Checking architecture: nds32
  Generating using jevents.c
  Generating using jevents.py
  Diffing
  Checking architecture: powerpc
  Generating using jevents.c
  Generating using jevents.py
  Diffing
  Checking architecture: s390
  Generating using jevents.c
  Generating using jevents.py
  Diffing
  Checking architecture: x86
  Generating using jevents.c
  Generating using jevents.py
  Diffing
  make: Leaving directory '/var/home/acme/git/perf/tools/perf'
  $

Signed-off-by: Ian Rogers 
Tested-by: John Garry 
Tested-by: Thomas Richter 
Tested-by: Xing Zhengjun 
Tested-by: Arnaldo Carvalho de Melo 
Cc: Alexander Shishkin 
Cc: Ananth Narayan 
Cc: Andi Kleen 
Cc: Andrew Kilroy 
Cc: Caleb Biggers 
Cc: Felix Fietkau 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jiri Olsa 
Cc: Kajol Jain 
Cc: Kan Liang 
Cc: Kshipra Bopardikar 
Cc: Like Xu 
Cc: Mark Rutland 
Cc: Mathieu Poirier 
Cc: Namhyung Kim 
Cc: Nick Forrington 
Cc: Paul Clarke 
Cc: Perry Taylor 
Cc: Peter Zijlstra 
Cc: Qi Liu 
Cc: Ravi Bangoria 
Cc: Sandipan Das 
Cc: Santosh Shukla 
Cc: Stephane Eranian 
Cc: Will Deacon 
Link: https://lore.kernel.org/r/20220629182505.406269-3-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf record: Enable off-cpu analysis with BPF

2022-05-26T15:36:57+00:00

Add --off-cpu option to enable the off-cpu profiling with BPF.  It'd
use a bpf_output event and rename it to "offcpu-time".  Samples will
be synthesized at the end of the record session using data from a BPF
map which contains the aggregated off-cpu time at context switches.
So it needs root privilege to get the off-cpu profiling.

Each sample will have a separate user stacktrace so it will skip
kernel threads.  The sample ip will be set from the stacktrace and
other sample data will be updated accordingly.  Currently it only
handles some basic sample types.

The sample timestamp is set to a dummy value just not to bother with
other events during the sorting.  So it has a very big initial value
and increase it on processing each samples.

Good thing is that it can be used together with regular profiling like
cpu cycles.  If you don't want to that, you can use a dummy event to
enable off-cpu profiling only.

Example output:
  $ sudo perf record --off-cpu perf bench sched messaging -l 1000

  $ sudo perf report --stdio --call-graph=no
  # Total Lost Samples: 0
  #
  # Samples: 41K of event 'cycles'
  # Event count (approx.): 42137343851
  ...

  # Samples: 1K of event 'offcpu-time'
  # Event count (approx.): 587990831640
  #
  # Children      Self  Command          Shared Object       Symbol
  # ........  ........  ...............  ..................  .........................
  #
      81.66%     0.00%  sched-messaging  libc-2.33.so        [.] __libc_start_main
      81.66%     0.00%  sched-messaging  perf                [.] cmd_bench
      81.66%     0.00%  sched-messaging  perf                [.] main
      81.66%     0.00%  sched-messaging  perf                [.] run_builtin
      81.43%     0.00%  sched-messaging  perf                [.] bench_sched_messaging
      40.86%    40.86%  sched-messaging  libpthread-2.33.so  [.] __read
      37.66%    37.66%  sched-messaging  libpthread-2.33.so  [.] __write
       2.91%     2.91%  sched-messaging  libc-2.33.so        [.] __poll
  ...

As you can see it spent most of off-cpu time in read and write in
bench_sched_messaging().  The --call-graph=no was added just to make
the output concise here.

It uses perf hooks facility to control BPF program during the record
session rather than adding new BPF/off-cpu specific calls.

Signed-off-by: Namhyung Kim 
Acked-by: Ian Rogers 
Tested-by: Arnaldo Carvalho de Melo 
Cc: Andi Kleen 
Cc: Blake Jones 
Cc: Hao Luo 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Milian Wolff 
Cc: Peter Zijlstra 
Cc: Song Liu 
Cc: bpf@vger.kernel.org
Link: https://lore.kernel.org/r/20220518224725.742882-3-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Delete perf-with-kcore.sh script

2022-04-27T23:11:26+00:00

It has been obsolete since the introduction of the 'perf record --kcore'
option.

Signed-off-by: Adrian Hunter 
Cc: Jiri Olsa 
Link: http://lore.kernel.org/lkml/20220427141946.269523-1-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Stop depending on .git files for building PERF-VERSION-FILE

2022-04-01T19:19:35+00:00

This essentially reverts commit c72e3f04b45fb2e5 ("tools/perf/build:
Speed up git-version test on re-make") and commit 4e666cdb06eede20
("perf tools: Fix dependency for version file creation")

In commit c72e3f04b45fb2e5 ("tools/perf/build: Speed up git-version test
on re-make"), a makefile dependency on .git/HEAD was added. The
background is that running PERF-VERSION-FILE is relatively slow, and
commands like "git describe" are particularly slow.

In commit 4e666cdb06eede20 ("perf tools: Fix dependency for version file
creation"), an additional dependency on .git/ORIG_HEAD was added, as
.git/HEAD may not change for "git reset --hard HEAD^" command. However,
depending on whether we're on a branch or not, a "git cherry-pick" may
not lead to the version being updated.

As discussed with the git community in [0], using git internal files for
dependencies is not reliable. Commit 4e666cdb06ee also breaks some build
scenarios [1].

As mentioned, c72e3f04b45fb2e5 ("tools/perf/build: Speed up git-version
test on re-make") was added to speed up the build. However in commit
7572733b84997d23 ("perf tools: Fix version kernel tag") we removed the
call to "git describe", so just revert Makefile.perf back to same as pre
c72e3f04b45fb2e5 ("tools/perf/build: Speed up git-version test on
re-make") and the build should not be so slow, as below:

Pre 7572733b8499:

  $> time util/PERF-VERSION-GEN
    PERF_VERSION = 5.17.rc8.g4e666cdb06ee

  real    0m0.110s
  user    0m0.091s
  sys     0m0.019s

Post 7572733b8499:

  $> time util/PERF-VERSION-GEN
    PERF_VERSION = 5.17.rc8.g7572733b8499

  real    0m0.039s
  user    0m0.036s
  sys     0m0.007s

[0] https://lore.kernel.org/git/87wngkpddp.fsf@igel.home/T/#m4a4dd6de52fdbe21179306cd57b3761eb07f45f8
[1] https://lore.kernel.org/linux-perf-users/20220329093120.4173283-1-matthieu.baerts@tessares.net/T/#u

Committer testing:

After a fresh rebuild using 'make -C tools/perf O=/tmp/build/perf install-bin':

  $ perf -v
  perf version 5.17.g162f9db407b6
  $ git log --oneline -1
  162f9db407b6a6e5 (HEAD -> perf/core) perf tools: Stop depending on .git files for building PERF-VERSION-FILE
  $

Now using a detached tarball, i.e. outside the kernel source tree:

  $ ls -la perf*tar
  ls: cannot access 'perf*tar': No such file or directory
  $ make perf-tar-src-pkg
    TAR
    PERF_VERSION = 5.17.g31d10b3ef133
  $ ls -la perf*tar
  -rw-r--r--. 1 acme acme 22241280 Mar 30 13:26 perf-5.17.0.tar
  $ mv perf-5.17.0.tar /tmp
  $ cd /tmp
  $ tar xf perf-5.17.0.tar
  $ cd perf-5.17.0/
  $ make -C tools/perf |& tail
    CC      util/pmu.o
    CC      util/pmu-flex.o
    CC      util/expr-flex.o
    CC      util/expr.o
    LD      util/scripting-engines/perf-in.o
    LD      util/intel-pt-decoder/perf-in.o
    LD      util/perf-in.o
    LD      perf-in.o
    LINK    perf
  make: Leaving directory '/tmp/perf-5.17.0/tools/perf'
  $ tools/perf/perf -v
  perf version 5.17.g31d10b3ef133
  $ pwd
  /tmp/perf-5.17.0
  $ cat PERF-VERSION-FILE
  #define PERF_VERSION "5.17.g31d10b3ef133"
  $

Fixes: 4e666cdb06eede20 ("perf tools: Fix dependency for version file creation")
Reported-by: Matthieu Baerts 
Signed-off-by: John Garry 
Tested-by: Arnaldo Carvalho de Melo 
Tested-by: Matthieu Baerts 
Cc: Alexander Shishkin 
Cc: Ian Rogers 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Link: https://lore.kernel.org/r/1648635774-14581-1-git-send-email-john.garry@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo