linux-stable.git/tools/build/Makefile.feature, branch linux-5.14.y

tools build: Allow deferring printing the results of feature detection

2021-04-29T13:31:00+00:00

By setting FEATURE_DISPLAY_DEFERRED=1 a tool may ask for the printout
of the detected features in tools/build/Makefile.feature to be done
later adter extra feature checks are done that are tool specific.

The perf tool will do it via its tools/perf/Makefile.config, as it
performs such extra feature checks.

Acked-by: Jiri Olsa 
Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Enable libtraceevent dynamic linking

2021-04-29T13:31:00+00:00

Currently we support only static linking with kernel's libtraceevent
(tools/lib/traceevent). This patch adds libtraceevent package detection
and support to link perf with it dynamically.

  The libtraceevent package status is displayed with:
  $ make VF=1 LIBTRACEEVENT_DYNAMIC=1
  ...
  ...                 libtraceevent: [ on  ]

Default behavior remains the same (static linking).

Committer testing:

  $ make LIBTRACEEVENT_DYNAMIC=1 VF=1 O=/tmp/build/perf -C tools/perf install-bin |& grep traceevent
  Makefile.config:1090: *** Error: No libtraceevent devel library found, please install libtraceevent-devel.  Stop.
  $

Signed-off-by: Michael Petlan 
Tested-by: Arnaldo Carvalho de Melo 
Cc: Jiri Olsa 
LPU-Reference: 20210428092023.4009-1-mpetlan@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo

perf build: Support build BPF skeletons with perf

2021-01-15T18:49:07+00:00

BPF programs are useful in perf to profile BPF programs.

BPF skeleton is by far the easiest way to write BPF tools. Enable
building BPF skeletons in util/bpf_skel. A dummy bpf skeleton is added.
More bpf skeletons will be added for different use cases.

Signed-off-by: Song Liu 
Tested-by: Arnaldo Carvalho de Melo 
Acked-by: Jiri Olsa 
Acked-by: Namhyung Kim 
Cc: Alexander Shishkin 
Cc: Mark Rutland 
Cc: Peter Zijlstra 
Cc: kernel-team@fb.com
Link: http://lore.kernel.org/lkml/20201229214214.3413833-3-songliubraving@fb.com
Signed-off-by: Arnaldo Carvalho de Melo

Merge tag 'perf-tools-for-v5.10-2020-10-15' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux

2020-10-17T18:47:46+00:00

Pull perf tools updates from Arnaldo Carvalho de Melo:

- cgroup improvements for 'perf stat', allowing for compact
specification of events and cgroups in the command line.

- Support per thread topdown metrics in 'perf stat'.

- Support sample-read topdown metric group in 'perf record'

- Show start of latency in addition to its start in 'perf sched
latency'.

- Add min, max to 'perf script' futex-contention output, in addition to
avg.

- Allow usage of 'perf_event_attr->exclusive' attribute via the new
':e' event modifier.

- Add 'snapshot' command to 'perf record --control', using it with
Intel PT.

- Support FIFO file names as alternative options to 'perf record
--control'.

- Introduce branch history "streams", to compare 'perf record' runs
with 'perf diff' based on branch records and report hot streams.

- Support PE executable symbol tables using libbfd, to profile, for
instance, wine binaries.

- Add filter support for option 'perf ftrace -F/--funcs'.

- Allow configuring the 'disassembler_style' 'perf annotate' knob via
'perf config'

- Update CascadelakeX and SkylakeX JSON vendor events files.

- Add support for parsing perchip/percore JSON vendor events.

- Add power9 hv_24x7 core level metric events.

- Add L2 prefetch, ITLB instruction fetch hits JSON events for AMD
zen1.

- Enable Family 19h users by matching Zen2 AMD vendor events.

- Use debuginfod in 'perf probe' when required debug files not found
locally.

- Display negative tid in non-sample events in 'perf script'.

- Make GTK2 support opt-in

- Add build test with GTK+

- Add missing -lzstd to the fast path feature detection

- Add scripts to auto generate 'mmap', 'mremap' string<->id tables for
use in 'perf trace'.

- Show python test script in verbose mode.

- Fix uncore metric expressions

- Msan uninitialized use fixes.

- Use condition variables in 'perf bench numa'

- Autodetect python3 binary in systems without python2.

- Support md5 build ids in addition to sha1.

- Add build id 'perf test' regression test.

- Fix printable strings in python3 scripts.

- Fix off by ones in 'perf trace' in arches using libaudit.

- Fix JSON event code for events referencing std arch events.

- Introduce 'perf test' shell script for Arm CoreSight testing.

- Add rdtsc() for Arm64 for used in the PERF_RECORD_TIME_CONV metadata
event and in 'perf test tsc'.

- 'perf c2c' improvements: Add "RMT Load Hit" metric, "Total Stores",
fixes and documentation update.

- Fix usage of reloc_sym in 'perf probe' when using both kallsyms and
debuginfo files.

- Do not print 'Metric Groups:' unnecessarily in 'perf list'

- Refcounting fixes in the event parsing code.

- Add expand cgroup event 'perf test' entry.

- Fix out of bounds CPU map access when handling armv8_pmu events in
'perf stat'.

- Add build-id injection 'perf bench' benchmark.

- Enter namespace when reading build-id in 'perf inject'.

- Do not load map/dso when injecting build-id speeding up the 'perf
inject' process.

- Add --buildid-all option to avoid processing all samples, just the
mmap metadata events.

- Add feature test to check if libbfd has buildid support

- Add 'perf test' entry for PE binary format support.

- Fix typos in power8 PMU vendor events JSON files.

- Hide libtraceevent non API functions.

* tag 'perf-tools-for-v5.10-2020-10-15' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux: (113 commits)
perf c2c: Update documentation for metrics reorganization
perf c2c: Add metrics "RMT Load Hit"
perf c2c: Correct LLC load hit metrics
perf c2c: Change header for LLC local hit
perf c2c: Use more explicit headers for HITM
perf c2c: Change header from "LLC Load Hitm" to "Load Hitm"
perf c2c: Organize metrics based on memory hierarchy
perf c2c: Display "Total Stores" as a standalone metrics
perf c2c: Display the total numbers continuously
perf bench: Use condition variables in numa.
perf jevents: Fix event code for events referencing std arch events
perf diff: Support hot streams comparison
perf streams: Report hot streams
perf streams: Calculate the sum of total streams hits
perf streams: Link stream pair
perf streams: Compare two streams
perf streams: Get the evsel_streams by evsel_idx
perf streams: Introduce branch history "streams"
perf intel-pt: Improve PT documentation slightly
perf tools: Add support for exclusive groups/events
...

perf tools: Make GTK2 support opt-in

2020-09-04T20:11:59+00:00

This is bitrotting, nobody is stepping up to work on it, and since we
treat warnings as errors, feature detection is failing in its main,
faster test (tools/build/feature/test-all.c) because of the GTK+2
infobar check.

So make this opt-in, at some point ditch this if nobody volunteers to
take care of this.

Cc: Adrian Hunter 
Cc: Ian Rogers 
Cc: Jiri Olsa 
Cc: Namhyung Kim 
Signed-off-by: Arnaldo Carvalho de Melo

tools features: Add feature test to check if libbfd has buildid support

2020-09-04T17:38:15+00:00

Which is needed by the PE executable support, for instance.

Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Ian Rogers 
Cc: Jacek Caban 
Cc: Jiri Olsa 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Remi Bernon 
Signed-off-by: Arnaldo Carvalho de Melo

tools: Remove feature-libelf-mmap feature detection

2020-08-19T01:38:25+00:00

It's trivial to handle missing ELF_C_MMAP_READ support in libelf the way that
objtool has solved it in
("774bec3fddcc objtool: Add fallback from ELF_C_READ_MMAP to ELF_C_READ").

So instead of having an entire feature detector for that, just do what objtool
does for perf and libbpf. And keep their Makefiles a bit simpler.

Signed-off-by: Andrii Nakryiko 
Signed-off-by: Alexei Starovoitov 
Link: https://lore.kernel.org/bpf/20200819013607.3607269-5-andriin@fb.com

perf build-ids: Fall back to debuginfod query if debuginfo not found

2020-08-14T12:51:14+00:00

During a perf-record, use the -ldebuginfod API to query a debuginfod
server, should the debug data not be found in the usual system
locations.  If successful, the usual $HOME/.debug dir is populated.

Tested with:

  $ find .
  .
  ./ctags-debuginfo-5.8-26.fc31.x86_64.rpm
  ./usr
  ./usr/lib
  ./usr/lib/debug
  ./usr/lib/debug/.build-id
  ./usr/lib/debug/.build-id/ca
  ./usr/lib/debug/.build-id/ca/46f6ae6a0cee57d85abc1d461c49074248908d
  ./usr/lib/debug/.build-id/ca/46f6ae6a0cee57d85abc1d461c49074248908d.debug
  ./usr/lib/debug/usr
  ./usr/lib/debug/usr/bin
  ./usr/lib/debug/usr/bin/ctags-5.8-26.fc31.x86_64.debug

  $ debuginfod  -F .
  ...

  $ rm -rf ~/.debug/ ; mkdir ~/.debug

  $ perf record make tags
    BUILD:   Doing 'make -j8' parallel build
    GEN      tags
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.107 MB perf.data (1483 samples) ]

  $ find ~/.debug | grep ctags
  /home/jolsa/.debug/usr/bin/ctags
  /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d
  /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d/elf
  /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d/probes

  $ rm -rf ~/.debug/ ; mkdir ~/.debug

  $ DEBUGINFOD_URLS=http://localhost:8002 perf record make tags
    BUILD:   Doing 'make -j8' parallel build
    GEN      tags
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.108 MB perf.data (1531 samples) ]

  $ find ~/.debug | grep ctag
  /home/jolsa/.debug/usr/bin/ctags
  /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d
  /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d/debug
  /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d/elf
  /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d/probes

Note the 'debug' file is created in the last run.

Note that currently the debuginfo data are downloaded only on record path,
we still need add this support to perf build-id/report.. and test ;-)

Tested-by: Jiri Olsa 
Signed-off-by: Jiri Olsa 
Signed-off-by: Frank Ch. Eigler 
Signed-off-by: Arnaldo Carvalho de Melo

tools build feature: Quote CC and CXX for their arguments

2020-08-13T13:00:19+00:00

When using a cross-compilation environment, such as OpenEmbedded,
the CC an CXX variables are set to something more than just a
command: there are arguments (such as --sysroot) that need to be
passed on to the compiler so that the right set of headers and
libraries are used.

For the particular case that our systems detected, CC is set to
the following:

  export CC="aarch64-linaro-linux-gcc  --sysroot=/oe/build/tmp/work/machine/perf/1.0-r9/recipe-sysroot"

Without quotes, detection is as follows:

  Auto-detecting system features:
  ...                         dwarf: [ OFF ]
  ...            dwarf_getlocations: [ OFF ]
  ...                         glibc: [ OFF ]
  ...                          gtk2: [ OFF ]
  ...                        libbfd: [ OFF ]
  ...                        libcap: [ OFF ]
  ...                        libelf: [ OFF ]
  ...                       libnuma: [ OFF ]
  ...        numa_num_possible_cpus: [ OFF ]
  ...                       libperl: [ OFF ]
  ...                     libpython: [ OFF ]
  ...                     libcrypto: [ OFF ]
  ...                     libunwind: [ OFF ]
  ...            libdw-dwarf-unwind: [ OFF ]
  ...                          zlib: [ OFF ]
  ...                          lzma: [ OFF ]
  ...                     get_cpuid: [ OFF ]
  ...                           bpf: [ OFF ]
  ...                        libaio: [ OFF ]
  ...                       libzstd: [ OFF ]
  ...        disassembler-four-args: [ OFF ]

  Makefile.config:414: *** No gnu/libc-version.h found, please install glibc-dev[el].  Stop.
  Makefile.perf:230: recipe for target 'sub-make' failed
  make[1]: *** [sub-make] Error 2
  Makefile:69: recipe for target 'all' failed
  make: *** [all] Error 2

With CC and CXX quoted, some of those features are now detected.

Fixes: e3232c2f39ac ("tools build feature: Use CC and CXX from parent")
Signed-off-by: Daniel Díaz 
Reviewed-by: Thomas Hebb 
Cc: Alexei Starovoitov 
Cc: Andrii Nakryiko 
Cc: Daniel Borkmann 
Cc: Jiri Olsa 
Cc: John Fastabend 
Cc: KP Singh 
Cc: Martin KaFai Lau 
Cc: Namhyung Kim 
Cc: Song Liu 
Cc: Stephane Eranian 
Cc: Yonghong Song 
Link: http://lore.kernel.org/lkml/20200812221518.2869003-1-daniel.diaz@linaro.org
Signed-off-by: Arnaldo Carvalho de Melo

tools build feature: Use CC and CXX from parent

2020-07-31T12:27:58+00:00

commit c8c188679ccf ("tools build: Use the same CC for feature detection
and actual build") changed these assignments from unconditional (:=) to
conditional (?=) so that they wouldn't clobber values from the
environment. However, conditional assignment does not work properly for
variables that Make implicitly sets, among which are CC and CXX. To
quote tools/scripts/Makefile.include, which handles this properly:

  # Makefiles suck: This macro sets a default value of $(2) for the
  # variable named by $(1), unless the variable has been set by
  # environment or command line. This is necessary for CC and AR
  # because make sets default values, so the simpler ?= approach
  # won't work as expected.

In other words, the conditional assignments will not run even if the
variables are not overridden in the environment; Make will set CC to
"cc" and CXX to "g++" when it starts[1], meaning the variables are not
empty by the time the conditional assignments are evaluated. This breaks
cross-compilation when CROSS_COMPILE is set but CC isn't, since "cc"
gets used for feature detection instead of the cross compiler (and
likewise for CXX).

To fix the issue, just pass down the values of CC and CXX computed by
the parent Makefile, which gets included by the Makefile that actually
builds whatever we're detecting features for and so is guaranteed to
have good values. This is a better solution anyway, since it means we
aren't trying to replicate the logic of the parent build system and so
don't risk it getting out of sync.

Leave PKG_CONFIG alone, since 1) there's no common logic to compute it
in Makefile.include, and 2) it's not an implicit variable, so
conditional assignment works properly.

[1] https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html

Fixes: c8c188679ccf ("tools build: Use the same CC for feature detection and actual build")
Signed-off-by: Thomas Hebb 
Acked-by: Jiri Olsa 
Cc: David Carrillo-Cisneros 
Cc: Ian Rogers 
Cc: Igor Lubashev 
Cc: Namhyung Kim 
Cc: Quentin Monnet 
Cc: Song Liu 
Cc: Stephane Eranian 
Cc: thomas hebb 
Link: http://lore.kernel.org/lkml/0a6e69d1736b0fa231a648f50b0cce5d8a6734ef.1595822871.git.tommyhebb@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo