linux-stable.git/tools/perf/util/scripting-engines, branch linux-4.19.y

perf scripting python: Avoid declaring function pointers with a visibility attribute

2020-11-18T18:18:52+00:00

commit d0e7b0c71fbb653de90a7163ef46912a96f0bdaf upstream.

To avoid this:

  util/scripting-engines/trace-event-python.c: In function 'python_start_script':
  util/scripting-engines/trace-event-python.c:1595:2: error: 'visibility' attribute ignored [-Werror=attributes]
   1595 |  PyMODINIT_FUNC (*initfunc)(void);
        |  ^~~~~~~~~~~~~~

That started breaking when building with PYTHON=python3 and these gcc
versions (I haven't checked with the clang ones, maybe it breaks there
as well):

  # export PERF_TARBALL=http://192.168.86.5/perf/perf-5.9.0.tar.xz
  # dm  fedora:33 fedora:rawhide
     1   107.80 fedora:33         : Ok   gcc (GCC) 10.2.1 20201005 (Red Hat 10.2.1-5), clang version 11.0.0 (Fedora 11.0.0-1.fc33)
     2    92.47 fedora:rawhide    : Ok   gcc (GCC) 10.2.1 20201016 (Red Hat 10.2.1-6), clang version 11.0.0 (Fedora 11.0.0-1.fc34)
  #

Avoid that by ditching that 'initfunc' function pointer with its:

    #define Py_EXPORTED_SYMBOL _attribute_ ((visibility ("default")))
    #define PyMODINIT_FUNC Py_EXPORTED_SYMBOL PyObject*

And just call PyImport_AppendInittab() at the end of the ifdef python3
block with the functions that were being attributed to that initfunc.

Cc: Adrian Hunter 
Cc: Ian Rogers 
Cc: Jiri Olsa 
Cc: Namhyung Kim 
Signed-off-by: Arnaldo Carvalho de Melo 
Signed-off-by: Tapas Kundu 
Signed-off-by: Greg Kroah-Hartman

perf script python: Add trace_context extension module to sys.modules

2019-04-05T20:33:14+00:00

[ Upstream commit cc437642255224e4140fed1f3e3156fc8ad91903 ]

In Python3, the result of PyModule_Create (called from
scripts/python/Perf-Trace-Util/Context.c) is not automatically added to
sys.modules.  See: https://bugs.python.org/issue4592

Below is the observed behavior without the fix:

  # ldd /usr/bin/perf | grep -i python
	libpython3.6m.so.1.0 => /usr/lib64/libpython3.6m.so.1.0 (0x00007f8e1dfb2000)

  # perf record /bin/false
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.015 MB perf.data (17 samples) ]

  # perf script -g python | cat
  generated Python script: perf-script.py

  # perf script -s ./perf-script.py
  Traceback (most recent call last):
    File "./perf-script.py", line 18, in 
      from perf_trace_context import *
  ModuleNotFoundError: No module named 'perf_trace_context'
  Error running python script ./perf-script.py
  #

Committer notes:

To build with python3 use:

  $ make -C tools/perf PYTHON=python3

Use a non-const variable to pass the 'name' arg to
PyImport_AppendInittab(), as python2.6 has that as 'char *', which ends
up trowing this in some environments:

   CC       /tmp/build/perf/util/parse-branch-options.o
  util/scripting-engines/trace-event-python.c: In function 'python_start_script':
  util/scripting-engines/trace-event-python.c:1520:2: error: passing argument 1 of 'PyImport_AppendInittab' discards 'const' qualifier from pointer target type [-Werror]
    PyImport_AppendInittab("perf_trace_context", initfunc);
    ^
  In file included from /usr/include/python2.6/Python.h:130:0,
                   from util/scripting-engines/trace-event-python.c:22:
  /usr/include/python2.6/import.h:54:17: note: expected 'char *' but argument is of type 'const char *'
   PyAPI_FUNC(int) PyImport_AppendInittab(char *name, void (*initfunc)(void));
                   ^
  cc1: all warnings being treated as errors

Signed-off-by: Tony Jones 
Acked-by: Jiri Olsa 
Tested-by: Arnaldo Carvalho de Melo 
Cc: Jaroslav Škarvada 
Cc: Jonathan Corbet 
Cc: Ravi Bangoria 
Cc: Seeteena Thoufeek 
Fixes: 66dfdff03d19 ("perf tools: Add Python 3 support")
Link: http://lkml.kernel.org/r/20190124005229.16146-2-tonyj@suse.de
Signed-off-by: Arnaldo Carvalho de Melo 
Signed-off-by: Sasha Levin

perf script python: Use PyBytes for attr in trace-event-python

2019-04-05T20:33:14+00:00

[ Upstream commit 72e0b15cb24a497d7d0d4707cf51ff40c185ae8c ]

With Python3.  PyUnicode_FromStringAndSize is unsafe to call on attr and will
return NULL.  Use _PyBytes_FromStringAndSize (as with raw_buf).

Below is the observed behavior without the fix.  Note it is first necessary
to apply the prior fix (Add trace_context extension module to sys,modules):

  # ldd /usr/bin/perf | grep -i python
          libpython3.6m.so.1.0 => /usr/lib64/libpython3.6m.so.1.0 (0x00007f8e1dfb2000)

  # perf record -e raw_syscalls:sys_enter /bin/false
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.018 MB perf.data (21 samples) ]

  # perf script -g python | cat
  generated Python script: perf-script.py

  # perf script -s ./perf-script.py
  in trace_begin
  Segmentation fault (core dumped)

Signed-off-by: Tony Jones 
Acked-by: Jiri Olsa 
Tested-by: Arnaldo Carvalho de Melo 
Cc: Jaroslav Škarvada 
Cc: Jonathan Corbet 
Cc: Ravi Bangoria 
Cc: Seeteena Thoufeek 
Fixes: 66dfdff03d19 ("perf tools: Add Python 3 support")
Link: http://lkml.kernel.org/r/20190124005229.16146-3-tonyj@suse.de
Signed-off-by: Arnaldo Carvalho de Melo 
Signed-off-by: Sasha Levin

perf script: Use fallbacks for branch stacks

2019-01-09T16:38:42+00:00

commit 692d0e63324d2954a0c63a812a8588e97023a295 upstream.

Branch stacks do not necessarily have the same cpumode as the 'ip'. Use
the fallback functions in those cases.

This patch depends on patch "perf tools: Add fallback functions for cases
where cpumode is insufficient".

Signed-off-by: Adrian Hunter 
Cc: Andi Kleen 
Cc: David S. Miller 
Cc: Jiri Olsa 
Cc: Leo Yan 
Cc: Mathieu Poirier 
Cc: stable@vger.kernel.org # 4.19
Link: http://lkml.kernel.org/r/20181106210712.12098-4-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo 
Signed-off-by: Greg Kroah-Hartman

tools lib traceevent, perf tools: Rename pevent_read_number_* APIs

2018-08-13T18:22:05+00:00

In order to make libtraceevent into a proper library, variables, data
structures and functions require a unique prefix to prevent name space
conflicts. That prefix will be "tep_" and not "pevent_". This changes
APIs: pevent_read_number, pevent_read_number_field

Signed-off-by: Tzvetomir Stoyanov (VMware) 
Cc: Andrew Morton 
Cc: Peter Zijlstra 
Cc: Yordan Karadzhov (VMware) 
Cc: linux-trace-devel@vger.kernel.org
Link: http://lkml.kernel.org/r/20180808180700.804271434@goodmis.org
Signed-off-by: Steven Rostedt 
Signed-off-by: Arnaldo Carvalho de Melo

tools lib traceevent, perf tools: Rename struct pevent to struct tep_handle

2018-08-10T18:29:35+00:00

In order to make libtraceevent into a proper library, variables, data
structures and functions require a unique prefix to prevent name space
conflicts. That prefix will be "tep_" and not "pevent_". This changes
the struct pevent to struct tep_handle.

Signed-off-by: Tzvetomir Stoyanov (VMware) 
Cc: Andrew Morton 
Cc: Peter Zijlstra 
Cc: Yordan Karadzhov (VMware) 
Cc: linux-trace-devel@vger.kernel.org
Link: http://lkml.kernel.org/r/20180808180659.706175783@goodmis.org
Signed-off-by: Steven Rostedt 
Signed-off-by: Arnaldo Carvalho de Melo

perf script python: Fix dict reference counting

2018-07-11T13:45:24+00:00

The dictionaries are attached to the parameter tuple that steals the
references and takes care of releasing them when appropriate.  The code
should not decrement the reference counts explicitly.  E.g. if libpython
has been built with reference debugging enabled, the superfluous DECREFs
will trigger this error when running perf script:

  Fatal Python error: Objects/tupleobject.c:238 object at
  0x7f10f2041b40 has negative ref count -1
  Aborted (core dumped)

If the reference debugging is not enabled, the superfluous DECREFs might
cause the dict objects to be silently released while they are still in
use. This may trigger various other assertions or just cause perf
crashes and/or weird and unexpected data changes in the stored Python
objects.

Signed-off-by: Janne Huttunen 
Acked-by: Jiri Olsa 
Acked-by: Namhyung Kim 
Cc: Alexander Shishkin 
Cc: Andi Kleen 
Cc: Jaroslav Skarvada 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Link: http://lkml.kernel.org/r/1531133990-17485-1-git-send-email-janne.huttunen@nokia.com
Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Generate a Python script compatible with Python 2 and 3

2018-07-11T13:01:50+00:00

When generating a Python script with "perf script -g python", produce
one that is compatible with Python 2 and 3. The difference between the
two generated scripts is:

  --- python2-perf-script.py	2018-05-08 15:35:00.865889705 -0400
  +++ python3-perf-script.py	2018-05-08 15:34:49.019789564 -0400
  @@ -7,6 +7,8 @@
   # be retrieved using Python functions of the form common_*(context).
   # See the perf-script-python Documentation for the list of available functions.

  +from __future__ import print_function
  +
   import os
   import sys

  @@ -18,10 +20,10 @@

   def trace_begin():
  -	print "in trace_begin"
  +	print("in trace_begin")

   def trace_end():
  -	print "in trace_end"
  +	print("in trace_end")

   def raw_syscalls__sys_enter(event_name, context, common_cpu,
   	common_secs, common_nsecs, common_pid, common_comm,
  @@ -29,26 +31,26 @@
   		print_header(event_name, common_cpu, common_secs, common_nsecs,
   			common_pid, common_comm)

  -		print "id=%d, args=%s" % \
  -		(id, args)
  +		print("id=%d, args=%s" % \
  +		(id, args))

  -		print 'Sample: {'+get_dict_as_string(perf_sample_dict['sample'], ', ')+'}'
  +		print('Sample: {'+get_dict_as_string(perf_sample_dict['sample'], ', ')+'}')

   		for node in common_callchain:
   			if 'sym' in node:
  -				print "\t[%x] %s" % (node['ip'], node['sym']['name'])
  +				print("\t[%x] %s" % (node['ip'], node['sym']['name']))
   			else:
  -				print "	[%x]" % (node['ip'])
  +				print("	[%x]" % (node['ip']))

  -		print "\n"
  +		print()

   def trace_unhandled(event_name, context, event_fields_dict, perf_sample_dict):
  -		print get_dict_as_string(event_fields_dict)
  -		print 'Sample: {'+get_dict_as_string(perf_sample_dict['sample'], ', ')+'}'
  +		print(get_dict_as_string(event_fields_dict))
  +		print('Sample: {'+get_dict_as_string(perf_sample_dict['sample'], ', ')+'}')

   def print_header(event_name, cpu, secs, nsecs, pid, comm):
  -	print "%-20s %5u %05u.%09u %8u %-20s " % \
  -	(event_name, cpu, secs, nsecs, pid, comm),
  +	print("%-20s %5u %05u.%09u %8u %-20s " % \
  +	(event_name, cpu, secs, nsecs, pid, comm), end="")

   def get_dict_as_string(a_dict, delimiter=' '):
   	return delimiter.join(['%s=%s'%(k,str(v))for k,v in sorted(a_dict.items())])

Signed-off-by: Jeremy Cline 
Cc: Alexander Shishkin 
Cc: Herton Krzesinski 
Cc: Jiri Olsa 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Link: http://lkml.kernel.org/r/0100016341a7278a-d178c724-2b0f-49ca-be93-80a7d51aaa0d-000000@email.amazonses.com
Signed-off-by: Arnaldo Carvalho de Melo

perf script python: Add more PMU fields to event handler dict

2018-06-06T18:38:26+00:00

When doing pmu sampling and then running a script with perf script -s
script.py, the process_event function gets dictionary with some fields
from the perf ring buffer (like ip, sym, callchain etc).

But we miss quite a few fields we report now, for example, LBRs, data
source, weight, transaction, iregs, uregs, etc.

This patch reports these fields for perf script python processing.

  New keys/items:
  ---------------
  key  : brstack
  items: from, to, from_dsoname, to_dsoname, mispred,
         predicted, in_tx, abort, cycles.

  key  : brstacksym
  items: from, to, pred, in_tx, abort (converted string)

  key  : datasrc
  key  : datasrc_decode (decoded string)
  key  : iregs
  key  : uregs
  key  : weight
  key  : transaction

  v2:
  ---
  Add new fields for dso.
  Use PyBool_FromLong() for mispred/predicted/in_tx/abort

Committer notes:

!sym->name isn't valid, as its not a pointer, its a [0] array, use
!sym->name[0] instead, guaranteed to be the case by symbol__new.

This was caught by just one of the containers:

  52    54.22 ubuntu:17.04                  : FAIL gcc (Ubuntu 6.3.0-12ubuntu2) 6.3.0 20170406

    CC       /tmp/build/perf/util/scripting-engines/trace-event-python.o
  util/scripting-engines/trace-event-python.c:534:20: error: address of array 'sym->name' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
          if (!sym || !sym->name)
                    ~~~~~~^~~~
  1 error generated.
  mv: cannot stat '/tmp/build/perf/util/scripting-engines/.trace-event-python.o.tmp': No such file or directory
  /git/linux/tools/build/Makefile.build:96: recipe for target '/tmp/build/perf/util/scripting-engines/trace-event-python.o' failed
  make[5]: *** [/tmp/build/perf/util/scripting-engines/trace-event-python.o] Error 1

Signed-off-by: Jin Yao 
Reviewed-by: Andi Kleen 
Cc: Alexander Shishkin 
Cc: Jin Yao 
Cc: Jiri Olsa 
Cc: Kan Liang 
Cc: Peter Zijlstra 
Link: http://lkml.kernel.org/r/1527843663-32288-3-git-send-email-yao.jin@linux.intel.com
Signed-off-by: Arnaldo Carvalho de Melo

perf script python: Move dsoname code to a new function

2018-06-06T15:52:09+00:00

This patch creates a new function get_dsoname() and move the code which
gets the dsoname string to this function.

That's because in next patch, when we process LBR data, we will also
need get_dsoname() to return dsoname for branch from/to.

Signed-off-by: Jin Yao 
Reviewed-by: Andi Kleen 
Cc: Alexander Shishkin 
Cc: Jiri Olsa 
Cc: Kan Liang 
Cc: Peter Zijlstra 
Link: http://lkml.kernel.org/r/1527843663-32288-2-git-send-email-yao.jin@linux.intel.com
Signed-off-by: Arnaldo Carvalho de Melo