linux.git/tools/testing/selftests/lib.mk, branch v4.14-rc2

selftests: lib.mk: change RUN_TESTS to print messages in TAP13 format

2017-08-25T16:17:12+00:00

Change common RUN_TESTS to print messages in user friendly TAP13 format.
This change add TAP13 header at the start of RUN_TESTS target run, and
prints the resulting pass/fail messages with test number information in
the TAP 13 format for each test in the run tests list.

This change covers test scripts as well as test programs. Test programs
have an option to use ksft_ API, however test scripts won't be able to.
With this change, test scripts can print TAP13 format output without any
changes to individual scripts.

Test programs can provide TAP13 format output as needed as some tests
already do. Tests that haven't been converted will benefit from this
change. Tests that are converted benefit from the test counts for all
the tests in each test directory.

Running firmware tests:
make --silent -C tools/testing/selftests/firmware/ run_tests

Before the change:

modprobe: ERROR: could not insert 'test_firmware': Operation not
permitted
./fw_filesystem.sh: /sys/devices/virtual/misc/test_firmware not present
You must have the following enabled in your kernel:
CONFIG_TEST_FIRMWARE=y
selftests:  fw_filesystem.sh [FAIL]
modprobe: ERROR: could not insert 'test_firmware': Operation not
permitted
selftests:  fw_fallback.sh [FAIL]

After the change:

TAP version 13
selftests: fw_filesystem.sh
========================================
modprobe: ERROR: could not insert 'test_firmware': Operation not
permitted
./fw_filesystem.sh: /sys/devices/virtual/misc/test_firmware not present
You must have the following enabled in your kernel:
CONFIG_TEST_FIRMWARE=y
not ok 1..1 selftests:  fw_filesystem.sh [FAIL]
selftests: fw_fallback.sh
========================================
modprobe: ERROR: could not insert 'test_firmware': Operation not
permitted
not ok 1..2 selftests:  fw_fallback.sh [FAIL]

Signed-off-by: Shuah Khan

selftests: change lib.mk RUN_TESTS to take test list as an argument

2017-08-25T16:15:24+00:00

Change lib.mk RUN_TESTS to take test list as an argument. This will
allow it to be called from individual test makefiles to run additional
tests that aren't suitable for a default kselftest run. As an example,
timers test includes destructive tests that aren't included in the
common run_tests target.

Change times/Makefile to use RUN_TESTS call with destructive test list
as an argument instead of using its own RUN_TESTS target.

Signed-off-by: Shuah Khan

selftests: lib.mk: suppress "cd" output from run_tests target

2017-08-25T16:07:45+00:00

Suppress "cd" output from run_tests while running tests to declutter the
test results.

Running efivarfs test:
make --silent -C tools/testing/selftests/efivarfs/ run_tests

Before the change:

skip all tests: must be run as root
selftests: efivarfs.sh [PASS]
/lkml/linux-kselftest/tools/testing/selftests/efivarfs

After the change:

skip all tests: must be run as root
selftests: efivarfs.sh [PASS]

Signed-off-by: Shuah Khan

selftests: warn if failure is due to lack of executable bit

2017-08-09T20:41:20+00:00

Executing selftests is fragile as if someone forgot to set a secript
as executable the test will fail, and you won't know for sure if the
failure was caused by the lack of proper permissions or something else.

Setting scripts as executable is required, this also enable folks to
execute selftests as independent units.

Signed-off-by: Luis R. Rodriguez 
Signed-off-by: Shuah Khan

selftests: lib.mk: define CLEAN macro to allow Makefiles to override clean

2017-04-25T15:46:45+00:00

Define CLEAN macro to allow Makefiles to override common clean target
in lib.mk. This will help fix the following failures:

warning: overriding recipe for target 'clean'
../lib.mk:55: warning: ignoring old recipe for target 'clean'

Fixes: 88baa78d1f31 ("selftests: remove duplicated all and clean target")

Signed-off-by: Shuah Khan 
Acked-by: Michael Ellerman

selftests: lib.mk Fix individual test builds

2017-03-02T14:53:01+00:00

In commit a8ba798bc8ec ("selftests: enable O and KBUILD_OUTPUT"), added
support to generate compile targets in a user specified directory. OUTPUT
variable controls the location which is undefined when tests are built in
the test directory or with "make -C tools/testing/selftests/x86".

make -C tools/testing/selftests/x86/
make: Entering directory '/lkml/linux_4.11/tools/testing/selftests/x86'
Makefile:44: warning: overriding recipe for target 'clean'
../lib.mk:51: warning: ignoring old recipe for target 'clean'
gcc -m64 -o /single_step_syscall_64 -O2 -g -std=gnu99 -pthread -Wall  single_step_syscall.c -lrt -ldl
/usr/bin/ld: cannot open output file /single_step_syscall_64: Permission denied
collect2: error: ld returned 1 exit status
Makefile:50: recipe for target '/single_step_syscall_64' failed
make: *** [/single_step_syscall_64] Error 1
make: Leaving directory '/lkml/linux_4.11/tools/testing/selftests/x86'

Same failure with "cd tools/testing/selftests/x86/;make" run.

Fix this with a change to lib.mk to define OUTPUT to be the pwd when
MAKELEVEL is 0. This covers both cases mentioned above.

Reported-by: Ingo Molnar 
Signed-off-by: Shuah Khan

selftests: Fix the .S and .S -> .o rules

2017-02-14T15:02:01+00:00

Both these rules incorrectly use $< (first prerequisite) rather than
$^ (all prerequisites), meaning they don't work if we're using more than
one .S file as input. Switch them to using $^.

They also don't include $(CPPFLAGS) and other variables used in the
default rules, which breaks targets that require those. Fix that by
using the builtin $(COMPILE.S) and $(LINK.S) rules.

Fixes: a8ba798bc8ec ("selftests: enable O and KBUILD_OUTPUT")
Signed-off-by: Michael Ellerman 
Tested by: Bamvor Jian Zhang 
Signed-off-by: Shuah Khan

selftests: Fix the .c linking rule

2017-02-14T15:01:55+00:00

Currently we can't build some tests, for example:

  $ make -C tools/testing/selftests/ TARGETS=vm
  ...
  gcc -Wall -I ../../../../usr/include   -lrt -lpthread ../../../../usr/include/linux/kernel.h userfaultfd.c -o tools/testing/selftests/vm/userfaultfd
  /tmp/ccmOkQSM.o: In function `stress':
  userfaultfd.c:(.text+0xc60): undefined reference to `pthread_create'
  userfaultfd.c:(.text+0xca5): undefined reference to `pthread_create'
  userfaultfd.c:(.text+0xcee): undefined reference to `pthread_create'
  userfaultfd.c:(.text+0xd30): undefined reference to `pthread_create'
  userfaultfd.c:(.text+0xd77): undefined reference to `pthread_join'
  userfaultfd.c:(.text+0xe7d): undefined reference to `pthread_join'
  userfaultfd.c:(.text+0xe9f): undefined reference to `pthread_cancel'
  userfaultfd.c:(.text+0xec6): undefined reference to `pthread_join'
  userfaultfd.c:(.text+0xf14): undefined reference to `pthread_join'
  /tmp/ccmOkQSM.o: In function `userfaultfd_stress':
  userfaultfd.c:(.text+0x13e2): undefined reference to `pthread_attr_setstacksize'
  collect2: error: ld returned 1 exit status

This is because the rule for linking .c files to binaries is incorrect.

The first bug is that it uses $< (first prerequisite) instead of $^ (all
preqrequisites), fix it by using ^$.

Secondly the ordering of the prerequisites vs $(LDLIBS) is wrong,
meaning on toolchains that use --as-needed we fail to link (as above).
Fix that by placing $(LDLIBS) *after* ^$.

Finally switch to using the default rule $(LINK.c), so that we get
$(CPPFLAGS) etc. included.

Fixes: a8ba798bc8ec ("selftests: enable O and KBUILD_OUTPUT")
Signed-off-by: Michael Ellerman 
Tested by: Bamvor Jian Zhang 
Signed-off-by: Shuah Khan

selftests: Fix selftests build to just build, not run tests

2017-02-14T15:01:44+00:00

In commit 88baa78d1f31 ("selftests: remove duplicated all and clean
target"), the "all" target was removed from individual Makefiles and
added to lib.mk.

However the "all" target was added to lib.mk *after* the existing
"runtests" target. This means "runtests" becomes the first (default)
target for most of our Makefiles.

This has the effect of causing a plain "make" to build *and run* the
tests. Which is at best rude, but depending on which tests are run could
oops someone's build machine.

  $ make -C tools/testing/selftests/
  ...
  make[1]: Entering directory 'tools/testing/selftests/bpf'
  gcc -Wall -O2 -I../../../../usr/include   test_verifier.c -o tools/testing/selftests/bpf/test_verifier
  gcc -Wall -O2 -I../../../../usr/include   test_maps.c -o tools/testing/selftests/bpf/test_maps
  gcc -Wall -O2 -I../../../../usr/include   test_lru_map.c -o tools/testing/selftests/bpf/test_lru_map
  #0 add+sub+mul FAIL
  Failed to load prog 'Function not implemented'!
  #1 unreachable FAIL
  Unexpected error message!
  #2 unreachable2 FAIL
  ...

Fix it by moving the "all" target to the start of lib.mk, making it the
default target.

Fixes: 88baa78d1f31 ("selftests: remove duplicated all and clean target")
Signed-off-by: Michael Ellerman 
Tested by: Bamvor Jian Zhang 
Signed-off-by: Shuah Khan

selftests: enable O and KBUILD_OUTPUT

2017-01-05T20:42:22+00:00

Enable O and KBUILD_OUTPUT for kselftest. User could compile kselftest
to another directory by passing O or KBUILD_OUTPUT. And O is high
priority than KBUILD_OUTPUT.

Signed-off-by: Bamvor Jian Zhang 
Signed-off-by: Shuah Khan