linux.git/tools/testing/selftests/kselftest, branch for-next Linux kernel source tree selftests: support interpreted scripts with ksft_runner.sh 2024-09-10T00:38:02+00:00 Willem de Bruijn willemb@google.com 2024-09-05T23:15:51+00:00 dbd61921a6adbd231d8ca91537f1e7b62e9fea61 Support testcases that are themselves not executable, but need an interpreter to run them. If a test file is not executable, but an executable file ksft_runner.sh exists in the TARGET dir, kselftest will run ./ksft_runner.sh ./$BASENAME_TEST Packetdrill may add hundreds of packetdrill scripts for testing. These scripts must be passed to the packetdrill process. Have kselftest run each test directly, as it already solves common runner requirements like parallel execution and isolation (netns). A previous RFC added a wrapper in between, which would have to reimplement such functionality. Link: https://lore.kernel.org/netdev/66d4d97a4cac_3df182941a@willemb.c.googlers.com.notmuch/T/ Signed-off-by: Willem de Bruijn <willemb@google.com> Link: https://patch.msgid.link/20240905231653.2427327-2-willemdebruijn.kernel@gmail.com Signed-off-by: Jakub Kicinski <kuba@kernel.org> 
Support testcases that are themselves not executable, but need an
interpreter to run them.

If a test file is not executable, but an executable file
ksft_runner.sh exists in the TARGET dir, kselftest will run

    ./ksft_runner.sh ./$BASENAME_TEST

Packetdrill may add hundreds of packetdrill scripts for testing. These
scripts must be passed to the packetdrill process.

Have kselftest run each test directly, as it already solves common
runner requirements like parallel execution and isolation (netns).
A previous RFC added a wrapper in between, which would have to
reimplement such functionality.

Link: https://lore.kernel.org/netdev/66d4d97a4cac_3df182941a@willemb.c.googlers.com.notmuch/T/
Signed-off-by: Willem de Bruijn <willemb@google.com>
Link: https://patch.msgid.link/20240905231653.2427327-2-willemdebruijn.kernel@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
selftests: ksft: Fix finished() helper exit code on skipped tests 2024-07-31T17:38:56+00:00 Laura Nao laura.nao@collabora.com 2024-07-30T10:29:28+00:00 170c966cbe274e664288cfc12ee919d5e706dc50 The Python finished() helper currently exits with KSFT_FAIL when there are only passed and skipped tests. Fix the logic to exit with KSFT_PASS instead, making it consistent with its C and bash counterparts (ksft_finished() and ktap_finished() respectively). Reviewed-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> Fixes: dacf1d7a78bf ("kselftest: Add test to verify probe of devices from discoverable buses") Signed-off-by: Laura Nao <laura.nao@collabora.com> Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
The Python finished() helper currently exits with KSFT_FAIL when there
are only passed and skipped tests. Fix the logic to exit with KSFT_PASS
instead, making it consistent with its C and bash counterparts
(ksft_finished() and ktap_finished() respectively).

Reviewed-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
Fixes: dacf1d7a78bf ("kselftest: Add test to verify probe of devices from discoverable buses")
Signed-off-by: Laura Nao <laura.nao@collabora.com>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kselftest: Move ksft helper module to common directory 2024-07-12T08:59:06+00:00 Nícolas F. R. A. Prado nfraprado@collabora.com 2024-07-05T23:29:55+00:00 0e7b7bde460304f44e8c6b212c3195ac2f69f6fe Move the ksft python module, which provides generic helpers for kselftests, to a common directory so it can be more easily shared between different tests. Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> Acked-by: Shuah Khan <skhan@linuxfoundation.org> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Link: https://lore.kernel.org/r/20240705-dev-err-log-selftest-v2-2-163b9cd7b3c1@collabora.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
Move the ksft python module, which provides generic helpers for
kselftests, to a common directory so it can be more easily shared
between different tests.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/r/20240705-dev-err-log-selftest-v2-2-163b9cd7b3c1@collabora.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
selftests: ktap_helpers: Make it POSIX-compliant 2024-05-06T19:57:20+00:00 Nícolas F. R. A. Prado nfraprado@collabora.com 2024-04-15T15:32:15+00:00 45d5a2b1886a3ff0fe5627ebee84c089db7ff5f2 There are a couple uses of bash specific syntax in the script. Change them to the equivalent POSIX syntax. This doesn't change functionality and allows non-bash test scripts to make use of these helpers. Reported-by: Mike Looijmans <mike.looijmans@topic.nl> Closes: https://lore.kernel.org/all/efae4037-c22a-40be-8ba9-7c1c12ece042@topic.nl/ Fixes: 2dd0b5a8fcc4 ("selftests: ktap_helpers: Add a helper to finish the test") Fixes: 14571ab1ad21 ("kselftest: Add new test for detecting unprobed Devicetree devices") Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
There are a couple uses of bash specific syntax in the script. Change
them to the equivalent POSIX syntax. This doesn't change functionality
and allows non-bash test scripts to make use of these helpers.

Reported-by: Mike Looijmans <mike.looijmans@topic.nl>
Closes: https://lore.kernel.org/all/efae4037-c22a-40be-8ba9-7c1c12ece042@topic.nl/
Fixes: 2dd0b5a8fcc4 ("selftests: ktap_helpers: Add a helper to finish the test")
Fixes: 14571ab1ad21 ("kselftest: Add new test for detecting unprobed Devicetree devices")
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
selftests: ktap_helpers: Add a helper to finish the test 2024-02-20T22:53:20+00:00 Nícolas F. R. A. Prado nfraprado@collabora.com 2024-01-31T22:02:43+00:00 2dd0b5a8fcc4f6fef307c79a3ff1d91d9595004d Similar to the C counterpart, keep track of the number of test cases in the test plan and add a helper function to be called at the end of the test to print the results and exit with the corresponding exit code. Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
Similar to the C counterpart, keep track of the number of test cases in
the test plan and add a helper function to be called at the end of the
test to print the results and exit with the corresponding exit code.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
selftests: ktap_helpers: Add a helper to abort the test 2024-02-20T22:53:15+00:00 Nícolas F. R. A. Prado nfraprado@collabora.com 2024-01-31T22:02:42+00:00 d63fde98b8929da625cc36c2f1143e837c0e6d10 Similar to the C counterpart, add a helper function to abort the remainder of the test. Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
Similar to the C counterpart, add a helper function to abort the
remainder of the test.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
selftests: ktap_helpers: Add helper to pass/fail test based on exit code 2024-02-20T22:53:08+00:00 Nícolas F. R. A. Prado nfraprado@collabora.com 2024-01-31T22:02:41+00:00 d90b7c705c5d9b5cb6b71b8aaa773c031084056c Similar to the C counterpart, add a helper function that runs a command and passes or fails the test based on the result. Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
Similar to the C counterpart, add a helper function that runs a command
and passes or fails the test based on the result.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
selftests: ktap_helpers: Add helper to print diagnostic messages 2024-02-20T22:53:02+00:00 Nícolas F. R. A. Prado nfraprado@collabora.com 2024-01-31T22:02:40+00:00 6934eea2690b3c8a95224f2d1dacc2f13d3bf4ce Similar to the C counterpart, add a helper to print a diagnostic message. Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
Similar to the C counterpart, add a helper to print a diagnostic
message.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
selftests: Move KTAP bash helpers to selftests common folder 2024-02-20T22:52:55+00:00 Laura Nao laura.nao@collabora.com 2024-01-02T14:15:28+00:00 7c079e909b21f8f6e5d8e501c037752ff302a15e Move bash helpers for outputting in KTAP format to the common selftests folder. This allows kselftests other than the dt one to source the file and make use of the helper functions. Define pass, fail and skip codes in the same file too. Signed-off-by: Laura Nao <laura.nao@collabora.com> Reviewed-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> Tested-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> Acked-by: Rob Herring <robh@kernel.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
Move bash helpers for outputting in KTAP format to the common selftests
folder. This allows kselftests other than the dt one to source the file
and make use of the helper functions.
Define pass, fail and skip codes in the same file too.

Signed-off-by: Laura Nao <laura.nao@collabora.com>
Reviewed-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
Tested-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kselftest/runner.sh: add netns support 2023-12-23T00:26:32+00:00 Hangbin Liu liuhangbin@gmail.com 2023-12-19T09:48:56+00:00 9d0b4ad82d6117e6d7ead50f64be54ec782aa1fe Add a variable RUN_IN_NETNS if the user wants to run all the selected tests in namespace in parallel. With this, we can save a lot of testing time. Note that some tests may not fit to run in namespace, e.g. net/drop_monitor_tests.sh, as the dwdump needs to be run in init ns. I also added another parameter -p to make all the logs reported separately instead of mixing them in the stdout or output.log. Nit: the NUM in run_one is not used, rename it to test_num. Acked-by: David Ahern <dsahern@kernel.org> Signed-off-by: Hangbin Liu <liuhangbin@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net> 
Add a variable RUN_IN_NETNS if the user wants to run all the selected tests
in namespace in parallel. With this, we can save a lot of testing time.

Note that some tests may not fit to run in namespace, e.g.
net/drop_monitor_tests.sh, as the dwdump needs to be run in init ns.

I also added another parameter -p to make all the logs reported separately
instead of mixing them in the stdout or output.log.

Nit: the NUM in run_one is not used, rename it to test_num.

Acked-by: David Ahern <dsahern@kernel.org>
Signed-off-by: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>