linux.git/lib/kunit/test.c, branch v6.8-rc2

kunit: Fix NULL-dereference in kunit_init_suite() if suite->log is NULL

2024-01-03T16:06:19+00:00

suite->log must be checked for NULL before passing it to
string_stream_clear(). This was done in kunit_init_test() but was missing
from kunit_init_suite().

Signed-off-by: Richard Fitzgerald 
Fixes: 6d696c4695c5 ("kunit: add ability to run tests after boot using debugfs")
Reviewed-by: Rae Moar 
Acked-by: David Gow 
Reviewed-by: Muhammad Usama Anjum 
Signed-off-by: Shuah Khan

kunit: Reset test->priv after each param iteration

2023-12-18T20:39:05+00:00

If we run parameterized test that uses test->priv to prepare some
custom data, then value of test->priv will leak to the next param
iteration and may be unexpected.  This could be easily seen if
we promote example_priv_test to parameterized test as then only
first test iteration will be successful:

$ ./tools/testing/kunit/kunit.py run \
	--kunitconfig ./lib/kunit/.kunitconfig *.example_priv*

[ ] Starting KUnit Kernel (1/1)...
[ ] ============================================================
[ ] =================== example (1 subtest) ====================
[ ] ==================== example_priv_test  ====================
[ ] [PASSED] example value 3
[ ] # example_priv_test: initializing
[ ] # example_priv_test: ASSERTION FAILED at lib/kunit/kunit-example-test.c:230
[ ] Expected test->priv == ((void *)0), but
[ ]     test->priv == 0000000060dfe290
[ ]     ((void *)0) == 0000000000000000
[ ] # example_priv_test: cleaning up
[ ] [FAILED] example value 2
[ ] # example_priv_test: initializing
[ ] # example_priv_test: ASSERTION FAILED at lib/kunit/kunit-example-test.c:230
[ ] Expected test->priv == ((void *)0), but
[ ]     test->priv == 0000000060dfe290
[ ]     ((void *)0) == 0000000000000000
[ ] # example_priv_test: cleaning up
[ ] [FAILED] example value 1
[ ] # example_priv_test: initializing
[ ] # example_priv_test: ASSERTION FAILED at lib/kunit/kunit-example-test.c:230
[ ] Expected test->priv == ((void *)0), but
[ ]     test->priv == 0000000060dfe290
[ ]     ((void *)0) == 0000000000000000
[ ] # example_priv_test: cleaning up
[ ] [FAILED] example value 0
[ ] # example_priv_test: initializing
[ ] # example_priv_test: cleaning up
[ ] # example_priv_test: pass:1 fail:3 skip:0 total:4
[ ] ================ [FAILED] example_priv_test ================
[ ]     # example: initializing suite
[ ]     # module: kunit_example_test
[ ]     # example: exiting suite
[ ] # Totals: pass:1 fail:3 skip:0 total:4
[ ] ===================== [FAILED] example =====================

Fix that by resetting test->priv after each param iteration, in
similar way what we did for the test->status.

Signed-off-by: Michal Wajdeczko 
Cc: David Gow 
Cc: Rae Moar 
Reviewed-by: David Gow 
Signed-off-by: Shuah Khan

kunit: Add APIs for managing devices

2023-12-18T20:28:08+00:00

Tests for drivers often require a struct device to pass to other
functions. While it's possible to create these with
root_device_register(), or to use something like a platform device, this
is both a misuse of those APIs, and can be difficult to clean up after,
for example, a failed assertion.

Add some KUnit-specific functions for registering and unregistering a
struct device:
- kunit_device_register()
- kunit_device_register_with_driver()
- kunit_device_unregister()

These helpers allocate a on a 'kunit' bus which will either probe the
driver passed in (kunit_device_register_with_driver), or will create a
stub driver (kunit_device_register) which is cleaned up on test shutdown.

Devices are automatically unregistered on test shutdown, but can be
manually unregistered earlier with kunit_device_unregister() in order
to, for example, test device release code.

Reviewed-by: Matti Vaittinen 
Reviewed-by: Maxime Ripard 
Signed-off-by: David Gow 
Reviewed-by: Greg Kroah-Hartman 
Signed-off-by: Shuah Khan

kunit: add ability to run tests after boot using debugfs

2023-12-18T20:25:49+00:00

Add functionality to run built-in tests after boot by writing to a
debugfs file.

Add a new debugfs file labeled "run" for each test suite to use for
this purpose.

As an example, write to the file using the following:

echo "any string" > /sys/kernel/debugfs/kunit//run

This will trigger the test suite to run and will print results to the
kernel log.

To guard against running tests concurrently with this feature, add a
mutex lock around running kunit. This supports the current practice of
not allowing tests to be run concurrently on the same kernel.

This new functionality could be used to design a parameter
injection feature in the future.

Fixed up merge conflict duing rebase to Linux 6.7-rc6
Signed-off-by: Shuah Khan 

Reviewed-by: David Gow 
Signed-off-by: Rae Moar 
Signed-off-by: Shuah Khan

kunit: add KUNIT_INIT_TABLE to init linker section

2023-12-18T20:21:15+00:00

Add KUNIT_INIT_TABLE to the INIT_DATA linker section.

Alter the KUnit macros to create init tests:
kunit_test_init_section_suites

Update lib/kunit/executor.c to run both the suites in KUNIT_TABLE and
KUNIT_INIT_TABLE.

Reviewed-by: David Gow 
Signed-off-by: Rae Moar 
Signed-off-by: Shuah Khan

kunit: Add a macro to wrap a deferred action function

2023-12-18T20:21:14+00:00

KUnit's deferred action API accepts a void(*)(void *) function pointer
which is called when the test is exited. However, we very frequently
want to use existing functions which accept a single pointer, but which
may not be of type void*. While this is probably dodgy enough to be on
the wrong side of the C standard, it's been often used for similar
callbacks, and gcc's -Wcast-function-type seems to ignore cases where
the only difference is the type of the argument, assuming it's
compatible (i.e., they're both pointers to data).

However, clang 16 has introduced -Wcast-function-type-strict, which no
longer permits any deviation in function pointer type. This seems to be
because it'd break CFI, which validates the type of function calls.

This rather ruins our attempts to cast functions to defer them, and
leaves us with a few options. The one we've chosen is to implement a
macro which will generate a wrapper function which accepts a void*, and
casts the argument to the appropriate type.

For example, if you were trying to wrap:
void foo_close(struct foo *handle);
you could use:
KUNIT_DEFINE_ACTION_WRAPPER(kunit_action_foo_close,
			    foo_close,
			    struct foo *);

This would create a new kunit_action_foo_close() function, of type
kunit_action_t, which could be passed into kunit_add_action() and
similar functions.

In addition to defining this macro, update KUnit and its tests to use
it.

Link: https://github.com/ClangBuiltLinux/linux/issues/1750
Reviewed-by: Nathan Chancellor 
Tested-by: Nathan Chancellor 
Acked-by: Daniel Vetter 
Reviewed-by: Maxime Ripard 
Signed-off-by: David Gow 
Signed-off-by: Shuah Khan

kunit: Reset suite counter right before running tests

2023-11-14T20:01:47+00:00

Today we reset the suite counter as part of the suite cleanup,
called from the module exit callback, but it might not work that
well as one can try to collect results without unloading a previous
test (either unintentionally or due to dependencies).

For easy reproduction try to load the kunit-test.ko and then
collect and parse results from the kunit-example-test.ko load.
Parser will complain about mismatch of expected test number:

[ ] KTAP version 1
[ ] 1..1
[ ]     # example: initializing suite
[ ]     KTAP version 1
[ ]     # Subtest: example
..
[ ] # example: pass:5 fail:0 skip:4 total:9
[ ] # Totals: pass:6 fail:0 skip:6 total:12
[ ] ok 7 example

[ ] [ERROR] Test: example: Expected test number 1 but found 7
[ ] ===================== [PASSED] example =====================
[ ] ============================================================
[ ] Testing complete. Ran 12 tests: passed: 6, skipped: 6, errors: 1

Since we are now printing suite test plan on every module load,
right before running suite tests, we should make sure that suite
counter will also start from 1. Easiest solution seems to be move
counter reset to the __kunit_test_suites_init() function.

Signed-off-by: Michal Wajdeczko 
Cc: David Gow 
Cc: Rae Moar 
Reviewed-by: David Gow 
Signed-off-by: Shuah Khan

kunit: Warn if tests are slow

2023-11-14T20:01:37+00:00

Kunit recently gained support to setup attributes, the first one being
the speed of a given test, then allowing to filter out slow tests.

A slow test is defined in the documentation as taking more than one
second. There's an another speed attribute called "super slow" but whose
definition is less clear.

Add support to the test runner to check the test execution time, and
report tests that should be marked as slow but aren't.

Signed-off-by: Maxime Ripard 
Reviewed-by: David Gow 
Signed-off-by: Shuah Khan

kunit: Reset test status on each param iteration

2023-09-18T16:46:56+00:00

If we skip one parametrized test case then test status remains
SKIP for all subsequent test params leading to wrong reports:

$ ./tools/testing/kunit/kunit.py run \
	--kunitconfig ./lib/kunit/.kunitconfig *.example_params*
	--raw_output \

[ ] Starting KUnit Kernel (1/1)...
KTAP version 1
1..1
    # example: initializing suite
    KTAP version 1
    # Subtest: example
    # module: kunit_example_test
    1..1
        KTAP version 1
        # Subtest: example_params_test
    # example_params_test: initializing
    # example_params_test: cleaning up
        ok 1 example value 3 # SKIP unsupported param value 3
    # example_params_test: initializing
    # example_params_test: cleaning up
        ok 2 example value 2 # SKIP unsupported param value 3
    # example_params_test: initializing
    # example_params_test: cleaning up
        ok 3 example value 1 # SKIP unsupported param value 3
    # example_params_test: initializing
    # example_params_test: cleaning up
        ok 4 example value 0 # SKIP unsupported param value 0
    # example_params_test: pass:0 fail:0 skip:4 total:4
    ok 1 example_params_test # SKIP unsupported param value 0
    # example: exiting suite
ok 1 example # SKIP

Reset test status and status comment after each param iteration
to avoid using stale results.

Signed-off-by: Michal Wajdeczko 
Cc: David Gow 
Cc: Rae Moar 
Reviewed-by: David Gow 
Signed-off-by: Shuah Khan

kunit: Use string_stream for test log

2023-09-18T16:45:53+00:00

Replace the fixed-size log buffer with a string_stream so that the
log can grow as lines are added.

The existing kunit log tests have been updated for using a
string_stream as the log. No new test have been added because there
are already tests for the underlying string_stream.

As the log tests now depend on string_stream functions they cannot
build when kunit-test is a module. They have been surrounded by
a #if to replace them with skipping version when the test is
build as a module. Though this isn't pretty, it avoids moving
code to another file while that code is also being changed.

Signed-off-by: Richard Fitzgerald 
Reviewed-by: David Gow 
Signed-off-by: Shuah Khan