linux.git/Documentation/dev-tools/kunit/usage.rst, branch v6.3 Linux kernel source tree kunit: Add "hooks" to call into KUnit when it's built as a module 2023-02-08T21:26:25+00:00 David Gow davidgow@google.com 2023-01-28T07:10:07+00:00 7170b7ed6acbde523c5d362c8978c60df4c30f30 KUnit has several macros and functions intended for use from non-test code. These hooks, currently the kunit_get_current_test() and kunit_fail_current_test() macros, didn't work when CONFIG_KUNIT=m. In order to support this case, the required functions and static data need to be available unconditionally, even when KUnit itself is not built-in. The new 'hooks.c' file is therefore always included, and has both the static key required for kunit_get_current_test(), and a table of function pointers in struct kunit_hooks_table. This is filled in with the real implementations by kunit_install_hooks(), which is kept in hooks-impl.h and called when the kunit module is loaded. This can be extended for future features which require similar "hook" behaviour, such as static stubs, by simply adding new entries to the struct, and the appropriate code to set them. Fixed white-space errors during commit: Shuah Khan <skhan@linuxfoundation.org> Resolved merge conflicts with: db105c37a4d6 ("kunit: Export kunit_running()") This patch supersedes the above. Shuah Khan <skhan@linuxfoundation.org> Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Rae Moar <rmoar@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
KUnit has several macros and functions intended for use from non-test
code. These hooks, currently the kunit_get_current_test() and
kunit_fail_current_test() macros, didn't work when CONFIG_KUNIT=m.

In order to support this case, the required functions and static data
need to be available unconditionally, even when KUnit itself is not
built-in. The new 'hooks.c' file is therefore always included, and has
both the static key required for kunit_get_current_test(), and a table
of function pointers in struct kunit_hooks_table. This is filled in with
the real implementations by kunit_install_hooks(), which is kept in
hooks-impl.h and called when the kunit module is loaded.

This can  be extended for future features which require similar
"hook" behaviour, such as static stubs, by simply adding new entries to
the struct, and the appropriate code to set them.

Fixed white-space errors during commit:
Shuah Khan <skhan@linuxfoundation.org>

Resolved merge conflicts with:
db105c37a4d6 ("kunit: Export kunit_running()")
This patch supersedes the above.
Shuah Khan <skhan@linuxfoundation.org>

Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kunit: Use the static key when retrieving the current test 2022-12-12T21:13:47+00:00 David Gow davidgow@google.com 2022-11-25T08:43:05+00:00 91e93592219f74c4d5cd4f27006d726ac86ae15d In order to detect if a KUnit test is running, and to access its context, the 'kunit_test' member of the current task_struct is used. Usually, this is accessed directly or via the kunit_fail_current_task() function. In order to speed up the case where no test is running, add a wrapper, kunit_get_current_test(), which uses the static key to fail early. Equally, Speed up kunit_fail_current_test() by using the static key. This should make it convenient for code to call this unconditionally in fakes or error paths, without worrying that this will slow the code down significantly. If CONFIG_KUNIT=n (or m), this compiles away to nothing. If CONFIG_KUNIT=y, it will compile down to a NOP (on most architectures) if no KUnit test is currently running. Note that kunit_get_current_test() does not work if KUnit is built as a module. This mirrors the existing restriction on kunit_fail_current_test(). Note that the definition of kunit_fail_current_test() still wraps an empty, inline function if KUnit is not built-in. This is to ensure that the printf format string __attribute__ will still work. Also update the documentation to suggest users use the new kunit_get_current_test() function, update the example, and to describe the behaviour when KUnit is disabled better. Cc: Jonathan Corbet <corbet@lwn.net> Cc: Sadiya Kazi <sadiyakazi@google.com> Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Daniel Latypov <dlatypov@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
In order to detect if a KUnit test is running, and to access its
context, the 'kunit_test' member of the current task_struct is used.
Usually, this is accessed directly or via the kunit_fail_current_task()
function.

In order to speed up the case where no test is running, add a wrapper,
kunit_get_current_test(), which uses the static key to fail early.
Equally, Speed up kunit_fail_current_test() by using the static key.

This should make it convenient for code to call this
unconditionally in fakes or error paths, without worrying that this will
slow the code down significantly.

If CONFIG_KUNIT=n (or m), this compiles away to nothing. If
CONFIG_KUNIT=y, it will compile down to a NOP (on most architectures) if
no KUnit test is currently running.

Note that kunit_get_current_test() does not work if KUnit is built as a
module. This mirrors the existing restriction on kunit_fail_current_test().

Note that the definition of kunit_fail_current_test() still wraps an
empty, inline function if KUnit is not built-in. This is to ensure that
the printf format string __attribute__ will still work.

Also update the documentation to suggest users use the new
kunit_get_current_test() function, update the example, and to describe
the behaviour when KUnit is disabled better.

Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Sadiya Kazi <sadiyakazi@google.com>
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Documentation: KUnit: reword description of assertions 2022-12-12T21:13:47+00:00 Daniel Latypov dlatypov@google.com 2022-11-11T18:29:05+00:00 ec0a42a17e463ee5b1ebd2d60337e8ae8e5ace2b The existing wording implies that kunit_kmalloc_array() is "the method under test". We're actually testing the sort() function in that example. This is because the example was changed in commit 953574390634 ("Documentation: KUnit: Rework writing page to focus on writing tests"), but the wording was not. Also add a `note` telling people they can use the KUNIT_ASSERT_EQ() macros from any function. Some users might be coming from a framework like gUnit where that'll compile but silently do the wrong thing. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Sadiya Kazi <sadiyakazi@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
The existing wording implies that kunit_kmalloc_array() is "the method
under test". We're actually testing the sort() function in that example.
This is because the example was changed in commit 953574390634
("Documentation: KUnit: Rework writing page to focus on writing tests"),
but the wording was not.

Also add a `note` telling people they can use the KUNIT_ASSERT_EQ()
macros from any function. Some users might be coming from a framework
like gUnit where that'll compile but silently do the wrong thing.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Sadiya Kazi <sadiyakazi@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Documentation: KUnit: make usage.rst a superset of tips.rst, remove duplication 2022-12-12T21:13:47+00:00 Daniel Latypov dlatypov@google.com 2022-11-11T18:29:04+00:00 65c48a48ead042856525b92cedf673d2bf5bdfc9 usage.rst had most of the content of the tips.rst page copied over. But it's missing https://www.kernel.org/doc/html/v6.0/dev-tools/kunit/tips.html#customizing-error-messages Copy it over so we can retire tips.rst w/o losing content. And in that process, it also gained a duplicate section about how KUNIT_ASSERT_*() exit the test case early. Remove that. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Sadiya Kazi <sadiyakazi@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
usage.rst had most of the content of the tips.rst page copied over.
But it's missing https://www.kernel.org/doc/html/v6.0/dev-tools/kunit/tips.html#customizing-error-messages
Copy it over so we can retire tips.rst w/o losing content.

And in that process, it also gained a duplicate section about how
KUNIT_ASSERT_*() exit the test case early. Remove that.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Sadiya Kazi <sadiyakazi@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Documentation: KUnit: Fix non-uml anchor 2022-09-30T19:19:03+00:00 Tales Aparecida tales.aparecida@gmail.com 2022-08-13T04:20:54+00:00 43ca52a910d15026b474160c2b625db5be687f51 The section was rewritten but its anchor got left behind. Fix the anchor and add some references to running on QEMU. Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
The section was rewritten but its anchor got left behind.
Fix the anchor and add some references to running on QEMU.

Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Documentation: KUnit: Fix example with compilation error 2022-07-28T19:06:35+00:00 Maíra Canal mairacanal@riseup.net 2022-07-20T18:57:19+00:00 4c392516accfe51a1aaf80ed163517646f8f0476 The Parameterized Testing example contains a compilation error, as the signature for the description helper function is void(*)(const struct sha1_test_case *, char *), and the struct is non-const. This is warned by Clang: error: initialization of ‘void (*)(struct sha1_test_case *, char *)’ from incompatible pointer type ‘void (*)(const struct sha1_test_case *, char *)’ [-Werror=incompatible-pointer-types] 33 | KUNIT_ARRAY_PARAM(sha1, cases, case_to_desc); | ^~~~~~~~~~~~ ../include/kunit/test.h:1339:70: note: in definition of macro ‘KUNIT_ARRAY_PARAM’ 1339 | void (*__get_desc)(typeof(__next), char *) = get_desc; \ Signed-off-by: Maíra Canal <mairacanal@riseup.net> Reviewed-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
The Parameterized Testing example contains a compilation error, as the
signature for the description helper function is void(*)(const struct
sha1_test_case *, char *), and the struct is non-const. This is
warned by Clang:

error: initialization of ‘void (*)(struct sha1_test_case *, char *)’
from incompatible pointer type ‘void (*)(const struct sha1_test_case *,
char *)’ [-Werror=incompatible-pointer-types]
33 | KUNIT_ARRAY_PARAM(sha1, cases, case_to_desc);
   |                                ^~~~~~~~~~~~
../include/kunit/test.h:1339:70: note: in definition of macro
‘KUNIT_ARRAY_PARAM’
1339 |                         void
   (*__get_desc)(typeof(__next), char *) = get_desc; \

Signed-off-by: Maíra Canal <mairacanal@riseup.net>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kunit: add ability to specify suite-level init and exit functions 2022-05-02T18:35:51+00:00 Daniel Latypov dlatypov@google.com 2022-04-29T18:12:57+00:00 1cdba21db2ca31514c60b9732fc3963ae24c59e0 KUnit has support for setup/cleanup logic for each test case in a suite. But it lacks the ability to specify setup/cleanup for the entire suite itself. This can be used to do setup that is too expensive or cumbersome to do for each test. Or it can be used to do simpler things like log debug information after the suite completes. It's a fairly common feature, so the lack of it is noticeable. Some examples in other frameworks and languages: * https://docs.python.org/3/library/unittest.html#setupclass-and-teardownclass * https://google.github.io/googletest/reference/testing.html#Test::SetUpTestSuite Meta: This is very similar to this patch here: https://lore.kernel.org/linux-kselftest/20210805043503.20252-3-bvanassche@acm.org/ The changes from that patch: * pass in `struct kunit *` so users can do stuff like `kunit_info(suite, "debug message")` * makes sure the init failure is bubbled up as a failure * updates kunit-example-test.c to use a suite init * Updates kunit/usage.rst to mention the new support * some minor cosmetic things * use `suite_{init,exit}` instead of `{init/exit}_suite` * make suite init error message more consistent w/ test init * etc. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
KUnit has support for setup/cleanup logic for each test case in a suite.
But it lacks the ability to specify setup/cleanup for the entire suite
itself.

This can be used to do setup that is too expensive or cumbersome to do
for each test.
Or it can be used to do simpler things like log debug information after
the suite completes.
It's a fairly common feature, so the lack of it is noticeable.

Some examples in other frameworks and languages:
* https://docs.python.org/3/library/unittest.html#setupclass-and-teardownclass
* https://google.github.io/googletest/reference/testing.html#Test::SetUpTestSuite

Meta:
This is very similar to this patch here: https://lore.kernel.org/linux-kselftest/20210805043503.20252-3-bvanassche@acm.org/
The changes from that patch:
* pass in `struct kunit *` so users can do stuff like
  `kunit_info(suite, "debug message")`
* makes sure the init failure is bubbled up as a failure
* updates kunit-example-test.c to use a suite init
* Updates kunit/usage.rst to mention the new support
* some minor cosmetic things
  * use `suite_{init,exit}` instead of `{init/exit}_suite`
  * make suite init error message more consistent w/ test init
  * etc.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Documentation: KUnit: Fix usage bug 2022-02-08T20:16:20+00:00 Akira Kawata akirakawata1@gmail.com 2022-02-07T11:20:44+00:00 92a68053c3468705e2c7c752c9a3f256304a35a6 Fix a bug of kunit documentation. Link: https://bugzilla.kernel.org/show_bug.cgi?id=205773 : Quoting Steve Pfetsch: : : kunit documentation is incorrect: : https://kunit.dev/third_party/stable_kernel/docs/usage.html : struct rectangle *self = container_of(this, struct shape, parent); : : : Shouldn't it be: : struct rectangle *self = container_of(this, struct rectangle, parent); : ? Signed-off-by: Akira Kawata <akirakawata1@gmail.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
Fix a bug of kunit documentation.
Link: https://bugzilla.kernel.org/show_bug.cgi?id=205773

: Quoting Steve Pfetsch:
:
: kunit documentation is incorrect:
: https://kunit.dev/third_party/stable_kernel/docs/usage.html
: struct rectangle *self = container_of(this, struct shape, parent);
:
:
: Shouldn't it be:
: struct rectangle *self = container_of(this, struct rectangle, parent);
: ?

Signed-off-by: Akira Kawata <akirakawata1@gmail.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Documentation: KUnit: Rework writing page to focus on writing tests 2021-12-23T19:49:43+00:00 Harinder Singh sharinder@google.com 2021-12-17T04:49:09+00:00 9535743906345f02f7bea2469a2137509db0ac4c We now have dedicated pages on running tests. Therefore refocus the usage page on writing tests and add content from tips page and information on other architectures. Signed-off-by: Harinder Singh <sharinder@google.com> Reviewed-by: Tim Bird <tim.bird@sony.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Link: https://lore.kernel.org/r/20211217044911.798817-6-sharinder@google.com Signed-off-by: Jonathan Corbet <corbet@lwn.net> 
We now have dedicated pages on running tests. Therefore refocus the
usage page on writing tests and add content from tips page and
information on other architectures.

Signed-off-by: Harinder Singh <sharinder@google.com>
Reviewed-by: Tim Bird <tim.bird@sony.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Link: https://lore.kernel.org/r/20211217044911.798817-6-sharinder@google.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Merge tag 'linux-kselftest-kunit-fixes-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest 2021-07-02T19:58:26+00:00 Linus Torvalds torvalds@linux-foundation.org 2021-07-02T19:58:26+00:00 a48ad6e7a35dc3f3b521249204daf4c9427628e5 Pull KUnit update from Shuah Khan: "Fixes and features: - add support for skipped tests - introduce kunit_kmalloc_array/kunit_kcalloc() helpers - add gnu_printf specifiers - add kunit_shutdown - add unit test for filtering suites by names - convert lib/test_list_sort.c to use KUnit - code organization moving default config to tools/testing/kunit - refactor of internal parser input handling - cleanups and updates to documentation - code cleanup related to casts" * tag 'linux-kselftest-kunit-fixes-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: (29 commits) kunit: add unit test for filtering suites by names kasan: test: make use of kunit_skip() kunit: test: Add example tests which are always skipped kunit: tool: Support skipped tests in kunit_tool kunit: Support skipped tests thunderbolt: test: Reinstate a few casts of bitfields kunit: tool: internal refactor of parser input handling lib/test: convert lib/test_list_sort.c to use KUnit kunit: introduce kunit_kmalloc_array/kunit_kcalloc() helpers kunit: Remove the unused all_tests.config kunit: Move default config from arch/um -> tools/testing/kunit kunit: arch/um/configs: Enable KUNIT_ALL_TESTS by default kunit: Add gnu_printf specifiers lib/cmdline_kunit: Remove a cast which are no-longer required kernel/sysctl-test: Remove some casts which are no-longer required thunderbolt: test: Remove some casts which are no longer required mmc: sdhci-of-aspeed: Remove some unnecessary casts from KUnit tests iio: Remove a cast in iio-test-format which is no longer required device property: Remove some casts in property-entry-test Documentation: kunit: Clean up some string casts in examples ... 
Pull KUnit update from Shuah Khan:
 "Fixes and features:

   - add support for skipped tests

   - introduce kunit_kmalloc_array/kunit_kcalloc() helpers

   - add gnu_printf specifiers

   - add kunit_shutdown

   - add unit test for filtering suites by names

   - convert lib/test_list_sort.c to use KUnit

   - code organization moving default config to tools/testing/kunit

   - refactor of internal parser input handling

   - cleanups and updates to documentation

   - code cleanup related to casts"

* tag 'linux-kselftest-kunit-fixes-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: (29 commits)
  kunit: add unit test for filtering suites by names
  kasan: test: make use of kunit_skip()
  kunit: test: Add example tests which are always skipped
  kunit: tool: Support skipped tests in kunit_tool
  kunit: Support skipped tests
  thunderbolt: test: Reinstate a few casts of bitfields
  kunit: tool: internal refactor of parser input handling
  lib/test: convert lib/test_list_sort.c to use KUnit
  kunit: introduce kunit_kmalloc_array/kunit_kcalloc() helpers
  kunit: Remove the unused all_tests.config
  kunit: Move default config from arch/um -> tools/testing/kunit
  kunit: arch/um/configs: Enable KUNIT_ALL_TESTS by default
  kunit: Add gnu_printf specifiers
  lib/cmdline_kunit: Remove a cast which are no-longer required
  kernel/sysctl-test: Remove some casts which are no-longer required
  thunderbolt: test: Remove some casts which are no longer required
  mmc: sdhci-of-aspeed: Remove some unnecessary casts from KUnit tests
  iio: Remove a cast in iio-test-format which is no longer required
  device property: Remove some casts in property-entry-test
  Documentation: kunit: Clean up some string casts in examples
  ...