linux-stable.git/lib/kunit/test.c, branch v6.3.2

kunit: fix bug in the order of lines in debugfs logs

2023-05-11T14:16:56+00:00

[ Upstream commit f9a301c3317daa921375da0aec82462ddf019928 ]

Fix bug in debugfs logs that causes an incorrect order of lines in the
debugfs log.

Currently, the test counts lines that show the number of tests passed,
failed, and skipped, as well as any suite diagnostic lines,
appear prior to the individual results, which is a bug.

Ensure the order of printing for the debugfs log is correct. Additionally,
add a KTAP header to so the debugfs logs can be valid KTAP.

This is an example of a log prior to these fixes:

     KTAP version 1

     # Subtest: kunit_status
     1..2
 # kunit_status: pass:2 fail:0 skip:0 total:2
 # Totals: pass:2 fail:0 skip:0 total:2
     ok 1 kunit_status_set_failure_test
     ok 2 kunit_status_mark_skipped_test
 ok 1 kunit_status

Note the two lines with stats are out of order. This is the same debugfs
log after the fixes (in combination with the third patch to remove the
extra line):

 KTAP version 1
 1..1
     KTAP version 1
     # Subtest: kunit_status
     1..2
     ok 1 kunit_status_set_failure_test
     ok 2 kunit_status_mark_skipped_test
 # kunit_status: pass:2 fail:0 skip:0 total:2
 # Totals: pass:2 fail:0 skip:0 total:2
 ok 1 kunit_status

Signed-off-by: Rae Moar 
Reviewed-by: David Gow 
Signed-off-by: Shuah Khan 
Signed-off-by: Sasha Levin

kunit: Add printf attribute to fail_current_test_impl

2023-02-09T01:08:14+00:00

Add the gnu_printf (__printf()) attribute to the
kunit_fail_current_test() implementation in
__kunit_fail_current_test_impl(). While it's not actually useful here,
as this function is never called directly, it nevertheless was
triggering -Wsuggest-attribute=format warnings, so we should add it to
reduce the noise.

Fixes: cc3ed2fe5c93 ("kunit: Add "hooks" to call into KUnit when it's built as a module")
Reported-by: kernel test robot 
Signed-off-by: David Gow 
Reviewed-by: Rae Moar 
Signed-off-by: Shuah Khan

kunit: Add "hooks" to call into KUnit when it's built as a module

2023-02-08T21:26:25+00:00

KUnit has several macros and functions intended for use from non-test
code. These hooks, currently the kunit_get_current_test() and
kunit_fail_current_test() macros, didn't work when CONFIG_KUNIT=m.

In order to support this case, the required functions and static data
need to be available unconditionally, even when KUnit itself is not
built-in. The new 'hooks.c' file is therefore always included, and has
both the static key required for kunit_get_current_test(), and a table
of function pointers in struct kunit_hooks_table. This is filled in with
the real implementations by kunit_install_hooks(), which is kept in
hooks-impl.h and called when the kunit module is loaded.

This can  be extended for future features which require similar
"hook" behaviour, such as static stubs, by simply adding new entries to
the struct, and the appropriate code to set them.

Fixed white-space errors during commit:
Shuah Khan 

Resolved merge conflicts with:
db105c37a4d6 ("kunit: Export kunit_running()")
This patch supersedes the above.
Shuah Khan 

Signed-off-by: David Gow 
Reviewed-by: Rae Moar 
Reviewed-by: Brendan Higgins 
Signed-off-by: Shuah Khan

kunit: Export kunit_running()

2023-01-20T17:03:04+00:00

Using kunit_fail_current_test() in a loadable module causes a link
error like:

ERROR: modpost: "kunit_running" [drivers/gpu/drm/vc4/vc4.ko] undefined!

Export the symbol to allow using it from modules.

Fixes: da43ff045c3f ("drm/vc4: tests: Fail the current test if we access a register")
Signed-off-by: Arnd Bergmann 
Reviewed-by: David Gow 
Signed-off-by: Shuah Khan

Merge tag 'linux-kselftest-kunit-next-6.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest

2022-12-13T00:42:57+00:00

Pull KUnit updates from Shuah Khan:
 "Several enhancements, fixes, clean-ups, documentation updates,
  improvements to logging and KTAP compliance of KUnit test output:

   - log numbers in decimal and hex

   - parse KTAP compliant test output

   - allow conditionally exposing static symbols to tests when KUNIT is
     enabled

   - make static symbols visible during kunit testing

   - clean-ups to remove unused structure definition"

* tag 'linux-kselftest-kunit-next-6.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: (29 commits)
  Documentation: dev-tools: Clarify requirements for result description
  apparmor: test: make static symbols visible during kunit testing
  kunit: add macro to allow conditionally exposing static symbols to tests
  kunit: tool: make parser preserve whitespace when printing test log
  Documentation: kunit: Fix "How Do I Use This" / "Next Steps" sections
  kunit: tool: don't include KTAP headers and the like in the test log
  kunit: improve KTAP compliance of KUnit test output
  kunit: tool: parse KTAP compliant test output
  mm: slub: test: Use the kunit_get_current_test() function
  kunit: Use the static key when retrieving the current test
  kunit: Provide a static key to check if KUnit is actively running tests
  kunit: tool: make --json do nothing if --raw_ouput is set
  kunit: tool: tweak error message when no KTAP found
  kunit: remove KUNIT_INIT_MEM_ASSERTION macro
  Documentation: kunit: Remove redundant 'tips.rst' page
  Documentation: KUnit: reword description of assertions
  Documentation: KUnit: make usage.rst a superset of tips.rst, remove duplication
  kunit: eliminate KUNIT_INIT_*_ASSERT_STRUCT macros
  kunit: tool: remove redundant file.close() call in unit test
  kunit: tool: unit tests all check parser errors, standardize formatting a bit
  ...

kunit: improve KTAP compliance of KUnit test output

2022-12-12T21:13:47+00:00

Change KUnit test output to better comply with KTAP v1 specifications
found here: https://kernel.org/doc/html/latest/dev-tools/ktap.html.
1) Use "KTAP version 1" instead of "TAP version 14" as test output header
2) Remove '-' between test number and test name on test result lines
2) Add KTAP version lines to each subtest header as well

Note that the new KUnit output still includes the “# Subtest” line now
located after the KTAP version line. This does not completely match the
KTAP v1 spec but since it is classified as a diagnostic line, it is not
expected to be disruptive or break any existing parsers. This
“# Subtest” line comes from the TAP 14 spec
(https://testanything.org/tap-version-14-specification.html) and it is
used to define the test name before the results.

Original output:

 TAP version 14
 1..1
   # Subtest: kunit-test-suite
   1..3
   ok 1 - kunit_test_1
   ok 2 - kunit_test_2
   ok 3 - kunit_test_3
 # kunit-test-suite: pass:3 fail:0 skip:0 total:3
 # Totals: pass:3 fail:0 skip:0 total:3
 ok 1 - kunit-test-suite

New output:

 KTAP version 1
 1..1
   KTAP version 1
   # Subtest: kunit-test-suite
   1..3
   ok 1 kunit_test_1
   ok 2 kunit_test_2
   ok 3 kunit_test_3
 # kunit-test-suite: pass:3 fail:0 skip:0 total:3
 # Totals: pass:3 fail:0 skip:0 total:3
 ok 1 kunit-test-suite

Signed-off-by: Rae Moar 
Reviewed-by: Daniel Latypov 
Reviewed-by: David Gow 
Tested-by: Anders Roxell 
Signed-off-by: Shuah Khan

kunit: Provide a static key to check if KUnit is actively running tests

2022-12-12T21:13:47+00:00

KUnit does a few expensive things when enabled. This hasn't been a
problem because KUnit was only enabled on test kernels, but with a few
people enabling (but not _using_) KUnit on production systems, we need a
runtime way of handling this.

Provide a 'kunit_running' static key (defaulting to false), which allows
us to hide any KUnit code behind a static branch. This should reduce the
performance impact (on other code) of having KUnit enabled to a single
NOP when no tests are running.

Note that, while it looks unintuitive, tests always run entirely within
__kunit_test_suites_init(), so it's safe to decrement the static key at
the end of this function, rather than in __kunit_test_suites_exit(),
which is only there to clean up results in debugfs.

Signed-off-by: David Gow 
Reviewed-by: Daniel Latypov 
Signed-off-by: Shuah Khan

kunit: update NULL vs IS_ERR() tests

2022-10-18T21:08:42+00:00

The alloc_string_stream() functions were changed from returning NULL on
error to returning error pointers so these caller needs to be updated
as well.

Fixes: 78b1c6584fce ("kunit: string-stream: Simplify resource use")
Signed-off-by: Dan Carpenter 
Reviewed-by: Daniel Latypov 
Reviewed-by: David Gow 
Signed-off-by: Shuah Khan

kunit: remove format func from struct kunit_assert, get it to 0 bytes

2022-10-07T16:16:38+00:00

Each calll to a KUNIT_EXPECT_*() macro creates a local variable which
contains a struct kunit_assert.

Normally, we'd hope the compiler would be able to optimize this away,
but we've seen cases where it hasn't, see
https://groups.google.com/g/kunit-dev/c/i3fZXgvBrfA/m/GbrMNej2BAAJ.

In changes like commit 21957f90b28f ("kunit: split out part of
kunit_assert into a static const"), we've moved more and more parts out
of struct kunit_assert and its children types (kunit_binary_assert).

This patch removes the final field and gets us to:
  sizeof(struct kunit_assert) == 0
  sizeof(struct kunit_binary_assert) == 24 (on UML x86_64).

This also reduces the amount of macro plumbing going on at the cost of
passing in one more arg to the base KUNIT_ASSERTION macro and
kunit_do_failed_assertion().

Signed-off-by: Daniel Latypov 
Reviewed-by: David Gow 
Signed-off-by: Shuah Khan

kunit: make kunit_kfree(NULL) a no-op to match kfree()

2022-10-07T16:15:56+00:00

The real kfree() function will silently return when given a NULL.
So a user might reasonably think they can write the following code:
  char *buffer = NULL;
  if (param->use_buffer) buffer = kunit_kzalloc(test, 10, GFP_KERNEL);
  ...
  kunit_kfree(test, buffer);

As-is, kunit_kfree() will mark the test as FAILED when buffer is NULL.
(And in earlier times, it would segfault).

Let's match the semantics of kfree().

Suggested-by: David Gow 
Signed-off-by: Daniel Latypov 
Reviewed-by: David Gow 
Reviewed-by: Brendan Higgins 
Signed-off-by: Shuah Khan