linux-stable.git/tools/testing/kunit, branch linux-6.5.y Linux kernel stable tree Merge tag 'hardening-v6.5-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux 2023-06-28T04:24:18+00:00 Linus Torvalds torvalds@linux-foundation.org 2023-06-28T04:24:18+00:00 582c161cf38cf016cd573af6f087fa5fa786949b Pull hardening updates from Kees Cook: "There are three areas of note: A bunch of strlcpy()->strscpy() conversions ended up living in my tree since they were either Acked by maintainers for me to carry, or got ignored for multiple weeks (and were trivial changes). The compiler option '-fstrict-flex-arrays=3' has been enabled globally, and has been in -next for the entire devel cycle. This changes compiler diagnostics (though mainly just -Warray-bounds which is disabled) and potential UBSAN_BOUNDS and FORTIFY _warning_ coverage. In other words, there are no new restrictions, just potentially new warnings. Any new FORTIFY warnings we've seen have been fixed (usually in their respective subsystem trees). For more details, see commit df8fc4e934c12b. The under-development compiler attribute __counted_by has been added so that we can start annotating flexible array members with their associated structure member that tracks the count of flexible array elements at run-time. It is possible (likely?) that the exact syntax of the attribute will change before it is finalized, but GCC and Clang are working together to sort it out. Any changes can be made to the macro while we continue to add annotations. As an example of that last case, I have a treewide commit waiting with such annotations found via Coccinelle: https://git.kernel.org/linus/adc5b3cb48a049563dc673f348eab7b6beba8a9b Also see commit dd06e72e68bcb4 for more details. Summary: - Fix KMSAN vs FORTIFY in strlcpy/strlcat (Alexander Potapenko) - Convert strreplace() to return string start (Andy Shevchenko) - Flexible array conversions (Arnd Bergmann, Wyes Karny, Kees Cook) - Add missing function prototypes seen with W=1 (Arnd Bergmann) - Fix strscpy() kerndoc typo (Arne Welzel) - Replace strlcpy() with strscpy() across many subsystems which were either Acked by respective maintainers or were trivial changes that went ignored for multiple weeks (Azeem Shaikh) - Remove unneeded cc-option test for UBSAN_TRAP (Nick Desaulniers) - Add KUnit tests for strcat()-family - Enable KUnit tests of FORTIFY wrappers under UML - Add more complete FORTIFY protections for strlcat() - Add missed disabling of FORTIFY for all arch purgatories. - Enable -fstrict-flex-arrays=3 globally - Tightening UBSAN_BOUNDS when using GCC - Improve checkpatch to check for strcpy, strncpy, and fake flex arrays - Improve use of const variables in FORTIFY - Add requested struct_size_t() helper for types not pointers - Add __counted_by macro for annotating flexible array size members" * tag 'hardening-v6.5-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux: (54 commits) netfilter: ipset: Replace strlcpy with strscpy uml: Replace strlcpy with strscpy um: Use HOST_DIR for mrproper kallsyms: Replace all non-returning strlcpy with strscpy sh: Replace all non-returning strlcpy with strscpy of/flattree: Replace all non-returning strlcpy with strscpy sparc64: Replace all non-returning strlcpy with strscpy Hexagon: Replace all non-returning strlcpy with strscpy kobject: Use return value of strreplace() lib/string_helpers: Change returned value of the strreplace() jbd2: Avoid printing outside the boundary of the buffer checkpatch: Check for 0-length and 1-element arrays riscv/purgatory: Do not use fortified string functions s390/purgatory: Do not use fortified string functions x86/purgatory: Do not use fortified string functions acpi: Replace struct acpi_table_slit 1-element array with flex-array clocksource: Replace all non-returning strlcpy with strscpy string: use __builtin_memcpy() in strlcpy/strlcat staging: most: Replace all non-returning strlcpy with strscpy drm/i2c: tda998x: Replace all non-returning strlcpy with strscpy ... 
Pull hardening updates from Kees Cook:
 "There are three areas of note:

  A bunch of strlcpy()->strscpy() conversions ended up living in my tree
  since they were either Acked by maintainers for me to carry, or got
  ignored for multiple weeks (and were trivial changes).

  The compiler option '-fstrict-flex-arrays=3' has been enabled
  globally, and has been in -next for the entire devel cycle. This
  changes compiler diagnostics (though mainly just -Warray-bounds which
  is disabled) and potential UBSAN_BOUNDS and FORTIFY _warning_
  coverage. In other words, there are no new restrictions, just
  potentially new warnings. Any new FORTIFY warnings we've seen have
  been fixed (usually in their respective subsystem trees). For more
  details, see commit df8fc4e934c12b.

  The under-development compiler attribute __counted_by has been added
  so that we can start annotating flexible array members with their
  associated structure member that tracks the count of flexible array
  elements at run-time. It is possible (likely?) that the exact syntax
  of the attribute will change before it is finalized, but GCC and Clang
  are working together to sort it out. Any changes can be made to the
  macro while we continue to add annotations.

  As an example of that last case, I have a treewide commit waiting with
  such annotations found via Coccinelle:

    https://git.kernel.org/linus/adc5b3cb48a049563dc673f348eab7b6beba8a9b

  Also see commit dd06e72e68bcb4 for more details.

  Summary:

   - Fix KMSAN vs FORTIFY in strlcpy/strlcat (Alexander Potapenko)

   - Convert strreplace() to return string start (Andy Shevchenko)

   - Flexible array conversions (Arnd Bergmann, Wyes Karny, Kees Cook)

   - Add missing function prototypes seen with W=1 (Arnd Bergmann)

   - Fix strscpy() kerndoc typo (Arne Welzel)

   - Replace strlcpy() with strscpy() across many subsystems which were
     either Acked by respective maintainers or were trivial changes that
     went ignored for multiple weeks (Azeem Shaikh)

   - Remove unneeded cc-option test for UBSAN_TRAP (Nick Desaulniers)

   - Add KUnit tests for strcat()-family

   - Enable KUnit tests of FORTIFY wrappers under UML

   - Add more complete FORTIFY protections for strlcat()

   - Add missed disabling of FORTIFY for all arch purgatories.

   - Enable -fstrict-flex-arrays=3 globally

   - Tightening UBSAN_BOUNDS when using GCC

   - Improve checkpatch to check for strcpy, strncpy, and fake flex
     arrays

   - Improve use of const variables in FORTIFY

   - Add requested struct_size_t() helper for types not pointers

   - Add __counted_by macro for annotating flexible array size members"

* tag 'hardening-v6.5-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux: (54 commits)
  netfilter: ipset: Replace strlcpy with strscpy
  uml: Replace strlcpy with strscpy
  um: Use HOST_DIR for mrproper
  kallsyms: Replace all non-returning strlcpy with strscpy
  sh: Replace all non-returning strlcpy with strscpy
  of/flattree: Replace all non-returning strlcpy with strscpy
  sparc64: Replace all non-returning strlcpy with strscpy
  Hexagon: Replace all non-returning strlcpy with strscpy
  kobject: Use return value of strreplace()
  lib/string_helpers: Change returned value of the strreplace()
  jbd2: Avoid printing outside the boundary of the buffer
  checkpatch: Check for 0-length and 1-element arrays
  riscv/purgatory: Do not use fortified string functions
  s390/purgatory: Do not use fortified string functions
  x86/purgatory: Do not use fortified string functions
  acpi: Replace struct acpi_table_slit 1-element array with flex-array
  clocksource: Replace all non-returning strlcpy with strscpy
  string: use __builtin_memcpy() in strlcpy/strlcat
  staging: most: Replace all non-returning strlcpy with strscpy
  drm/i2c: tda998x: Replace all non-returning strlcpy with strscpy
  ...
kunit: tool: Enable CONFIG_FORTIFY_SOURCE under UML 2023-05-16T21:07:05+00:00 Kees Cook keescook@chromium.org 2023-04-03T02:56:34+00:00 4d9060981f886ba881aa3e8de688433c1f1ed11f Since commit ba38961a069b ("um: Enable FORTIFY_SOURCE"), it's possible to run the FORTIFY tests under UML. Enable CONFIG_FORTIFY_SOURCE when running with --alltests to gain additional coverage, and by default under UML. Signed-off-by: Kees Cook <keescook@chromium.org> 
Since commit ba38961a069b ("um: Enable FORTIFY_SOURCE"), it's possible
to run the FORTIFY tests under UML. Enable CONFIG_FORTIFY_SOURCE when
running with --alltests to gain additional coverage, and by default under
UML.

Signed-off-by: Kees Cook <keescook@chromium.org>
kunit: tool: undo type subscripts for subprocess.Popen 2023-05-08T14:39:41+00:00 Daniel Latypov dlatypov@google.com 2023-05-01T18:16:10+00:00 e30f65c4b3d671115bf2a9d9ef142285387f2aff Writing `subprocess.Popen[str]` requires python 3.9+. kunit.py has an assertion that the python version is 3.7+, so we should try to stay backwards compatible. This conflicts a bit with commit 1da2e6220e11 ("kunit: tool: fix pre-existing `mypy --strict` errors and update run_checks.py"), since mypy complains like so > kunit_kernel.py:95: error: Missing type parameters for generic type "Popen" [type-arg] Note: `mypy --strict --python-version 3.7` does not work. We could annotate each file with comments like `# mypy: disable-error-code="type-arg" but then we might still get nudged to break back-compat in other files. This patch adds a `mypy.ini` file since it seems like the only way to disable specific error codes for all our files. Note: run_checks.py doesn't need to specify `--config_file mypy.ini`, but I think being explicit is better, particularly since most kernel devs won't be familiar with how mypy works. Fixes: 695e26030858 ("kunit: tool: add subscripts for type annotations where appropriate") Reported-by: SeongJae Park <sj@kernel.org> Link: https://lore.kernel.org/linux-kselftest/20230501171520.138753-1-sj@kernel.org Signed-off-by: Daniel Latypov <dlatypov@google.com> Tested-by: SeongJae Park <sj@kernel.org> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
Writing `subprocess.Popen[str]` requires python 3.9+.
kunit.py has an assertion that the python version is 3.7+, so we should
try to stay backwards compatible.

This conflicts a bit with commit 1da2e6220e11 ("kunit: tool: fix
pre-existing `mypy --strict` errors and update run_checks.py"), since
mypy complains like so
> kunit_kernel.py:95: error: Missing type parameters for generic type "Popen"  [type-arg]

Note: `mypy --strict --python-version 3.7` does not work.

We could annotate each file with comments like
  `# mypy: disable-error-code="type-arg"
but then we might still get nudged to break back-compat in other files.

This patch adds a `mypy.ini` file since it seems like the only way to
disable specific error codes for all our files.

Note: run_checks.py doesn't need to specify `--config_file mypy.ini`,
but I think being explicit is better, particularly since most kernel
devs won't be familiar with how mypy works.

Fixes: 695e26030858 ("kunit: tool: add subscripts for type annotations where appropriate")
Reported-by: SeongJae Park <sj@kernel.org>
Link: https://lore.kernel.org/linux-kselftest/20230501171520.138753-1-sj@kernel.org
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Tested-by: SeongJae Park <sj@kernel.org>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kunit: tool: Add support for SH under QEMU 2023-04-05T18:51:16+00:00 Geert Uytterhoeven geert+renesas@glider.be 2023-02-28T10:31:03+00:00 8110a3cab05ee4a26f36015f0423fb2b5cea1392 Add basic support to run SH under QEMU via kunit_tool using the virtualized r2d platform. Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
Add basic support to run SH under QEMU via kunit_tool using the
virtualized r2d platform.

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kunit: tool: Add support for overriding the QEMU serial port 2023-04-05T18:51:06+00:00 Geert Uytterhoeven geert+renesas@glider.be 2023-02-28T10:31:02+00:00 5ffb8629b133fecf7cdd36134ae6d6a1efb46eb5 On some platforms, the console is not the first serial port. To make this work, the first serial port in QEMU must be set to "null". Add support for this by adding an optional "serial" parameter, which defaults to "stdio", and can be overridden by platform-specific configuration. Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
On some platforms, the console is not the first serial port.  To make
this work, the first serial port in QEMU must be set to "null".

Add support for this by adding an optional "serial" parameter, which
defaults to "stdio", and can be overridden by platform-specific
configuration.

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kunit: tool: fix pre-existing `mypy --strict` errors and update run_checks.py 2023-03-17T18:28:30+00:00 Daniel Latypov dlatypov@google.com 2023-03-16T22:06:38+00:00 1da2e6220e1115930694c649605534baf6fa3dea Basically, get this command to be happy and make run_checks.py happy $ mypy --strict --exclude '_test.py$' --exclude qemu_configs/ ./tools/testing/kunit/ Primarily the changes are * add `-> None` return type annotations * add all the missing argument type annotations Previously, we had false positives from mypy in `main()`, see commit 09641f7c7d8f ("kunit: tool: surface and address more typing issues"). But after commit 2dc9d6ca52a4 ("kunit: kunit.py extract handlers") refactored things, the variable name reuse mypy hated is gone. Note: mypy complains we don't annotate the types the unused args in our signal handler. That's silly. But to make it happy, I've copy-pasted an appropriate annotation from https://github.com/python/typing/discussions/1042#discussioncomment-2013595. Reported-by: Johannes Berg <johannes.berg@intel.com> Link: https://lore.kernel.org/linux-kselftest/9a172b50457f4074af41fe1dc8e55dcaf4795d7e.camel@sipsolutions.net/ Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
Basically, get this command to be happy and make run_checks.py happy
 $ mypy --strict --exclude '_test.py$' --exclude qemu_configs/ ./tools/testing/kunit/

Primarily the changes are
* add `-> None` return type annotations
* add all the missing argument type annotations

Previously, we had false positives from mypy in `main()`, see commit
09641f7c7d8f ("kunit: tool: surface and address more typing issues").
But after commit 2dc9d6ca52a4 ("kunit: kunit.py extract handlers")
refactored things, the variable name reuse mypy hated is gone.

Note: mypy complains we don't annotate the types the unused args in our
signal handler. That's silly.
But to make it happy, I've copy-pasted an appropriate annotation from
https://github.com/python/typing/discussions/1042#discussioncomment-2013595.

Reported-by: Johannes Berg <johannes.berg@intel.com>
Link: https://lore.kernel.org/linux-kselftest/9a172b50457f4074af41fe1dc8e55dcaf4795d7e.camel@sipsolutions.net/
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kunit: tool: remove unused imports and variables 2023-03-17T18:28:25+00:00 Daniel Latypov dlatypov@google.com 2023-03-16T22:06:37+00:00 126901ba3499880c9ed033633817cf7493120fda We don't run a linter regularly over kunit.py code (the default settings on most don't like kernel style, e.g. tabs) so some of these imports didn't get removed when they stopped being used. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
We don't run a linter regularly over kunit.py code (the default settings
on most don't like kernel style, e.g. tabs) so some of these imports
didn't get removed when they stopped being used.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kunit: tool: add subscripts for type annotations where appropriate 2023-03-17T18:28:18+00:00 Daniel Latypov dlatypov@google.com 2023-03-16T22:06:36+00:00 695e26030858b27648ca107b77095fed53377b4b E.g. for subprocess.Popen, it can be opened in `text=True` mode where it returns strings, or `text=False` where it returns bytes. To differentiate, you can annotate types as `Popen[str]` or `Popen[bytes]`. This patch should add subscripts in all the places we were missing them. Reported-by: Johannes Berg <johannes.berg@intel.com> Link: https://lore.kernel.org/linux-kselftest/20230315105055.9b2be0153625.I7a2cb99b95dff216c0feed4604255275e0b156a7@changeid/ Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
E.g. for subprocess.Popen, it can be opened in `text=True` mode where it
returns strings, or `text=False` where it returns bytes.
To differentiate, you can annotate types as `Popen[str]` or
`Popen[bytes]`.

This patch should add subscripts in all the places we were missing them.

Reported-by: Johannes Berg <johannes.berg@intel.com>
Link: https://lore.kernel.org/linux-kselftest/20230315105055.9b2be0153625.I7a2cb99b95dff216c0feed4604255275e0b156a7@changeid/
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kunit: tool: Add support for m68k under QEMU 2023-03-07T00:11:45+00:00 Geert Uytterhoeven geert@linux-m68k.org 2023-02-27T11:12:45+00:00 60684c2bd35064043360e6f716d1b7c20e967b7d Add basic support to run m68k under QEMU via kunit_tool. Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
Add basic support to run m68k under QEMU via kunit_tool.

Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
kunit: kunit.py extract handlers 2023-02-08T21:25:30+00:00 Alexander Pantyukhin apantykhin@gmail.com 2023-01-21T21:27:17+00:00 2dc9d6ca52a47fd00822e818c2a5e48fc5fbbd53 The main function contains a wide if-elif block that handles different subcommands. It's possible to make code refactoring to extract subcommands handlers. Fixed commit summary line. Shuah Khan <skhan@linuxfoundation.org> Signed-off-by: Alexander Pantyukhin <apantykhin@gmail.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> 
The main function contains a wide if-elif block that handles different
subcommands. It's possible to make code refactoring to extract
subcommands handlers.

Fixed commit summary line.
Shuah Khan <skhan@linuxfoundation.org>

Signed-off-by: Alexander Pantyukhin <apantykhin@gmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>