linux.git/arch/arm64/kernel/vdso.c, branch v6.16

mseal sysmap: enable arm64

2025-04-01T22:17:15+00:00

Provide support for CONFIG_MSEAL_SYSTEM_MAPPINGS on arm64, covering the
vdso, vvar, and compat-mode vectors and sigpage mappings.

Production release testing passes on Android and Chrome OS.

Link: https://lkml.kernel.org/r/20250305021711.3867874-5-jeffxu@google.com
Signed-off-by: Jeff Xu 
Reviewed-by: Lorenzo Stoakes 
Reviewed-by: Liam R. Howlett 
Reviewed-by: Kees Cook 
Cc: Adhemerval Zanella 
Cc: Alexander Mikhalitsyn 
Cc: Alexey Dobriyan 
Cc: Andrei Vagin 
Cc: Anna-Maria Behnsen 
Cc: Ard Biesheuvel 
Cc: Benjamin Berg 
Cc: Christoph Hellwig 
Cc: Dave Hansen 
Cc: David Rientjes 
Cc: David S. Miller 
Cc: Elliot Hughes 
Cc: Florian Faineli 
Cc: Greg Ungerer 
Cc: Guenter Roeck 
Cc: Heiko Carstens 
Cc: Helge Deller 
Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Cc: Ingo Molnar 
Cc: Jann Horn 
Cc: Jason A. Donenfeld 
Cc: Johannes Berg 
Cc: Jorge Lucangeli Obes 
Cc: Linus Waleij 
Cc: Mark Rutland 
Cc: Matthew Wilcow (Oracle) 
Cc: Michael Ellerman 
Cc: Michal Hocko 
Cc: Miguel Ojeda 
Cc: Mike Rapoport 
Cc: Oleg Nesterov 
Cc: Pedro Falcato 
Cc: Peter Xu 
Cc: Randy Dunlap 
Cc: Stephen Röttger 
Cc: Thomas Weißschuh 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton

arm64: vdso: Switch to generic storage implementation

2025-02-21T08:54:01+00:00

The generic storage implementation provides the same features as the
custom one. However it can be shared between architectures, making
maintenance easier.

This switch also moves the random state data out of the time data page.
The currently used hardcoded __VDSO_RND_DATA_OFFSET does not take into
account changes to the time data page layout.

Co-developed-by: Nam Cao 
Signed-off-by: Nam Cao 
Signed-off-by: Thomas Weißschuh 
Signed-off-by: Thomas Gleixner 
Link: https://lore.kernel.org/all/20250204-vdso-store-rng-v3-8-13a4669dfc8c@linutronix.de

arm64: vdso: Use only one single vvar mapping

2024-11-02T11:37:33+00:00

The vvar mapping is the same for all processes. Use a single mapping to
simplify the logic and align it with the other architectures.

In addition this will enable the move of the vvar handling into generic code.

Signed-off-by: Thomas Weißschuh 
Signed-off-by: Thomas Gleixner 
Tested-by: Will Deacon 
Acked-by: Will Deacon 
Link: https://lore.kernel.org/all/20241010-vdso-generic-base-v1-5-b64f0842d512@linutronix.de

arm64: vdso: Remove timekeeper include

2024-10-15T15:50:29+00:00

Since the generic VDSO clock mode storage is used, this header file is
unused and can be removed.

Signed-off-by: Thomas Weißschuh 
Signed-off-by: Thomas Gleixner 
Acked-by: Will Deacon 
Link: https://lore.kernel.org/all/20241010-vdso-generic-arch_update_vsyscall-v1-3-7fe5a3ea4382@linutronix.de

arm64: vDSO: Wire up getrandom() vDSO implementation

2024-09-13T15:28:36+00:00

Hook up the generic vDSO implementation to the aarch64 vDSO data page.
The _vdso_rng_data required data is placed within the _vdso_data vvar
page, by using a offset larger than the vdso_data.

The vDSO function requires a ChaCha20 implementation that does not write
to the stack, and that can do an entire ChaCha20 permutation.  The one
provided uses NEON on the permute operation, with a fallback to the
syscall for chips that do not support AdvSIMD.

This also passes the vdso_test_chacha test along with
vdso_test_getrandom. The vdso_test_getrandom bench-single result on
Neoverse-N1 shows:

   vdso: 25000000 times in 0.783884250 seconds
   libc: 25000000 times in 8.780275399 seconds
syscall: 25000000 times in 8.786581518 seconds

A small fixup to arch/arm64/include/asm/mman.h was required to avoid
pulling kernel code into the vDSO, similar to what's already done in
arch/arm64/include/asm/rwonce.h.

Signed-off-by: Adhemerval Zanella 
Reviewed-by: Ard Biesheuvel 
Acked-by: Will Deacon 
Signed-off-by: Jason A. Donenfeld

arm64: vdso: Use generic union vdso_data_store

2024-02-20T19:56:00+00:00

There is already a generic union definition for vdso_data_store in vdso
datapage header.

Use this definition to prevent code duplication.

Signed-off-by: Anna-Maria Behnsen 
Signed-off-by: Thomas Gleixner 
Reviewed-by: Vincenzo Frascino 
Reviewed-by: Kees Cook 
Link: https://lore.kernel.org/r/20240219153939.75719-6-anna-maria@linutronix.de

arm64: Avoid cpus_have_const_cap() for ARM64_HAS_BTI

2023-10-16T13:17:04+00:00

In system_supports_bti() we use cpus_have_const_cap() to check for
ARM64_HAS_BTI, but this is not necessary and alternative_has_cap_*() or
cpus_have_final_*cap() would be preferable.

For historical reasons, cpus_have_const_cap() is more complicated than
it needs to be. Before cpucaps are finalized, it will perform a bitmap
test of the system_cpucaps bitmap, and once cpucaps are finalized it
will use an alternative branch. This used to be necessary to handle some
race conditions in the window between cpucap detection and the
subsequent patching of alternatives and static branches, where different
branches could be out-of-sync with one another (or w.r.t. alternative
sequences). Now that we use alternative branches instead of static
branches, these are all patched atomically w.r.t. one another, and there
are only a handful of cases that need special care in the window between
cpucap detection and alternative patching.

Due to the above, it would be nice to remove cpus_have_const_cap(), and
migrate callers over to alternative_has_cap_*(), cpus_have_final_cap(),
or cpus_have_cap() depending on when their requirements. This will
remove redundant instructions and improve code generation, and will make
it easier to determine how each callsite will behave before, during, and
after alternative patching.

When CONFIG_ARM64_BTI_KERNEL=y, the ARM64_HAS_BTI cpucap is a strict
boot cpu feature which is detected and patched early on the boot cpu.
All uses guarded by CONFIG_ARM64_BTI_KERNEL happen after the boot CPU
has detected ARM64_HAS_BTI and patched boot alternatives, and hence can
safely use alternative_has_cap_*() or cpus_have_final_boot_cap().

Regardless of CONFIG_ARM64_BTI_KERNEL, all other uses of ARM64_HAS_BTI
happen after system capabilities have been finalized and alternatives
have been patched. Hence these can safely use alternative_has_cap_*) or
cpus_have_final_cap().

This patch splits system_supports_bti() into system_supports_bti() and
system_supports_bti_kernel(), with the former handling where the cpucap
affects userspace functionality, and ther latter handling where the
cpucap affects kernel functionality. The use of cpus_have_const_cap() is
replaced by cpus_have_final_cap() in cpus_have_const_cap, and
cpus_have_final_boot_cap() in system_supports_bti_kernel(). This will
avoid generating code to test the system_cpucaps bitmap and should be
better for all subsequent calls at runtime. The use of
cpus_have_final_cap() and cpus_have_final_boot_cap() will make it easier
to spot if code is chaanged such that these run before the ARM64_HAS_BTI
cpucap is guaranteed to have been finalized.

Signed-off-by: Mark Rutland 
Reviewed-by: Mark Brown 
Cc: Ard Biesheuvel 
Cc: Suzuki K Poulose 
Cc: Will Deacon 
Signed-off-by: Catalin Marinas

arm64: vdso: Pass (void *) to virt_to_page()

2023-05-16T13:53:36+00:00

Like the other calls in this function virt_to_page() expects
a pointer, not an integer.

However since many architectures implement virt_to_pfn() as
a macro, this function becomes polymorphic and accepts both a
(unsigned long) and a (void *).

Fix this up with an explicit cast.

Signed-off-by: Linus Walleij 
Link: http://lists.infradead.org/pipermail/linux-arm-kernel/2023-May/832583.html
Signed-off-by: Will Deacon

mm: remove zap_page_range and create zap_vma_pages

2023-01-19T01:12:55+00:00

zap_page_range was originally designed to unmap pages within an address
range that could span multiple vmas.  While working on [1], it was
discovered that all callers of zap_page_range pass a range entirely within
a single vma.  In addition, the mmu notification call within zap_page
range does not correctly handle ranges that span multiple vmas.  When
crossing a vma boundary, a new mmu_notifier_range_init/end call pair with
the new vma should be made.

Instead of fixing zap_page_range, do the following:
- Create a new routine zap_vma_pages() that will remove all pages within
  the passed vma.  Most users of zap_page_range pass the entire vma and
  can use this new routine.
- For callers of zap_page_range not passing the entire vma, instead call
  zap_page_range_single().
- Remove zap_page_range.

[1] https://lore.kernel.org/linux-mm/20221114235507.294320-2-mike.kravetz@oracle.com/
Link: https://lkml.kernel.org/r/20230104002732.232573-1-mike.kravetz@oracle.com
Signed-off-by: Mike Kravetz 
Suggested-by: Peter Xu 
Acked-by: Michal Hocko 
Acked-by: Peter Xu 
Acked-by: Heiko Carstens 	[s390]
Reviewed-by: Christoph Hellwig 
Cc: Christian Borntraeger 
Cc: Christian Brauner 
Cc: Dave Hansen 
Cc: David Hildenbrand 
Cc: Eric Dumazet 
Cc: Matthew Wilcox 
Cc: Michael Ellerman 
Cc: Nadav Amit 
Cc: Palmer Dabbelt 
Cc: Rik van Riel 
Cc: Vlastimil Babka 
Cc: Will Deacon 
Signed-off-by: Andrew Morton

vdso/timens: Refactor copy-pasted find_timens_vvar_page() helper into one copy

2022-12-01T10:35:40+00:00

find_timens_vvar_page() is not architecture-specific, as can be seen from
how all five per-architecture versions of it are the same.

(arm64, powerpc and riscv are exactly the same; x86 and s390 have two
characters difference inside a comment, less blank lines, and mark the
!CONFIG_TIME_NS version as inline.)

Refactor the five copies into a central copy in kernel/time/namespace.c.

Signed-off-by: Jann Horn 
Signed-off-by: Thomas Gleixner 
Link: https://lore.kernel.org/r/20221130115320.2918447-1-jannh@google.com