linux.git/arch/arm64/include/asm/memory.h, branch v5.11

arm64: Use simpler arithmetics for the linear map macros

2021-02-02T17:45:09+00:00

Because of the tagged addresses, the __is_lm_address() and
__lm_to_phys() macros grew to some harder to understand bitwise
operations using PAGE_OFFSET. Since these macros only accept untagged
addresses, use a simple subtract operation.

Signed-off-by: Catalin Marinas 
Acked-by: Ard Biesheuvel 
Cc: Will Deacon 
Cc: Mark Rutland 
Link: https://lore.kernel.org/r/20210201190634.22942-3-catalin.marinas@arm.com

arm64: Do not pass tagged addresses to __is_lm_address()

2021-02-02T17:44:47+00:00

Commit 519ea6f1c82f ("arm64: Fix kernel address detection of
__is_lm_address()") fixed the incorrect validation of addresses below
PAGE_OFFSET. However, it no longer allowed tagged addresses to be passed
to virt_addr_valid().

Fix this by explicitly resetting the pointer tag prior to invoking
__is_lm_address(). This is consistent with the __lm_to_phys() macro.

Fixes: 519ea6f1c82f ("arm64: Fix kernel address detection of __is_lm_address()")
Signed-off-by: Catalin Marinas 
Acked-by: Ard Biesheuvel 
Cc:  # 5.4.x
Cc: Will Deacon 
Cc: Vincenzo Frascino 
Cc: Mark Rutland 
Link: https://lore.kernel.org/r/20210201190634.22942-2-catalin.marinas@arm.com

arm64: Fix kernel address detection of __is_lm_address()

2021-01-26T17:53:32+00:00

Currently, the __is_lm_address() check just masks out the top 12 bits
of the address, but if they are 0, it still yields a true result.
This has as a side effect that virt_addr_valid() returns true even for
invalid virtual addresses (e.g. 0x0).

Fix the detection checking that it's actually a kernel address starting
at PAGE_OFFSET.

Fixes: 68dd8ef32162 ("arm64: memory: Fix virt_addr_valid() using __is_lm_address()")
Cc:  # 5.4.x
Cc: Will Deacon 
Suggested-by: Catalin Marinas 
Reviewed-by: Catalin Marinas 
Acked-by: Mark Rutland 
Signed-off-by: Vincenzo Frascino 
Link: https://lore.kernel.org/r/20210126134056.45747-1-vincenzo.frascino@arm.com
Signed-off-by: Catalin Marinas

kasan, arm64: implement HW_TAGS runtime

2020-12-22T20:55:08+00:00

Provide implementation of KASAN functions required for the hardware
tag-based mode.  Those include core functions for memory and pointer
tagging (tags_hw.c) and bug reporting (report_tags_hw.c).  Also adapt
common KASAN code to support the new mode.

Link: https://lkml.kernel.org/r/cfd0fbede579a6b66755c98c88c108e54f9c56bf.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov 
Signed-off-by: Vincenzo Frascino 
Acked-by: Catalin Marinas 
Reviewed-by: Alexander Potapenko 
Tested-by: Vincenzo Frascino 
Cc: Andrey Ryabinin 
Cc: Branislav Rankov 
Cc: Dmitry Vyukov 
Cc: Evgenii Stepanov 
Cc: Kevin Brodsky 
Cc: Marco Elver 
Cc: Vasily Gorbik 
Cc: Will Deacon 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

kasan, arm64: expand CONFIG_KASAN checks

2020-12-22T20:55:08+00:00

Some #ifdef CONFIG_KASAN checks are only relevant for software KASAN modes
(either related to shadow memory or compiler instrumentation).  Expand
those into CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS.

Link: https://lkml.kernel.org/r/e6971e432dbd72bb897ff14134ebb7e169bdcf0c.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov 
Signed-off-by: Vincenzo Frascino 
Reviewed-by: Catalin Marinas 
Reviewed-by: Alexander Potapenko 
Tested-by: Vincenzo Frascino 
Cc: Andrey Ryabinin 
Cc: Branislav Rankov 
Cc: Dmitry Vyukov 
Cc: Evgenii Stepanov 
Cc: Kevin Brodsky 
Cc: Marco Elver 
Cc: Vasily Gorbik 
Cc: Will Deacon 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

arm64: kasan: add arch layer for memory tagging helpers

2020-12-22T20:55:08+00:00

This patch add a set of arch_*() memory tagging helpers currently only
defined for arm64 when hardware tag-based KASAN is enabled.  These helpers
will be used by KASAN runtime to implement the hardware tag-based mode.

The arch-level indirection level is introduced to simplify adding hardware
tag-based KASAN support for other architectures in the future by defining
the appropriate arch_*() macros.

Link: https://lkml.kernel.org/r/fc9e5bb71201c03131a2fc00a74125723568dda9.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov 
Co-developed-by: Vincenzo Frascino 
Signed-off-by: Vincenzo Frascino 
Reviewed-by: Catalin Marinas 
Tested-by: Vincenzo Frascino 
Cc: Alexander Potapenko 
Cc: Andrey Ryabinin 
Cc: Branislav Rankov 
Cc: Dmitry Vyukov 
Cc: Evgenii Stepanov 
Cc: Kevin Brodsky 
Cc: Marco Elver 
Cc: Vasily Gorbik 
Cc: Will Deacon 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

arm64: mm: don't assume struct page is always 64 bytes

2020-11-12T08:32:25+00:00

Commit 8c96400d6a39be7 simplified the page-to-virt and virt-to-page
conversions, based on the assumption that struct page is always 64
bytes in size, in which case we can use a single signed shift to
perform the conversion (provided that the vmemmap array is placed
appropriately in the kernel VA space)

Unfortunately, this assumption turns out not to hold, and so we need
to revert part of this commit, and go back to an affine transformation.
Given that all the quantities involved are compile time constants,
this should not make any practical difference.

Fixes: 8c96400d6a39 ("arm64: mm: make vmemmap region a projection of the linear region")
Reported-by: Geert Uytterhoeven 
Signed-off-by: Ard Biesheuvel 
Link: https://lore.kernel.org/r/20201110180511.29083-1-ardb@kernel.org
Tested-by: Geert Uytterhoeven 
Signed-off-by: Catalin Marinas

arm64: mm: tidy up top of kernel VA space

2020-11-09T17:15:37+00:00

Tidy up the way the top of the kernel VA space is organized, by mirroring
the 256 MB region we have below the vmalloc space, and populating it top
down with the PCI I/O space, some guard regions, and the fixmap region.
The latter region is itself populated top down, and today only covers
about 4 MB, and so 224 MB is ample, and no guard region is therefore
required.

The resulting layout is identical between 48-bit/4k and 52-bit/64k
configurations.

Signed-off-by: Ard Biesheuvel 
Reviewed-by: Steve Capper 
Link: https://lore.kernel.org/r/20201008153602.9467-5-ardb@kernel.org
Signed-off-by: Catalin Marinas

arm64: mm: make vmemmap region a projection of the linear region

2020-11-09T17:15:37+00:00

Now that we have reverted the introduction of the vmemmap struct page
pointer and the separate physvirt_offset, we can simplify things further,
and place the vmemmap region in the VA space in such a way that virtual
to page translations and vice versa can be implemented using a single
arithmetic shift.

One happy coincidence resulting from this is that the 48-bit/4k and
52-bit/64k configurations (which are assumed to be the two most
prevalent) end up with the same placement of the vmemmap region. In
a subsequent patch, we will take advantage of this, and unify the
memory maps even more.

Signed-off-by: Ard Biesheuvel 
Reviewed-by: Steve Capper 
Link: https://lore.kernel.org/r/20201008153602.9467-4-ardb@kernel.org
Signed-off-by: Catalin Marinas

arm64: mm: extend linear region for 52-bit VA configurations

2020-11-09T17:15:37+00:00

For historical reasons, the arm64 kernel VA space is configured as two
equally sized halves, i.e., on a 48-bit VA build, the VA space is split
into a 47-bit vmalloc region and a 47-bit linear region.

When support for 52-bit virtual addressing was added, this equal split
was kept, resulting in a substantial waste of virtual address space in
the linear region:

                           48-bit VA                     52-bit VA
  0xffff_ffff_ffff_ffff +-------------+               +-------------+
                        |   vmalloc   |               |   vmalloc   |
  0xffff_8000_0000_0000 +-------------+ _PAGE_END(48) +-------------+
                        |   linear    |               :             :
  0xffff_0000_0000_0000 +-------------+               :             :
                        :             :               :             :
                        :             :               :             :
                        :             :               :             :
                        :             :               :  currently  :
                        :  unusable   :               :             :
                        :             :               :   unused    :
                        :     by      :               :             :
                        :             :               :             :
                        :  hardware   :               :             :
                        :             :               :             :
  0xfff8_0000_0000_0000 :             : _PAGE_END(52) +-------------+
                        :             :               |             |
                        :             :               |             |
                        :             :               |             |
                        :             :               |             |
                        :             :               |             |
                        :  unusable   :               |             |
                        :             :               |   linear    |
                        :     by      :               |             |
                        :             :               |   region    |
                        :  hardware   :               |             |
                        :             :               |             |
                        :             :               |             |
                        :             :               |             |
                        :             :               |             |
                        :             :               |             |
                        :             :               |             |
  0xfff0_0000_0000_0000 +-------------+  PAGE_OFFSET  +-------------+

As illustrated above, the 52-bit VA kernel uses 47 bits for the vmalloc
space (as before), to ensure that a single 64k granule kernel image can
support any 64k granule capable system, regardless of whether it supports
the 52-bit virtual addressing extension. However, due to the fact that
the VA space is still split in equal halves, the linear region is only
2^51 bytes in size, wasting almost half of the 52-bit VA space.

Let's fix this, by abandoning the equal split, and simply assigning all
VA space outside of the vmalloc region to the linear region.

The KASAN shadow region is reconfigured so that it ends at the start of
the vmalloc region, and grows downwards. That way, the arrangement of
the vmalloc space (which contains kernel mappings, modules, BPF region,
the vmemmap array etc) is identical between non-KASAN and KASAN builds,
which aids debugging.

Signed-off-by: Ard Biesheuvel 
Reviewed-by: Steve Capper 
Link: https://lore.kernel.org/r/20201008153602.9467-3-ardb@kernel.org
Signed-off-by: Catalin Marinas