linux-stable.git/arch/x86/kernel/head64.c, branch linux-rolling-stable

x86/boot: Create a confined code area for startup code

2025-09-03T16:00:01+00:00

In order to be able to have tight control over which code may execute
from the early 1:1 mapping of memory, but still link vmlinux as a single
executable, prefix all symbol references in startup code with __pi_, and
invoke it from outside using the __pi_ prefix.

Use objtool to check that no absolute symbol references are present in
the startup code, as these cannot be used from code running from the 1:1
mapping.

Note that this also requires disabling the latent-entropy GCC plugin, as
the global symbol references that it injects would require explicit
exports, and given that the startup code rarely executes more than once,
it is not a useful source of entropy anyway.

Signed-off-by: Ard Biesheuvel 
Signed-off-by: Borislav Petkov (AMD) 
Link: https://lore.kernel.org/20250828102202.1849035-43-ardb+git@google.com

x86/boot: Provide PIC aliases for 5-level paging related constants

2025-09-03T15:59:40+00:00

Provide PIC aliases for the global variables related to 5-level paging, so
that the startup code can access them in order to populate the kernel page
tables.

Signed-off-by: Ard Biesheuvel 
Signed-off-by: Borislav Petkov (AMD) 
Link: https://lore.kernel.org/20250828102202.1849035-35-ardb+git@google.com

x86/mm/64: Make 5-level paging support unconditional

2025-05-17T08:38:16+00:00

Both Intel and AMD CPUs support 5-level paging, which is expected to
become more widely adopted in the future. All major x86 Linux
distributions have the feature enabled.

Remove CONFIG_X86_5LEVEL and related #ifdeffery for it to make it more readable.

Suggested-by: Borislav Petkov 
Signed-off-by: Kirill A. Shutemov 
Signed-off-by: Ingo Molnar 
Reviewed-by: Ard Biesheuvel 
Reviewed-by: Borislav Petkov (AMD) 
Cc: Dave Hansen 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Link: https://lore.kernel.org/r/20250516123306.3812286-4-kirill.shutemov@linux.intel.com

x86/mm/64: Always use dynamic memory layout

2025-05-17T08:33:44+00:00

Dynamic memory layout is used by KASLR and 5-level paging.

CONFIG_X86_5LEVEL is going to be removed, making 5-level paging support
unconditional which requires unconditional support of dynamic memory
layout.

Remove CONFIG_DYNAMIC_MEMORY_LAYOUT.

Signed-off-by: Kirill A. Shutemov 
Signed-off-by: Ingo Molnar 
Reviewed-by: Ard Biesheuvel 
Reviewed-by: Borislav Petkov (AMD) 
Cc: Jan Kiszka 
Cc: Kieran Bingham 
Cc: H. Peter Anvin 
Cc: Dave Hansen 
Cc: Linus Torvalds 
Link: https://lore.kernel.org/r/20250516123306.3812286-2-kirill.shutemov@linux.intel.com

x86/boot: Defer initialization of VM space related global variables

2025-05-14T08:06:35+00:00

The global pseudo-constants 'page_offset_base', 'vmalloc_base' and
'vmemmap_base' are not used extremely early during the boot, and cannot be
used safely until after the KASLR memory randomization code in
kernel_randomize_memory() executes, which may update their values.

So there is no point in setting these variables extremely early, and it
can wait until after the kernel itself is mapped and running from its
permanent virtual mapping.

Signed-off-by: Ard Biesheuvel 
Signed-off-by: Ingo Molnar 
Cc: Linus Torvalds 
Link: https://lore.kernel.org/r/20250513111157.717727-9-ardb+git@google.com

x86/boot: Add a bunch of PIC aliases

2025-05-04T13:59:43+00:00

Add aliases for all the data objects that the startup code references -
this is needed so that this code can be moved into its own confined area
where it can only access symbols that have a __pi_ prefix.

Signed-off-by: Ard Biesheuvel 
Signed-off-by: Ingo Molnar 
Cc: Arnd Bergmann 
Cc: David Woodhouse 
Cc: Dionna Amalie Glaze 
Cc: H. Peter Anvin 
Cc: Kees Cook 
Cc: Kevin Loughlin 
Cc: Len Brown 
Cc: Linus Torvalds 
Cc: Rafael J. Wysocki 
Cc: Tom Lendacky 
Cc: linux-efi@vger.kernel.org
Link: https://lore.kernel.org/r/20250504095230.2932860-39-ardb+git@google.com

x86/boot: Move early_setup_gdt() back into head64.c

2025-05-04T13:27:23+00:00

Move early_setup_gdt() out of the startup code that is callable from the
1:1 mapping - this is not needed, and instead, it is better to expose
the helper that does reside in __head directly.

This reduces the amount of code that needs special checks for 1:1
execution suitability. In particular, it avoids dealing with the GHCB
page (and its physical address) in startup code, which runs from the
1:1 mapping, making physical to virtual translations ambiguous.

Signed-off-by: Ard Biesheuvel 
Signed-off-by: Ingo Molnar 
Cc: Arnd Bergmann 
Cc: David Woodhouse 
Cc: Dionna Amalie Glaze 
Cc: H. Peter Anvin 
Cc: Kees Cook 
Cc: Kevin Loughlin 
Cc: Len Brown 
Cc: Linus Torvalds 
Cc: Rafael J. Wysocki 
Cc: Tom Lendacky 
Cc: linux-efi@vger.kernel.org
Link: https://lore.kernel.org/r/20250504095230.2932860-26-ardb+git@google.com

x86/boot: Move early kernel mapping code into startup/

2025-04-12T09:13:05+00:00

The startup code that constructs the kernel virtual mapping runs from
the 1:1 mapping of memory itself, and therefore, cannot use absolute
symbol references. Before making changes in subsequent patches, move
this code into a separate source file under arch/x86/boot/startup/ where
all such code will be kept from now on.

Signed-off-by: Ard Biesheuvel 
Signed-off-by: Ingo Molnar 
Cc: Dionna Amalie Glaze 
Cc: H. Peter Anvin 
Cc: Kees Cook 
Cc: Kevin Loughlin 
Cc: Linus Torvalds 
Cc: Tom Lendacky 
Cc: linux-efi@vger.kernel.org
Link: https://lore.kernel.org/r/20250410134117.3713574-16-ardb+git@google.com

x86/boot: Move the early GDT/IDT setup code into startup/

2025-04-12T09:13:04+00:00

Move the early GDT/IDT setup code that runs long before the kernel
virtual mapping is up into arch/x86/boot/startup/, and build it in a way
that ensures that the code tolerates being called from the 1:1 mapping
of memory. The code itself is left unchanged by this patch.

Also tweak the sed symbol matching pattern in the decompressor to match
on lower case 't' or 'b', as these will be emitted by Clang for symbols
with hidden linkage.

Signed-off-by: Ard Biesheuvel 
Signed-off-by: Ingo Molnar 
Cc: Dionna Amalie Glaze 
Cc: H. Peter Anvin 
Cc: Kees Cook 
Cc: Kevin Loughlin 
Cc: Linus Torvalds 
Cc: Tom Lendacky 
Cc: linux-efi@vger.kernel.org
Link: https://lore.kernel.org/r/20250410134117.3713574-15-ardb+git@google.com

x86/asm: Make rip_rel_ptr() usable from fPIC code

2025-04-12T09:13:04+00:00

RIP_REL_REF() is used in non-PIC C code that is called very early,
before the kernel virtual mapping is up, which is the mapping that the
linker expects. It is currently used in two different ways:

 - to refer to the value of a global variable, including as an lvalue in
   assignments;

 - to take the address of a global variable via the mapping that the code
   currently executes at.

The former case is only needed in non-PIC code, as PIC code will never
use absolute symbol references when the address of the symbol is not
being used. But taking the address of a variable in PIC code may still
require extra care, as a stack allocated struct assignment may be
emitted as a memcpy() from a statically allocated copy in .rodata.

For instance, this

  void startup_64_setup_gdt_idt(void)
  {
        struct desc_ptr startup_gdt_descr = {
                .address = (__force unsigned long)gdt_page.gdt,
                .size    = GDT_SIZE - 1,
        };

may result in an absolute symbol reference in PIC code, even though the
struct is allocated on the stack and populated at runtime.

To address this case, make rip_rel_ptr() accessible in PIC code, and
update any existing uses where the address of a global variable is
taken using RIP_REL_REF.

Once all code of this nature has been moved into arch/x86/boot/startup
and built with -fPIC, RIP_REL_REF() can be retired, and only
rip_rel_ptr() will remain.

Signed-off-by: Ard Biesheuvel 
Signed-off-by: Ingo Molnar 
Cc: Dionna Amalie Glaze 
Cc: H. Peter Anvin 
Cc: Kees Cook 
Cc: Kevin Loughlin 
Cc: Linus Torvalds 
Cc: Tom Lendacky 
Cc: linux-efi@vger.kernel.org
Link: https://lore.kernel.org/r/20250410134117.3713574-14-ardb+git@google.com