linux-stable.git/arch/x86/kernel/setup_percpu.c, branch linux-rolling-lts

x86/smp: Move this_cpu_off to percpu hot section

2025-03-04T19:30:33+00:00

No functional change.

Signed-off-by: Brian Gerst 
Signed-off-by: Ingo Molnar 
Acked-by: Uros Bizjak 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Link: https://lore.kernel.org/r/20250303165246.2175811-12-brgerst@gmail.com

x86/smp: Move cpu number to percpu hot section

2025-03-04T19:30:33+00:00

No functional change.

Signed-off-by: Brian Gerst 
Signed-off-by: Ingo Molnar 
Acked-by: Uros Bizjak 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Link: https://lore.kernel.org/r/20250303165246.2175811-5-brgerst@gmail.com

x86/percpu/64: Use relative percpu offsets

2025-02-18T09:15:27+00:00

The percpu section is currently linked at absolute address 0, because
older compilers hard-coded the stack protector canary value at a fixed
offset from the start of the GS segment.  Now that the canary is a
normal percpu variable, the percpu section does not need to be linked
at a specific address.

x86-64 will now calculate the percpu offsets as the delta between the
initial percpu address and the dynamically allocated memory, like other
architectures.  Note that GSBASE is limited to the canonical address
width (48 or 57 bits, sign-extended).  As long as the kernel text,
modules, and the dynamically allocated percpu memory are all in the
negative address space, the delta will not overflow this limit.

Signed-off-by: Brian Gerst 
Signed-off-by: Ingo Molnar 
Reviewed-by: Ard Biesheuvel 
Reviewed-by: Uros Bizjak 
Cc: Linus Torvalds 
Link: https://lore.kernel.org/r/20250123190747.745588-9-brgerst@gmail.com

x86/asm: Add DB flag to 32-bit percpu GDT entry

2023-12-20T09:57:51+00:00

The D/B size flag for the 32-bit percpu GDT entry was not set.

The Intel manual (vol 3, section 3.4.5) only specifies the meaning of
this flag for three cases:

 1) code segments used for %cs -- doesn't apply here

 2) stack segments used for %ss -- doesn't apply

 3) expand-down data segments -- but we don't have the expand-down flag
    set, so it also doesn't apply here

The flag likely doesn't do anything here, although the manual does also
say: "This flag should always be set to 1 for 32-bit code and data
segments [...]" so we should probably do it anyway.

Signed-off-by: Vegard Nossum 
Signed-off-by: Ingo Molnar 
Acked-by: Linus Torvalds 
Link: https://lore.kernel.org/r/20231219151200.2878271-6-vegard.nossum@oracle.com

x86/asm: Replace magic numbers in GDT descriptors, script-generated change

2023-12-20T09:57:38+00:00

Actually replace the numeric values by the new symbolic values.

I used this to find all the existing users of the GDT_ENTRY*() macros:

  $ git grep -P 'GDT_ENTRY(_INIT)?\('

Some of the lines will exceed 80 characters, but some of them will be
shorter again in the next couple of patches.

Signed-off-by: Vegard Nossum 
Signed-off-by: Ingo Molnar 
Acked-by: Linus Torvalds 
Link: https://lore.kernel.org/r/20231219151200.2878271-4-vegard.nossum@oracle.com

x86/apic/32: Remove x86_cpu_to_logical_apicid

2023-08-09T18:58:23+00:00

This per CPU variable is just yet another form of voodoo programming. The
boot ordering is:

  per_cpu(x86_cpu_to_logical_apicid, cpu) = 1U << cpu;

  .....

  setup_apic()
     apic->init_apic_ldr()
       default_init_apic_ldr()
         apic_write(SET_APIC_LOGICAL_ID(1UL << smp_processor_id(), APIC_LDR);

     id = GET_APIC_LOGICAL_ID(apic_read(APIC_LDR);
     WARN_ON(id != per_cpu(x86_cpu_to_logical_apicid, cpu));
     per_cpu(x86_cpu_to_logical_apicid, cpu) = id;

So first write the default into LDR and then validate it against the same default
which was set up during early boot APIC enumeration.

Brilliant, isn't it?

The comment above the per CPU variable declaration describes it well:
'Let's keep it ugly for now.'

Remove the useless gunk and use '1U << cpu' consistently all over the place.

Signed-off-by: Thomas Gleixner 
Signed-off-by: Dave Hansen 
Acked-by: Peter Zijlstra (Intel) 
Tested-by: Michael Kelley 
Tested-by: Sohil Mehta 
Tested-by: Juergen Gross  # Xen PV (dom0 and unpriv. guest)

x86/apic: Remove pointless x86_bios_cpu_apicid

2023-08-09T18:58:17+00:00

It's a useless copy of x86_cpu_to_apicid.

Signed-off-by: Thomas Gleixner 
Signed-off-by: Dave Hansen 
Acked-by: Peter Zijlstra (Intel) 
Tested-by: Michael Kelley 
Tested-by: Sohil Mehta 
Tested-by: Juergen Gross  # Xen PV (dom0 and unpriv. guest)

Merge tag 'x86_core_for_v6.2' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2022-12-14T23:03:00+00:00

Pull x86 core updates from Borislav Petkov:

 - Add the call depth tracking mitigation for Retbleed which has been
   long in the making. It is a lighterweight software-only fix for
   Skylake-based cores where enabling IBRS is a big hammer and causes a
   significant performance impact.

   What it basically does is, it aligns all kernel functions to 16 bytes
   boundary and adds a 16-byte padding before the function, objtool
   collects all functions' locations and when the mitigation gets
   applied, it patches a call accounting thunk which is used to track
   the call depth of the stack at any time.

   When that call depth reaches a magical, microarchitecture-specific
   value for the Return Stack Buffer, the code stuffs that RSB and
   avoids its underflow which could otherwise lead to the Intel variant
   of Retbleed.

   This software-only solution brings a lot of the lost performance
   back, as benchmarks suggest:

       https://lore.kernel.org/all/20220915111039.092790446@infradead.org/

   That page above also contains a lot more detailed explanation of the
   whole mechanism

 - Implement a new control flow integrity scheme called FineIBT which is
   based on the software kCFI implementation and uses hardware IBT
   support where present to annotate and track indirect branches using a
   hash to validate them

 - Other misc fixes and cleanups

* tag 'x86_core_for_v6.2' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (80 commits)
  x86/paravirt: Use common macro for creating simple asm paravirt functions
  x86/paravirt: Remove clobber bitmask from .parainstructions
  x86/debug: Include percpu.h in debugreg.h to get DECLARE_PER_CPU() et al
  x86/cpufeatures: Move X86_FEATURE_CALL_DEPTH from bit 18 to bit 19 of word 11, to leave space for WIP X86_FEATURE_SGX_EDECCSSA bit
  x86/Kconfig: Enable kernel IBT by default
  x86,pm: Force out-of-line memcpy()
  objtool: Fix weak hole vs prefix symbol
  objtool: Optimize elf_dirty_reloc_sym()
  x86/cfi: Add boot time hash randomization
  x86/cfi: Boot time selection of CFI scheme
  x86/ibt: Implement FineIBT
  objtool: Add --cfi to generate the .cfi_sites section
  x86: Add prefix symbols for function padding
  objtool: Add option to generate prefix symbols
  objtool: Avoid O(bloody terrible) behaviour -- an ode to libelf
  objtool: Slice up elf_create_section_symbol()
  kallsyms: Revert "Take callthunks into account"
  x86: Unconfuse CONFIG_ and X86_FEATURE_ namespaces
  x86/retpoline: Fix crash printing warning
  x86/paravirt: Fix a !PARAVIRT build warning
  ...

stackprotector: move get_random_canary() into stackprotector.h

2022-11-18T01:18:10+00:00

This has nothing to do with random.c and everything to do with stack
protectors. Yes, it uses randomness. But many things use randomness.
random.h and random.c are concerned with the generation of randomness,
not with each and every use. So move this function into the more
specific stackprotector.h file where it belongs.

Acked-by: Greg Kroah-Hartman 
Signed-off-by: Jason A. Donenfeld

x86/percpu: Move cpu_number next to current_task

2022-10-17T14:41:04+00:00

Also add cpu_number to the pcpu_hot structure, it is often referenced
and this cacheline is there.

Signed-off-by: Thomas Gleixner 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://lore.kernel.org/r/20220915111145.387678283@infradead.org