linux-stable.git/arch/arm64/kernel/sleep.S, branch linux-4.7.y

arm64: kernel: avoid literal load of virtual address with MMU off

2016-09-07T06:34:49+00:00

commit bc9f3d7788a88d080a30599bde68f383daf8f8a5 upstream.

Literal loads of virtual addresses are subject to runtime relocation when
CONFIG_RELOCATABLE=y, and given that the relocation routines run with the
MMU and caches enabled, literal loads of relocated values performed with
the MMU off are not guaranteed to return the latest value unless the
memory covering the literal is cleaned to the PoC explicitly.

So defer the literal load until after the MMU has been enabled, just like
we do for primary_switch() and secondary_switch() in head.S.

Fixes: 1e48ef7fcc37 ("arm64: add support for building vmlinux as a relocatable PIE binary")
Signed-off-by: Ard Biesheuvel 
Acked-by: Mark Rutland 
Signed-off-by: Catalin Marinas 
Signed-off-by: Greg Kroah-Hartman

arm64: Change cpu_resume() to enable mmu early then access sleep_sp by va

2016-04-28T11:05:46+00:00

By enabling the MMU early in cpu_resume(), the sleep_save_sp and stack can
be accessed by VA, which avoids the need to convert-addresses and clean to
PoC on the suspend path.

MMU setup is shared with the boot path, meaning the swapper_pg_dir is
restored directly: ttbr1_el1 is no longer saved/restored.

struct sleep_save_sp is removed, replacing it with a single array of
pointers.

cpu_do_{suspend,resume} could be further reduced to not restore: cpacr_el1,
mdscr_el1, tcr_el1, vbar_el1 and sctlr_el1, all of which are set by
__cpu_setup(). However these values all contain res0 bits that may be used
to enable future features.

Signed-off-by: James Morse 
Reviewed-by: Lorenzo Pieralisi 
Reviewed-by: Catalin Marinas 
Signed-off-by: Will Deacon

arm64: kernel: Rework finisher callback out of __cpu_suspend_enter()

2016-04-28T11:05:46+00:00

Hibernate could make use of the cpu_suspend() code to save/restore cpu
state, however it needs to be able to return '0' from the 'finisher'.

Rework cpu_suspend() so that the finisher is called from C code,
independently from the save/restore of cpu state. Space to save the context
in is allocated in the caller's stack frame, and passed into
__cpu_suspend_enter().

Hibernate's use of this API will look like a copy of the cpu_suspend()
function.

Signed-off-by: James Morse 
Acked-by: Lorenzo Pieralisi 
Reviewed-by: Catalin Marinas 
Signed-off-by: Will Deacon

arm64: kasan: clear stale stack poison

2016-03-09T23:43:42+00:00

Functions which the compiler has instrumented for KASAN place poison on
the stack shadow upon entry and remove this poison prior to returning.

In the case of cpuidle, CPUs exit the kernel a number of levels deep in
C code.  Any instrumented functions on this critical path will leave
portions of the stack shadow poisoned.

If CPUs lose context and return to the kernel via a cold path, we
restore a prior context saved in __cpu_suspend_enter are forgotten, and
we never remove the poison they placed in the stack shadow area by
functions calls between this and the actual exit of the kernel.

Thus, (depending on stackframe layout) subsequent calls to instrumented
functions may hit this stale poison, resulting in (spurious) KASAN
splats to the console.

To avoid this, clear any stale poison from the idle thread for a CPU
prior to bringing a CPU online.

Signed-off-by: Mark Rutland 
Acked-by: Catalin Marinas 
Reviewed-by: Andrey Ryabinin 
Reviewed-by: Lorenzo Pieralisi 
Cc: Alexander Potapenko 
Cc: Catalin Marinas 
Cc: Will Deacon 
Cc: Ingo Molnar 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

arm64: Store struct thread_info in sp_el0

2015-12-08T11:40:48+00:00

There is need for figuring out how to manage struct thread_info data when
IRQ stack is introduced. struct thread_info information should be copied
to IRQ stack under the current thread_info calculation logic whenever
context switching is invoked. This is too expensive to keep supporting
the approach.

Instead, this patch pays attention to sp_el0 which is an unused scratch
register in EL1 context. sp_el0 utilization not only simplifies the
management, but also prevents text section size from being increased
largely due to static allocated IRQ stack as removing masking operation
using THREAD_SIZE in many places.

Reviewed-by: Catalin Marinas 
Signed-off-by: Jungseok Lee 
Signed-off-by: James Morse 
Signed-off-by: Will Deacon

arm64: mm: ensure patched kernel text is fetched from PoU

2015-08-05T09:05:20+00:00

The arm64 booting document requires that the bootloader has cleaned the
kernel image to the PoC. However, when a CPU re-enters the kernel due to
either a CPU hotplug "on" event or resuming from a low-power state (e.g.
cpuidle), the kernel text may in-fact be dirty at the PoU due to things
like alternative patching or even module loading.

Thanks to I-cache speculation with the MMU off, stale instructions could
be fetched prior to enabling the MMU, potentially leading to crashes
when executing regions of code that have been modified at runtime.

This patch addresses the issue by ensuring that the local I-cache is
invalidated immediately after a CPU has enabled its MMU but before
jumping out of the identity mapping. Any stale instructions fetched from
the PoC will then be discarded and refetched correctly from the PoU.
Patching kernel text executed prior to the MMU being enabled is
prohibited, so the early entry code will always be clean.

Reviewed-by: Mark Rutland 
Tested-by: Mark Rutland 
Signed-off-by: Will Deacon

arm64: restore cpu suspend/resume functionality

2015-07-31T16:40:52+00:00

Commit 4b3dc9679cf7 ("arm64: force CONFIG_SMP=y and remove redundant #ifdefs")
accidentally retained code for !CONFIG_SMP in cpu_resume function. This
resulted in the hash index being zeroed in x7 after proper computation,
which is then used to get the cpu context pointer while resuming.

This patch removes the remanant code and restores back the cpu suspend/
resume functionality.

Fixes: 4b3dc9679cf7 ("arm64: force CONFIG_SMP=y and remove redundant #ifdefs")
Signed-off-by: Sudeep Holla 
Cc: Lorenzo Pieralisi 
Cc: Catalin Marinas 
Cc: Will Deacon 
Signed-off-by: Will Deacon

arm64: force CONFIG_SMP=y and remove redundant #ifdefs

2015-07-27T10:08:40+00:00

Nobody seems to be producing !SMP systems anymore, so this is just
becoming a source of kernel bugs, particularly if people want to use
coherent DMA with non-shared pages.

This patch forces CONFIG_SMP=y for arm64, removing a modest amount of
code in the process.

Signed-off-by: Will Deacon

arm64: drop sleep_idmap_phys and clean up cpu_resume()

2015-06-02T16:45:07+00:00

Two cleanups of the asm function cpu_resume():
- The global variable sleep_idmap_phys always points to idmap_pg_dir,
  so we can just use that value directly in the CPU resume path.
- Unclutter the load of sleep_save_sp::save_ptr_stash_phys.

Acked-by: Lorenzo Pieralisi 
Tested-by: Lorenzo Pieralisi 
Signed-off-by: Ard Biesheuvel 
Signed-off-by: Catalin Marinas

arm64: reduce ID map to a single page

2015-06-02T16:44:51+00:00

Commit ea8c2e112445 ("arm64: Extend the idmap to the whole kernel
image") changed the early page table code so that the entire kernel
Image is covered by the identity map. This allows functions that
need to enable or disable the MMU to reside anywhere in the kernel
Image.

However, this change has the unfortunate side effect that the Image
cannot cross a physical 512 MB alignment boundary anymore, since the
early page table code cannot deal with the Image crossing a /virtual/
512 MB alignment boundary.

So instead, reduce the ID map to a single page, that is populated by
the contents of the .idmap.text section. Only three functions reside
there at the moment: __enable_mmu(), cpu_resume_mmu() and cpu_reset().
If new code is introduced that needs to manipulate the MMU state, it
should be added to this section as well.

Reviewed-by: Mark Rutland 
Tested-by: Mark Rutland 
Signed-off-by: Ard Biesheuvel 
Signed-off-by: Catalin Marinas