linux.git/arch/arm64/kernel/sleep.S, branch v4.12

arm64: move sp_el0 and tpidr_el1 into cpu_suspend_ctx

2016-11-11T18:25:44+00:00

When returning from idle, we rely on the fact that thread_info lives at
the end of the kernel stack, and restore this by masking the saved stack
pointer. Subsequent patches will sever the relationship between the
stack and thread_info, and to cater for this we must save/restore sp_el0
explicitly, storing it in cpu_suspend_ctx.

As cpu_suspend_ctx must be doubleword aligned, this leaves us with an
extra slot in cpu_suspend_ctx. We can use this to save/restore tpidr_el1
in the same way, which simplifies the code, avoiding pointer chasing on
the restore path (as we no longer need to load thread_info::cpu followed
by the relevant slot in __per_cpu_offset based on this).

This patch stashes both registers in cpu_suspend_ctx.

Signed-off-by: Mark Rutland 
Tested-by: Laura Abbott 
Cc: James Morse 
Cc: Lorenzo Pieralisi 
Cc: Will Deacon 
Signed-off-by: Catalin Marinas

kprobes: Unpoison stack in jprobe_return() for KASAN

2016-10-16T09:02:31+00:00

I observed false KSAN positives in the sctp code, when
sctp uses jprobe_return() in jsctp_sf_eat_sack().

The stray 0xf4 in shadow memory are stack redzones:

[     ] ==================================================================
[     ] BUG: KASAN: stack-out-of-bounds in memcmp+0xe9/0x150 at addr ffff88005e48f480
[     ] Read of size 1 by task syz-executor/18535
[     ] page:ffffea00017923c0 count:0 mapcount:0 mapping:          (null) index:0x0
[     ] flags: 0x1fffc0000000000()
[     ] page dumped because: kasan: bad access detected
[     ] CPU: 1 PID: 18535 Comm: syz-executor Not tainted 4.8.0+ #28
[     ] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
[     ]  ffff88005e48f2d0 ffffffff82d2b849 ffffffff0bc91e90 fffffbfff10971e8
[     ]  ffffed000bc91e90 ffffed000bc91e90 0000000000000001 0000000000000000
[     ]  ffff88005e48f480 ffff88005e48f350 ffffffff817d3169 ffff88005e48f370
[     ] Call Trace:
[     ]  [] dump_stack+0x12e/0x185
[     ]  [] kasan_report+0x489/0x4b0
[     ]  [] __asan_report_load1_noabort+0x19/0x20
[     ]  [] memcmp+0xe9/0x150
[     ]  [] depot_save_stack+0x176/0x5c0
[     ]  [] save_stack+0xb1/0xd0
[     ]  [] kasan_slab_free+0x72/0xc0
[     ]  [] kfree+0xc8/0x2a0
[     ]  [] skb_free_head+0x79/0xb0
[     ]  [] skb_release_data+0x37a/0x420
[     ]  [] skb_release_all+0x4f/0x60
[     ]  [] consume_skb+0x138/0x370
[     ]  [] sctp_chunk_put+0xcb/0x180
[     ]  [] sctp_chunk_free+0x58/0x70
[     ]  [] sctp_inq_pop+0x68f/0xef0
[     ]  [] sctp_assoc_bh_rcv+0xd6/0x4b0
[     ]  [] sctp_inq_push+0x131/0x190
[     ]  [] sctp_backlog_rcv+0xe9/0xa20
[ ... ]
[     ] Memory state around the buggy address:
[     ]  ffff88005e48f380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[     ]  ffff88005e48f400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[     ] >ffff88005e48f480: f4 f4 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[     ]                    ^
[     ]  ffff88005e48f500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[     ]  ffff88005e48f580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[     ] ==================================================================

KASAN stack instrumentation poisons stack redzones on function entry
and unpoisons them on function exit. If a function exits abnormally
(e.g. with a longjmp like jprobe_return()), stack redzones are left
poisoned. Later this leads to random KASAN false reports.

Unpoison stack redzones in the frames we are going to jump over
before doing actual longjmp in jprobe_return().

Signed-off-by: Dmitry Vyukov 
Acked-by: Masami Hiramatsu 
Reviewed-by: Mark Rutland 
Cc: Mark Rutland 
Cc: Catalin Marinas 
Cc: Andrey Ryabinin 
Cc: Lorenzo Pieralisi 
Cc: Alexander Potapenko 
Cc: Will Deacon 
Cc: Andrew Morton 
Cc: Ananth N Mavinakayanahalli 
Cc: Anil S Keshavamurthy 
Cc: "David S. Miller" 
Cc: Masami Hiramatsu 
Cc: kasan-dev@googlegroups.com
Cc: surovegin@google.com
Cc: rostedt@goodmis.org
Link: http://lkml.kernel.org/r/1476454043-101898-1-git-send-email-dvyukov@google.com
Signed-off-by: Ingo Molnar

arm64: kernel: re-export _cpu_resume() from sleep.S

2016-09-05T09:24:55+00:00

Commit b5fe242972ef ("arm64: kernel: fix style issues in sleep.S")
changed the linkage of _cpu_resume() to local, even though the symbol
is also referenced from hibernate.c. So revert this change.

Signed-off-by: Ard Biesheuvel 
Signed-off-by: Will Deacon

arm64: kernel: use x30 for __enable_mmu return address

2016-09-02T10:47:51+00:00

Using x27 for passing to __enable_mmu what is essentially the return
address makes the code look more complicated than it needs to be. So
switch to x30/lr, and update the secondary and cpu_resume call sites to
simply call __enable_mmu as an ordinary function, with a bl instruction.
This requires the callers to be covered by .idmap.text.

Reviewed-by: Mark Rutland 
Signed-off-by: Ard Biesheuvel 
Signed-off-by: Will Deacon

arm64: kernel: fix style issues in sleep.S

2016-09-02T10:21:14+00:00

This fixes a number of style issues in sleep.S. No functional changes are
intended:
- replace absolute literal references with relative references in
  __cpu_suspend_enter(), which executes from its virtual address
- replace explicit lr assignment plus branch with bl in cpu_resume(), which
  aligns it with stext() and secondary_startup()
- don't export _cpu_resume()
- use adr_l for mpidr_hash reference, and fix the incorrect accompanying
  comment, which has been out of date since commit cabe1c81ea5be983 ("arm64:
  Change cpu_resume() to enable mmu early then access sleep_sp by va")
- replace leading spaces with tabs, and add a bit of whitespace for
  readability

Reviewed-by: Mark Rutland 
Signed-off-by: Ard Biesheuvel 
Signed-off-by: Will Deacon

arm64: vmlinux.ld: Add mmuoff data sections and move mmuoff text into idmap

2016-08-25T17:00:30+00:00

Resume from hibernate needs to clean any text executed by the kernel with
the MMU off to the PoC. Collect these functions together into the
.idmap.text section as all this code is tightly coupled and also needs
the same cleaning after resume.

Data is more complicated, secondary_holding_pen_release is written with
the MMU on, clean and invalidated, then read with the MMU off. In contrast
__boot_cpu_mode is written with the MMU off, the corresponding cache line
is invalidated, so when we read it with the MMU on we don't get stale data.
These cache maintenance operations conflict with each other if the values
are within a Cache Writeback Granule (CWG) of each other.
Collect the data into two sections .mmuoff.data.read and .mmuoff.data.write,
the linker script ensures mmuoff.data.write section is aligned to the
architectural maximum CWG of 2KB.

Signed-off-by: James Morse 
Cc: Ard Biesheuvel 
Cc: Mark Rutland 
Reviewed-by: Catalin Marinas 
Signed-off-by: Will Deacon

arm64: head.S: get rid of x25 and x26 with 'global' scope

2016-08-22T13:25:15+00:00

Currently, x25 and x26 hold the physical addresses of idmap_pg_dir
and swapper_pg_dir, respectively, when running early boot code. But
having registers with 'global' scope in files that contain different
sections with different lifetimes, and that are called by different
CPUs at different times is a bit messy, especially since stashing the
values does not buy us anything in terms of code size or clarity.

So simply replace each reference to x25 or x26 with an adrp instruction
referring to idmap_pg_dir or swapper_pg_dir directly.

Acked-by: Mark Rutland 
Signed-off-by: Ard Biesheuvel 
Signed-off-by: Will Deacon

arm64: kernel: avoid literal load of virtual address with MMU off

2016-08-17T16:37:37+00:00

Literal loads of virtual addresses are subject to runtime relocation when
CONFIG_RELOCATABLE=y, and given that the relocation routines run with the
MMU and caches enabled, literal loads of relocated values performed with
the MMU off are not guaranteed to return the latest value unless the
memory covering the literal is cleaned to the PoC explicitly.

So defer the literal load until after the MMU has been enabled, just like
we do for primary_switch() and secondary_switch() in head.S.

Fixes: 1e48ef7fcc37 ("arm64: add support for building vmlinux as a relocatable PIE binary")
Cc:  # 4.6+
Signed-off-by: Ard Biesheuvel 
Acked-by: Mark Rutland 
Signed-off-by: Catalin Marinas

arm64: Change cpu_resume() to enable mmu early then access sleep_sp by va

2016-04-28T11:05:46+00:00

By enabling the MMU early in cpu_resume(), the sleep_save_sp and stack can
be accessed by VA, which avoids the need to convert-addresses and clean to
PoC on the suspend path.

MMU setup is shared with the boot path, meaning the swapper_pg_dir is
restored directly: ttbr1_el1 is no longer saved/restored.

struct sleep_save_sp is removed, replacing it with a single array of
pointers.

cpu_do_{suspend,resume} could be further reduced to not restore: cpacr_el1,
mdscr_el1, tcr_el1, vbar_el1 and sctlr_el1, all of which are set by
__cpu_setup(). However these values all contain res0 bits that may be used
to enable future features.

Signed-off-by: James Morse 
Reviewed-by: Lorenzo Pieralisi 
Reviewed-by: Catalin Marinas 
Signed-off-by: Will Deacon

arm64: kernel: Rework finisher callback out of __cpu_suspend_enter()

2016-04-28T11:05:46+00:00

Hibernate could make use of the cpu_suspend() code to save/restore cpu
state, however it needs to be able to return '0' from the 'finisher'.

Rework cpu_suspend() so that the finisher is called from C code,
independently from the save/restore of cpu state. Space to save the context
in is allocated in the caller's stack frame, and passed into
__cpu_suspend_enter().

Hibernate's use of this API will look like a copy of the cpu_suspend()
function.

Signed-off-by: James Morse 
Acked-by: Lorenzo Pieralisi 
Reviewed-by: Catalin Marinas 
Signed-off-by: Will Deacon