linux.git/arch/s390/lib, branch v3.15

s390/uaccess: fix possible register corruption in strnlen_user_srst()

2014-04-11T11:53:33+00:00

The whole point of the out-of-line strnlen_user_srst() function was to
avoid corruption of register 0 due to register asm assignment.
However 'somebody' :) forgot to remove the update_primary_asce() function
call, which may clobber register 0 contents.
So let's remove that call and also move the size check to the calling
function.

Signed-off-by: Heiko Carstens 
Signed-off-by: Martin Schwidefsky

s390/uaccess: rework uaccess code - fix locking issues

2014-04-03T12:31:04+00:00

The current uaccess code uses a page table walk in some circumstances,
e.g. in case of the in atomic futex operations or if running on old
hardware which doesn't support the mvcos instruction.

However it turned out that the page table walk code does not correctly
lock page tables when accessing page table entries.
In other words: a different cpu may invalidate a page table entry while
the current cpu inspects the pte. This may lead to random data corruption.

Adding correct locking however isn't trivial for all uaccess operations.
Especially copy_in_user() is problematic since that requires to hold at
least two locks, but must be protected against ABBA deadlock when a
different cpu also performs a copy_in_user() operation.

So the solution is a different approach where we change address spaces:

User space runs in primary address mode, or access register mode within
vdso code, like it currently already does.

The kernel usually also runs in home space mode, however when accessing
user space the kernel switches to primary or secondary address mode if
the mvcos instruction is not available or if a compare-and-swap (futex)
instruction on a user space address is performed.
KVM however is special, since that requires the kernel to run in home
address space while implicitly accessing user space with the sie
instruction.

So we end up with:

User space:
- runs in primary or access register mode
- cr1 contains the user asce
- cr7 contains the user asce
- cr13 contains the kernel asce

Kernel space:
- runs in home space mode
- cr1 contains the user or kernel asce
  -> the kernel asce is loaded when a uaccess requires primary or
     secondary address mode
- cr7 contains the user or kernel asce, (changed with set_fs())
- cr13 contains the kernel asce

In case of uaccess the kernel changes to:
- primary space mode in case of a uaccess (copy_to_user) and uses
  e.g. the mvcp instruction to access user space. However the kernel
  will stay in home space mode if the mvcos instruction is available
- secondary space mode in case of futex atomic operations, so that the
  instructions come from primary address space and data from secondary
  space

In case of kvm the kernel runs in home space mode, but cr1 gets switched
to contain the gmap asce before the sie instruction gets executed. When
the sie instruction is finished cr1 will be switched back to contain the
user asce.

A context switch between two processes will always load the kernel asce
for the next process in cr1. So the first exit to user space is a bit
more expensive (one extra load control register instruction) than before,
however keeps the code rather simple.

In sum this means there is no need to perform any error prone page table
walks anymore when accessing user space.

The patch seems to be rather large, however it mainly removes the
the page table walk code and restores the previously deleted "standard"
uaccess code, with a couple of changes.

The uaccess without mvcos mode can be enforced with the "uaccess_primary"
kernel parameter.

Reported-by: Christian Borntraeger 
Signed-off-by: Heiko Carstens 
Signed-off-by: Martin Schwidefsky

s390/bitops: fix comment

2014-02-21T07:50:20+00:00

Fix some numbers in the comments describing the layout of the bit maps.

Signed-off-by: Heiko Carstens 
Signed-off-by: Martin Schwidefsky

s390/uaccess: introduce 'uaccesspt' kernel parameter

2014-02-21T07:50:17+00:00

The uaccesspt kernel parameter allows to enforce using the uaccess page
table walk variant. This is mainly for debugging purposes, so this mode
can also be enabled on machines which support the mvcos instruction.

Signed-off-by: Heiko Carstens 
Signed-off-by: Martin Schwidefsky

s390/setup: get rid of MACHINE_HAS_MVCOS machine flag

2014-02-21T07:50:15+00:00

MACHINE_HAS_MVCOS is used exactly once when the machine is brought up.
There is no need to cache the flag in the machine_flags.

Signed-off-by: Heiko Carstens 
Signed-off-by: Martin Schwidefsky

s390/uaccess: consistent types

2014-02-21T07:50:15+00:00

The types 'size_t' and 'unsigned long' have been used randomly for the
uaccess functions. This looks rather confusing.
So let's change all functions to use unsigned long instead and get rid
of size_t in order to have a consistent interface.

The only exception is strncpy_from_user() which uses 'long' since it
may return a signed value (-EFAULT).

Signed-off-by: Heiko Carstens 
Signed-off-by: Martin Schwidefsky

s390/uaccess: get rid of indirect function calls

2014-02-21T07:50:14+00:00

There are only two uaccess variants on s390 left: the version that is used
if the mvcos instruction is available, and the page table walk variant.
So there is no need for expensive indirect function calls.

By default the mvcos variant will be called. If the mvcos instruction is not
available it will call the page table walk variant.

For minimal performance impact the "if (mvcos_is_available)" is implemented
with a jump label, which will be a six byte nop on machines with mvcos.

Signed-off-by: Heiko Carstens 
Signed-off-by: Martin Schwidefsky

s390/uaccess: normalize order of parameters of indirect uaccess function calls

2014-02-21T07:50:13+00:00

For some unknown reason the indirect uaccess functions on s390 implement a
different parameter order than what is usual.

e.g.:

unsigned long copy_to_user(void *to, const void *from, unsigned long n);
vs.
size_t (*copy_to_user)(size_t n, void __user * to, const void *from);

Let's get rid of this confusing parameter reordering.

Signed-off-by: Heiko Carstens 
Signed-off-by: Martin Schwidefsky

s390/uaccess: remove dead extern declarations, make functions static

2014-01-22T13:02:17+00:00

Remove some dead uaccess extern declarations and also make some functions
static, since they are only used locally.

Signed-off-by: Heiko Carstens 
Signed-off-by: Martin Schwidefsky

s390/uaccess: test if current->mm is set before walking page tables

2014-01-22T13:02:16+00:00

If get_fs() == USER_DS we better test if current->mm is not zero before
walking page tables.
The page table walk code would try to lock mm->page_table_lock, however
if mm is zero this might crash.

Now it is arguably incorrect trying to access userspace if current->mm
is zero, however we have seen that and s390 would be the only architecture
which would crash in such a case.
So we better make the page table walk code a bit more robust and report
always a fault instead.

Signed-off-by: Heiko Carstens 
Signed-off-by: Martin Schwidefsky