linux-stable.git/tools/testing/selftests/vm/userfaultfd.c, branch linux-5.15.y

kselftest/vm: fix tests build with old libc

2022-03-19T12:47:50+00:00

[ Upstream commit b773827e361952b3f53ac6fa4c4e39ccd632102e ]

The error message when I build vm tests on debian10 (GLIBC 2.28):

    userfaultfd.c: In function `userfaultfd_pagemap_test':
    userfaultfd.c:1393:37: error: `MADV_PAGEOUT' undeclared (first use
    in this function); did you mean `MADV_RANDOM'?
      if (madvise(area_dst, test_pgsize, MADV_PAGEOUT))
                                         ^~~~~~~~~~~~
                                         MADV_RANDOM

This patch includes these newer definitions from UAPI linux/mman.h, is
useful to fix tests build on systems without these definitions in glibc
sys/mman.h.

Link: https://lkml.kernel.org/r/20220227055330.43087-2-zhouchengming@bytedance.com
Signed-off-by: Chengming Zhou 
Reviewed-by: Shuah Khan 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Sasha Levin

userfaultfd/selftests: fix hugetlb area allocations

2022-01-11T14:35:18+00:00

[ Upstream commit f5c73297181c6b3ad76537bad98eaad6d29b9333 ]

Currently, userfaultfd selftest for hugetlb as run from run_vmtests.sh
or any environment where there are 'just enough' hugetlb pages will
always fail with:

  testing events (fork, remap, remove):
		ERROR: UFFDIO_COPY error: -12 (errno=12, line=616)

The ENOMEM error code implies there are not enough hugetlb pages.
However, there are free hugetlb pages but they are all reserved.  There
is a basic problem with the way the test allocates hugetlb pages which
has existed since the test was originally written.

Due to the way 'cleanup' was done between different phases of the test,
this issue was masked until recently.  The issue was uncovered by commit
8ba6e8640844 ("userfaultfd/selftests: reinitialize test context in each
test").

For the hugetlb test, src and dst areas are allocated as PRIVATE
mappings of a hugetlb file.  This means that at mmap time, pages are
reserved for the src and dst areas.  At the start of event testing (and
other tests) the src area is populated which results in allocation of
huge pages to fill the area and consumption of reserves associated with
the area.  Then, a child is forked to fault in the dst area.  Note that
the dst area was allocated in the parent and hence the parent owns the
reserves associated with the mapping.  The child has normal access to
the dst area, but can not use the reserves created/owned by the parent.
Thus, if there are no other huge pages available allocation of a page
for the dst by the child will fail.

Fix by not creating reserves for the dst area.  In this way the child
can use free (non-reserved) pages.

Also, MAP_PRIVATE of a file only makes sense if you are interested in
the contents of the file before making a COW copy.  The test does not do
this.  So, just use MAP_ANONYMOUS | MAP_HUGETLB to create an anonymous
hugetlb mapping.  There is no need to create a hugetlb file in the
non-shared case.

Link: https://lkml.kernel.org/r/20211217172919.7861-1-mike.kravetz@oracle.com
Signed-off-by: Mike Kravetz 
Cc: Axel Rasmussen 
Cc: Peter Xu 
Cc: Andrea Arcangeli 
Cc: Mina Almasry 
Cc: Shuah Khan 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Sasha Levin

mm/userfaultfd: selftests: fix memory corruption with thp enabled

2021-10-19T06:22:02+00:00

In RHEL's gating selftests we've encountered memory corruption in the
uffd event test even with upstream kernel:

        # ./userfaultfd anon 128 4
        nr_pages: 32768, nr_pages_per_cpu: 32768
        bounces: 3, mode: rnd racing read, userfaults: 6240 missing (6240) 14729 wp (14729)
        bounces: 2, mode: racing read, userfaults: 1444 missing (1444) 28877 wp (28877)
        bounces: 1, mode: rnd read, userfaults: 6055 missing (6055) 14699 wp (14699)
        bounces: 0, mode: read, userfaults: 82 missing (82) 25196 wp (25196)
        testing uffd-wp with pagemap (pgsize=4096): done
        testing uffd-wp with pagemap (pgsize=2097152): done
        testing events (fork, remap, remove): ERROR: nr 32427 memory corruption 0 1 (errno=0, line=963)
        ERROR: faulting process failed (errno=0, line=1117)

It can be easily reproduced when global thp enabled, which is the
default for RHEL.

It's also known as a side effect of commit 0db282ba2c12 ("selftest: use
mmap instead of posix_memalign to allocate memory", 2021-07-23), which
is imho right itself on using mmap() to make sure the addresses will be
untagged even on arm.

The problem is, for each test we allocate buffers using two
allocate_area() calls.  We assumed these two buffers won't affect each
other, however they could, because mmap() could have found that the two
buffers are near each other and having the same VMA flags, so they got
merged into one VMA.

It won't be a big problem if thp is not enabled, but when thp is
agressively enabled it means when initializing the src buffer it could
accidentally setup part of the dest buffer too when there's a shared THP
that overlaps the two regions.  Then some of the dest buffer won't be
able to be trapped by userfaultfd missing mode, then it'll cause memory
corruption as described.

To fix it, do release_pages() after initializing the src buffer.

Since the previous two release_pages() calls are after
uffd_test_ctx_clear() which will unmap all the buffers anyway (which is
stronger than release pages; as unmap() also tear town pgtables), drop
them as they shouldn't really be anything useful.

We can mark the Fixes tag upon 0db282ba2c12 as it's reported to only
happen there, however the real "Fixes" IMHO should be 8ba6e8640844, as
before that commit we'll always do explicit release_pages() before
registration of uffd, and 8ba6e8640844 changed that logic by adding
extra unmap/map and we didn't release the pages at the right place.
Meanwhile I don't have a solid glue anyway on whether posix_memalign()
could always avoid triggering this bug, hence it's safer to attach this
fix to commit 8ba6e8640844.

Link: https://lkml.kernel.org/r/20210923232512.210092-1-peterx@redhat.com
Fixes: 8ba6e8640844 ("userfaultfd/selftests: reinitialize test context in each test")
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1994931
Signed-off-by: Peter Xu 
Reported-by: Li Wang 
Tested-by: Li Wang 
Reviewed-by: Axel Rasmussen 
Cc: Andrea Arcangeli 
Cc: Nadav Amit 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

selftests/vm/userfaultfd: wake after copy failure

2021-09-03T16:58:16+00:00

When userfaultfd copy-ioctl fails since the PTE already exists, an -EEXIST
error is returned and the faulting thread is not woken.  The current
userfaultfd test does not wake the faulting thread in such case.  The
assumption is presumably that another thread set the PTE through copy/wp
ioctl and would wake the faulting thread or that alternatively the fault
handler would realize there is no need to "must_wait" and continue.  This
is not necessarily true.

There is an assumption that the "must_wait" tests in handle_userfault()
are sufficient to provide definitive answer whether the offending PTE is
populated or not.  However, userfaultfd_must_wait() test is lockless.
Consequently, concurrent calls to ptep_modify_prot_start(), for instance,
can clear the PTE and can cause userfaultfd_must_wait() to wrongly assume
it is not populated and a wait is needed.

There are therefore 3 options:
(1) Change the tests to wake on copy failure.
(2) Wake faulting thread unconditionally on zero/copy ioctls before
    returning -EEXIST.
(3) Change the userfaultfd_must_wait() to hold locks.

This patch took the first approach, but the others are valid solutions
with different tradeoffs.

Link: https://lkml.kernel.org/r/20210808020724.1022515-4-namit@vmware.com
Signed-off-by: Nadav Amit 
Cc: Jens Axboe 
Cc: Andrea Arcangeli 
Cc: Peter Xu 
Cc: Alexander Viro 
Cc: Axel Rasmussen 
Cc: Mike Rapoport 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

selftest: use mmap instead of posix_memalign to allocate memory

2021-07-24T00:43:28+00:00

This test passes pointers obtained from anon_allocate_area to the
userfaultfd and mremap APIs.  This causes a problem if the system
allocator returns tagged pointers because with the tagged address ABI
the kernel rejects tagged addresses passed to these APIs, which would
end up causing the test to fail.  To make this test compatible with such
system allocators, stop using the system allocator to allocate memory in
anon_allocate_area, and instead just use mmap.

Link: https://lkml.kernel.org/r/20210714195437.118982-3-pcc@google.com
Link: https://linux-review.googlesource.com/id/Icac91064fcd923f77a83e8e133f8631c5b8fc241
Fixes: c47174fc362a ("userfaultfd: selftest")
Co-developed-by: Lokesh Gidra 
Signed-off-by: Lokesh Gidra 
Signed-off-by: Peter Collingbourne 
Reviewed-by: Catalin Marinas 
Cc: Vincenzo Frascino 
Cc: Dave Martin 
Cc: Will Deacon 
Cc: Andrea Arcangeli 
Cc: Alistair Delva 
Cc: William McVicker 
Cc: Evgenii Stepanov 
Cc: Mitch Phillips 
Cc: Andrey Konovalov 
Cc: 	[5.4]
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

userfaultfd/selftests: exercise minor fault handling shmem support

2021-07-01T03:47:28+00:00

Enable test_uffdio_minor for test_type == TEST_SHMEM, and modify the test
slightly to pass in / check for the right feature flags.

Link: https://lkml.kernel.org/r/20210503180737.2487560-11-axelrasmussen@google.com
Signed-off-by: Axel Rasmussen 
Reviewed-by: Peter Xu 
Cc: Alexander Viro 
Cc: Andrea Arcangeli 
Cc: Brian Geffon 
Cc: "Dr . David Alan Gilbert" 
Cc: Hugh Dickins 
Cc: Jerome Glisse 
Cc: Joe Perches 
Cc: Kirill A. Shutemov 
Cc: Lokesh Gidra 
Cc: Mike Kravetz 
Cc: Mike Rapoport 
Cc: Mina Almasry 
Cc: Oliver Upton 
Cc: Shaohua Li 
Cc: Shuah Khan 
Cc: Stephen Rothwell 
Cc: Wang Qing 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

userfaultfd/selftests: reinitialize test context in each test

2021-07-01T03:47:28+00:00

Currently, the context (fds, mmap-ed areas, etc.) are global.  Each test
mutates this state in some way, in some cases really "clobbering it"
(e.g., the events test mremap-ing area_dst over the top of area_src, or
the minor faults tests overwriting the count_verify values in the test
areas).  We run the tests in a particular order, each test is careful to
make the right assumptions about its starting state, etc.

But, this is fragile.  It's better for a test's success or failure to not
depend on what some other prior test case did to the global state.

To that end, clear and reinitialize the test context at the start of each
test case, so whatever prior test cases did doesn't affect future tests.

This is particularly relevant to this series because the events test's
mremap of area_dst screws up assumptions the minor fault test was relying
on.  This wasn't a problem for hugetlb, as we don't mremap in that case.

[peterx@redhat.com: fix conflict between this patch and the uffd pagemap series]
  Link: https://lkml.kernel.org/r/YKQqKrl+/cQ1utrb@t490s

Link: https://lkml.kernel.org/r/20210503180737.2487560-10-axelrasmussen@google.com
Signed-off-by: Axel Rasmussen 
Signed-off-by: Peter Xu 
Reviewed-by: Peter Xu 
Cc: Alexander Viro 
Cc: Andrea Arcangeli 
Cc: Brian Geffon 
Cc: "Dr . David Alan Gilbert" 
Cc: Hugh Dickins 
Cc: Jerome Glisse 
Cc: Joe Perches 
Cc: Kirill A. Shutemov 
Cc: Lokesh Gidra 
Cc: Mike Kravetz 
Cc: Mike Rapoport 
Cc: Mina Almasry 
Cc: Oliver Upton 
Cc: Shaohua Li 
Cc: Shuah Khan 
Cc: Stephen Rothwell 
Cc: Wang Qing 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

userfaultfd/selftests: create alias mappings in the shmem test

2021-07-01T03:47:27+00:00

Previously, we just allocated two shm areas: area_src and area_dst.  With
this commit, change this so we also allocate area_src_alias, and
area_dst_alias.

area_*_alias and area_* (respectively) point to the same underlying
physical pages, but are different VMAs.  In a future commit in this
series, we'll leverage this setup to exercise minor fault handling support
for shmem, just like we do in the hugetlb_shared test.

Link: https://lkml.kernel.org/r/20210503180737.2487560-9-axelrasmussen@google.com
Signed-off-by: Axel Rasmussen 
Reviewed-by: Peter Xu 
Cc: Alexander Viro 
Cc: Andrea Arcangeli 
Cc: Brian Geffon 
Cc: "Dr . David Alan Gilbert" 
Cc: Hugh Dickins 
Cc: Jerome Glisse 
Cc: Joe Perches 
Cc: Kirill A. Shutemov 
Cc: Lokesh Gidra 
Cc: Mike Kravetz 
Cc: Mike Rapoport 
Cc: Mina Almasry 
Cc: Oliver Upton 
Cc: Shaohua Li 
Cc: Shuah Khan 
Cc: Stephen Rothwell 
Cc: Wang Qing 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

userfaultfd/selftests: use memfd_create for shmem test type

2021-07-01T03:47:27+00:00

This is a preparatory commit.  In the future, we want to be able to setup
alias mappings for area_src and area_dst in the shmem test, like we do in
the hugetlb_shared test.  With a VMA obtained via mmap(MAP_ANONYMOUS |
MAP_SHARED), it isn't clear how to do this.

So, mmap() with an fd, so we can create alias mappings.  Use memfd_create
instead of actually passing in a tmpfs path like hugetlb does, since it's
more convenient / simpler to run, and works just as well.

Future commits will:

1. Setup the alias mappings.
2. Extend our tests to actually take advantage of this, to test new
   userfaultfd behavior being introduced in this series.

Also, a small fix in the area we're changing: when the hugetlb setup fails
in main(), pass in the right argv[] so we actually print out the hugetlb
file path.

Link: https://lkml.kernel.org/r/20210503180737.2487560-8-axelrasmussen@google.com
Signed-off-by: Axel Rasmussen 
Reviewed-by: Peter Xu 
Cc: Alexander Viro 
Cc: Andrea Arcangeli 
Cc: Brian Geffon 
Cc: "Dr . David Alan Gilbert" 
Cc: Hugh Dickins 
Cc: Jerome Glisse 
Cc: Joe Perches 
Cc: Kirill A. Shutemov 
Cc: Lokesh Gidra 
Cc: Mike Kravetz 
Cc: Mike Rapoport 
Cc: Mina Almasry 
Cc: Oliver Upton 
Cc: Shaohua Li 
Cc: Shuah Khan 
Cc: Stephen Rothwell 
Cc: Wang Qing 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

userfaultfd/selftests: add pagemap uffd-wp test

2021-07-01T03:47:27+00:00

Add one anonymous specific test to start using pagemap.  With pagemap
support, we can directly read the uffd-wp bit from pgtable without
triggering any fault, so it's easier to do sanity checks in unit tests.

Meanwhile this test also leverages the newly introduced MADV_PAGEOUT
madvise function to test swap ptes with uffd-wp bit set, and across
fork()s.

Link: https://lkml.kernel.org/r/20210428225030.9708-7-peterx@redhat.com
Signed-off-by: Peter Xu 
Cc: Alexander Viro 
Cc: Andrea Arcangeli 
Cc: Axel Rasmussen 
Cc: Brian Geffon 
Cc: "Dr . David Alan Gilbert" 
Cc: Hugh Dickins 
Cc: Jerome Glisse 
Cc: Joe Perches 
Cc: Kirill A. Shutemov 
Cc: Lokesh Gidra 
Cc: Mike Kravetz 
Cc: Mike Rapoport 
Cc: Mina Almasry 
Cc: Oliver Upton 
Cc: Shaohua Li 
Cc: Shuah Khan 
Cc: Stephen Rothwell 
Cc: Wang Qing 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds