linux-stable.git/include/linux/highmem.h, branch v5.15.208

mm, hwpoison: try to recover from copy-on write faults

2023-07-05T17:25:04+00:00

commit a873dfe1032a132bf89f9e19a6ac44f5a0b78754 upstream.

Patch series "Copy-on-write poison recovery", v3.

Part 1 deals with the process that triggered the copy on write fault with
a store to a shared read-only page.  That process is send a SIGBUS with
the usual machine check decoration to specify the virtual address of the
lost page, together with the scope.

Part 2 sets up to asynchronously take the page with the uncorrected error
offline to prevent additional machine check faults.  H/t to Miaohe Lin
 and Shuai Xue  for
pointing me to the existing function to queue a call to memory_failure().

On x86 there is some duplicate reporting (because the error is also
signalled by the memory controller as well as by the core that triggered
the machine check).  Console logs look like this:

This patch (of 2):

If the kernel is copying a page as the result of a copy-on-write
fault and runs into an uncorrectable error, Linux will crash because
it does not have recovery code for this case where poison is consumed
by the kernel.

It is easy to set up a test case. Just inject an error into a private
page, fork(2), and have the child process write to the page.

I wrapped that neatly into a test at:

  git://git.kernel.org/pub/scm/linux/kernel/git/aegl/ras-tools.git

just enable ACPI error injection and run:

  # ./einj_mem-uc -f copy-on-write

Add a new copy_user_highpage_mc() function that uses copy_mc_to_kernel()
on architectures where that is available (currently x86 and powerpc).
When an error is detected during the page copy, return VM_FAULT_HWPOISON
to caller of wp_page_copy(). This propagates up the call stack. Both x86
and powerpc have code in their fault handler to deal with this code by
sending a SIGBUS to the application.

Note that this patch avoids a system crash and signals the process that
triggered the copy-on-write action. It does not take any action for the
memory error that is still in the shared page. To handle that a call to
memory_failure() is needed. But this cannot be done from wp_page_copy()
because it holds mmap_lock(). Perhaps the architecture fault handlers
can deal with this loose end in a subsequent patch?

On Intel/x86 this loose end will often be handled automatically because
the memory controller provides an additional notification of the h/w
poison in memory, the handler for this will call memory_failure(). This
isn't a 100% solution. If there are multiple errors, not all may be
logged in this way.

Cc: 
[tony.luck@intel.com: add call to kmsan_unpoison_memory(), per Miaohe Lin]
  Link: https://lkml.kernel.org/r/20221031201029.102123-2-tony.luck@intel.com
Link: https://lkml.kernel.org/r/20221021200120.175753-1-tony.luck@intel.com
Link: https://lkml.kernel.org/r/20221021200120.175753-2-tony.luck@intel.com
Signed-off-by: Tony Luck 
Reviewed-by: Dan Williams 
Reviewed-by: Naoya Horiguchi 
Reviewed-by: Miaohe Lin 
Reviewed-by: Alexander Potapenko 
Tested-by: Shuai Xue 
Cc: Christophe Leroy 
Cc: Matthew Wilcox (Oracle) 
Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Signed-off-by: Andrew Morton 
[ Due to missing commits
  c89357e27f20d ("mm: support GUP-triggered unsharing of anonymous pages")
  662ce1dc9caf4 ("delayacct: track delays from write-protect copy")
  b073d7f8aee4e ("mm: kmsan: maintain KMSAN metadata for page operations")
  The impact of c89357e27f20d is a name change from cow_user_page() to
  __wp_page_copy_user().
  The impact of 662ce1dc9caf4 is the introduction of a new feature of
  tracking write-protect copy in delayacct.
  The impact of b073d7f8aee4e is an introduction of KASAN feature.
  None of these commits establishes meaningful dependency, hence resolve by
  ignoring them. - jane]
Signed-off-by: Jane Chu 
Signed-off-by: Greg Kroah-Hartman

mm: remove flush_kernel_dcache_page

2021-09-03T16:58:13+00:00

flush_kernel_dcache_page is a rather confusing interface that implements a
subset of flush_dcache_page by not being able to properly handle page
cache mapped pages.

The only callers left are in the exec code as all other previous callers
were incorrect as they could have dealt with page cache pages.  Replace
the calls to flush_kernel_dcache_page with calls to flush_dcache_page,
which for all architectures does either exactly the same thing, can
contains one or more of the following:

 1) an optimization to defer the cache flush for page cache pages not
    mapped into userspace
 2) additional flushing for mapped page cache pages if cache aliases
    are possible

Link: https://lkml.kernel.org/r/20210712060928.4161649-7-hch@lst.de
Signed-off-by: Christoph Hellwig 
Acked-by: Linus Torvalds 
Reviewed-by: Ira Weiny 
Cc: Alex Shi 
Cc: Geoff Levand 
Cc: Greentime Hu 
Cc: Guo Ren 
Cc: Helge Deller 
Cc: "James E.J. Bottomley" 
Cc: Nick Hu 
Cc: Paul Cercueil 
Cc: Rich Felker 
Cc: Russell King 
Cc: Thomas Bogendoerfer 
Cc: Ulf Hansson 
Cc: Vincent Chen 
Cc: Yoshinori Sato 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: use kmap_local_page in memzero_page

2021-07-24T00:43:28+00:00

The commit message introducing the global memzero_page explicitly
mentions switching to kmap_local_page in the commit log but doesn't
actually do that.

Link: https://lkml.kernel.org/r/20210713055231.137602-3-hch@lst.de
Fixes: 28961998f858 ("iov_iter: lift memzero_page() to highmem.h")
Signed-off-by: Christoph Hellwig 
Reviewed-by: Chaitanya Kulkarni 
Reviewed-by: Ira Weiny 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: call flush_dcache_page() in memcpy_to_page() and memzero_page()

2021-07-24T00:43:28+00:00

memcpy_to_page and memzero_page can write to arbitrary pages, which
could be in the page cache or in high memory, so call
flush_kernel_dcache_pages to flush the dcache.

This is a problem when using these helpers on dcache challeneged
architectures.  Right now there are just a few users, chances are no one
used the PC floppy driver, the aha1542 driver for an ISA SCSI HBA, and a
few advanced and optional btrfs and ext4 features on those platforms yet
since the conversion.

Link: https://lkml.kernel.org/r/20210713055231.137602-2-hch@lst.de
Fixes: bb90d4bc7b6a ("mm/highmem: Lift memcpy_[to|from]_page to core")
Fixes: 28961998f858 ("iov_iter: lift memzero_page() to highmem.h")
Signed-off-by: Christoph Hellwig 
Reviewed-by: Ira Weiny 
Cc: Chaitanya Kulkarni 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

arm64: mte: handle tags zeroing at page allocation time

2021-06-04T18:32:21+00:00

Currently, on an anonymous page fault, the kernel allocates a zeroed
page and maps it in user space. If the mapping is tagged (PROT_MTE),
set_pte_at() additionally clears the tags. It is, however, more
efficient to clear the tags at the same time as zeroing the data on
allocation. To avoid clearing the tags on any page (which may not be
mapped as tagged), only do this if the vma flags contain VM_MTE. This
requires introducing a new GFP flag that is used to determine whether
to clear the tags.

The DC GZVA instruction with a 0 top byte (and 0 tag) requires
top-byte-ignore. Set the TCR_EL1.{TBI1,TBID1} bits irrespective of
whether KASAN_HW is enabled.

Signed-off-by: Peter Collingbourne 
Co-developed-by: Catalin Marinas 
Signed-off-by: Catalin Marinas 
Link: https://linux-review.googlesource.com/id/Id46dc94e30fe11474f7e54f5d65e7658dbdddb26
Reviewed-by: Catalin Marinas 
Reviewed-by: Andrey Konovalov 
Link: https://lore.kernel.org/r/20210602235230.3928842-4-pcc@google.com
Signed-off-by: Will Deacon

mm: arch: remove indirection level in alloc_zeroed_user_highpage_movable()

2021-06-04T18:32:21+00:00

In an upcoming change we would like to add a flag to
GFP_HIGHUSER_MOVABLE so that it would no longer be an OR
of GFP_HIGHUSER and __GFP_MOVABLE. This poses a problem for
alloc_zeroed_user_highpage_movable() which passes __GFP_MOVABLE
into an arch-specific __alloc_zeroed_user_highpage() hook which ORs
in GFP_HIGHUSER.

Since __alloc_zeroed_user_highpage() is only ever called from
alloc_zeroed_user_highpage_movable(), we can remove one level
of indirection here. Remove __alloc_zeroed_user_highpage(),
make alloc_zeroed_user_highpage_movable() the hook, and use
GFP_HIGHUSER_MOVABLE in the hook implementations so that they will
pick up the new flag that we are going to add.

Signed-off-by: Peter Collingbourne 
Link: https://linux-review.googlesource.com/id/Ic6361c657b2cdcd896adbe0cf7cb5a7fbb1ed7bf
Acked-by: Catalin Marinas 
Link: https://lore.kernel.org/r/20210602235230.3928842-2-pcc@google.com
Signed-off-by: Will Deacon

iov_iter: lift memzero_page() to highmem.h

2021-05-05T18:27:27+00:00

Patch series "btrfs: Convert kmap/memset/kunmap to memzero_user()".

Lifting memzero_user(), convert it to kmap_local_page() and then use it
in btrfs.

This patch (of 3):

memzero_page() can replace the kmap/memset/kunmap pattern in other
places in the code.  While zero_user() has the same interface it is not
the same call and its use should be limited and some of those calls may
be better converted from zero_user() to memzero_page().[1] But that is
not addressed in this series.

Lift memzero_page() to highmem.

[1] https://lore.kernel.org/lkml/CAHk-=wijdojzo56FzYqE5TOYw2Vws7ik3LEMGj9SPQaJJ+Z73Q@mail.gmail.com/

Link: https://lkml.kernel.org/r/20210309212137.2610186-1-ira.weiny@intel.com
Link: https://lkml.kernel.org/r/20210309212137.2610186-2-ira.weiny@intel.com
Signed-off-by: Ira Weiny 
Cc: Alexander Viro 
Cc: David Sterba 
Cc: Chris Mason 
Cc: Josef Bacik 
Cc: Chaitanya Kulkarni 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/highmem: Add VM_BUG_ON() to mem*_page() calls

2021-02-11T18:56:15+00:00

Add VM_BUG_ON bounds checks to ensure the newly lifted and created page
memory operations do not result in corrupted data in neighbor pages.[1][2]

[1] https://lore.kernel.org/lkml/20201210053502.GS1563847@iweiny-DESK2.sc.intel.com/
[2] https://lore.kernel.org/lkml/20210209110931.00f00e47d9a0529fcee2ff01@linux-foundation.org/

Suggested-by: Matthew Wilcox 
Suggested-by: Andrew Morton 
Reviewed-by: Christoph Hellwig 
Signed-off-by: Ira Weiny 
Signed-off-by: David Sterba

mm/highmem: Introduce memcpy_page(), memmove_page(), and memset_page()

2021-02-11T18:55:37+00:00

3 more common kmap patterns are kmap/memcpy/kunmap, kmap/memmove/kunmap.
and kmap/memset/kunmap.

Add helper functions for those patterns which use kmap_local_page().

Cc: Andrew Morton 
Cc: Christoph Hellwig 
Reviewed-by: Chaitanya Kulkarni 
Reviewed-by: Christoph Hellwig 
Signed-off-by: Ira Weiny 
Signed-off-by: David Sterba

mm/highmem: Convert memcpy_[to|from]_page() to kmap_local_page()

2021-02-11T18:55:16+00:00

kmap_local_page() is more efficient and is well suited for these calls.
Convert the kmap() to kmap_local_page()

Cc: Andrew Morton 
Cc: Christoph Hellwig 
Reviewed-by: Chaitanya Kulkarni 
Reviewed-by: Christoph Hellwig 
Signed-off-by: Ira Weiny 
Signed-off-by: David Sterba