linux-stable.git/mm/rmap.c, branch linux-6.3.y

mm/hwpoison: convert TTU_IGNORE_HWPOISON to TTU_HWPOISON

2023-02-28T01:00:14+00:00

After a memory error happens on a clean folio, a process unexpectedly
receives SIGBUS when it accesses the error page.  This SIGBUS killing is
pointless and simply degrades the level of RAS of the system, because the
clean folio can be dropped without any data lost on memory error handling
as we do for a clean pagecache.

When memory_failure() is called on a clean folio, try_to_unmap() is called
twice (one from split_huge_page() and one from hwpoison_user_mappings()). 
The root cause of the issue is that pte conversion to hwpoisoned entry is
now done in the first call of try_to_unmap() because PageHWPoison is
already set at this point, while it's actually expected to be done in the
second call.  This behavior disturbs the error handling operation like
removing pagecache, which results in the malfunction described above.

So convert TTU_IGNORE_HWPOISON into TTU_HWPOISON and set TTU_HWPOISON only
when we really intend to convert pte to hwpoison entry.  This can prevent
other callers of try_to_unmap() from accidentally converting to hwpoison
entries.

Link: https://lkml.kernel.org/r/20230221085905.1465385-1-naoya.horiguchi@linux.dev
Fixes: a42634a6c07d ("readahead: Use a folio in read_pages()")
Signed-off-by: Naoya Horiguchi 
Cc: David Hildenbrand 
Cc: Hugh Dickins 
Cc: Matthew Wilcox 
Cc: Miaohe Lin 
Cc: Minchan Kim 
Cc: Vlastimil Babka 
Cc: 
Signed-off-by: Andrew Morton

migrate_pages: batch flushing TLB

2023-02-17T04:43:54+00:00

The TLB flushing will cost quite some CPU cycles during the folio
migration in some situations.  For example, when migrate a folio of a
process with multiple active threads that run on multiple CPUs.  After
batching the _unmap and _move in migrate_pages(), the TLB flushing can be
batched easily with the existing TLB flush batching mechanism.  This patch
implements that.

We use the following test case to test the patch.

On a 2-socket Intel server,

- Run pmbench memory accessing benchmark

- Run `migratepages` to migrate pages of pmbench between node 0 and
  node 1 back and forth.

With the patch, the TLB flushing IPI reduces 99.1% during the test and the
number of pages migrated successfully per second increases 291.7%.

Haoxin helped to test the patchset on an ARM64 server with 128 cores, 2
NUMA nodes.  Test results show that the page migration performance
increases up to 78%.

NOTE: TLB flushing is batched only for normal folios, not for THP folios. 
Because the overhead of TLB flushing for THP folios is much lower than
that for normal folios (about 1/512 on x86 platform).

Link: https://lkml.kernel.org/r/20230213123444.155149-9-ying.huang@intel.com
Signed-off-by: "Huang, Ying" 
Tested-by: Xin Hao 
Reviewed-by: Zi Yan 
Reviewed-by: Xin Hao 
Cc: Yang Shi 
Cc: Baolin Wang 
Cc: Oscar Salvador 
Cc: Matthew Wilcox 
Cc: Bharata B Rao 
Cc: Alistair Popple 
Cc: Minchan Kim 
Cc: Mike Kravetz 
Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Signed-off-by: Andrew Morton

mm/hugetlb: convert hugetlb fault paths to use alloc_hugetlb_folio()

2023-02-13T23:54:29+00:00

Change alloc_huge_page() to alloc_hugetlb_folio() by changing all callers
to handle the now folio return type of the function.  In this conversion,
alloc_huge_page_vma() is also changed to alloc_hugetlb_folio_vma() and
hugepage_add_new_anon_rmap() is changed to take in a folio directly.  Many
additions of '&folio->page' are cleaned up in subsequent patches.

hugetlbfs_fallocate() is also refactored to use the RCU +
page_cache_next_miss() API.

Link: https://lkml.kernel.org/r/20230125170537.96973-5-sidhartha.kumar@oracle.com
Suggested-by: Mike Kravetz 
Reported-by: kernel test robot 
Signed-off-by: Sidhartha Kumar 
Cc: Gerald Schaefer 
Cc: John Hubbard 
Cc: Matthew Wilcox 
Cc: Muchun Song 
Signed-off-by: Andrew Morton

mm/mmap: remove __vma_adjust()

2023-02-10T00:51:38+00:00

Inline the work of __vma_adjust() into vma_merge().  This reduces code
size and has the added benefits of the comments for the cases being
located with the code.

Change the comments referencing vma_adjust() accordingly.

[Liam.Howlett@oracle.com: fix vma_merge() offset when expanding the next vma]
  Link: https://lkml.kernel.org/r/20230130195713.2881766-1-Liam.Howlett@oracle.com
Link: https://lkml.kernel.org/r/20230120162650.984577-49-Liam.Howlett@oracle.com
Signed-off-by: Liam R. Howlett 
Signed-off-by: Andrew Morton

rmap: add folio parameter to __page_set_anon_rmap()

2023-02-03T06:33:21+00:00

Avoid the compound_head() call in PageAnon() by passing in the folio that
all callers have.  Also save me from wondering whether page->mapping can
ever be overwritten on a tail page (I don't think it can, but I'm not 100%
sure).

Link: https://lkml.kernel.org/r/20230116192959.2147032-1-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) 
Reviewed-by: William Kucharski 
Signed-off-by: Andrew Morton

mm: remove munlock_vma_page()

2023-02-03T06:33:20+00:00

All callers now have a folio and can call munlock_vma_folio().  Update the
documentation to refer to munlock_vma_folio().

Link: https://lkml.kernel.org/r/20230116192827.2146732-4-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) 
Signed-off-by: Andrew Morton

mm: remove mlock_vma_page()

2023-02-03T06:33:20+00:00

All callers now have a folio and can call mlock_vma_folio().  Update the
documentation to refer to mlock_vma_folio().

Link: https://lkml.kernel.org/r/20230116192827.2146732-3-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) 
Signed-off-by: Andrew Morton

mm: remove __HAVE_ARCH_PTE_SWP_EXCLUSIVE

2023-02-03T06:33:11+00:00

__HAVE_ARCH_PTE_SWP_EXCLUSIVE is now supported by all architectures that
support swp PTEs, so let's drop it.

Link: https://lkml.kernel.org/r/20230113171026.582290-27-david@redhat.com
Signed-off-by: David Hildenbrand 
Signed-off-by: Andrew Morton

mm: mlock: update the interface to use folios

2023-02-03T06:33:04+00:00

Update the mlock interface to accept folios rather than pages, bringing
the interface in line with the internal implementation.

munlock_vma_page() still requires a page_folio() conversion, however this
is consistent with the existent mlock_vma_page() implementation and a
product of rmap still dealing in pages rather than folios.

Link: https://lkml.kernel.org/r/cba12777c5544305014bc0cbec56bb4cc71477d8.1673526881.git.lstoakes@gmail.com
Signed-off-by: Lorenzo Stoakes 
Acked-by: Vlastimil Babka 
Cc: Christian Brauner 
Cc: Geert Uytterhoeven 
Cc: Hugh Dickins 
Cc: Joel Fernandes (Google) 
Cc: Jonathan Corbet 
Cc: Liam R. Howlett 
Cc: Matthew Wilcox 
Cc: Mike Rapoport (IBM) 
Cc: William Kucharski 
Signed-off-by: Andrew Morton

mm: convert deferred_split_huge_page() to deferred_split_folio()

2023-02-03T06:33:00+00:00

Now that both callers use a folio, pass the folio in and save a call to
compound_head().

Link: https://lkml.kernel.org/r/20230111142915.1001531-28-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) 
Signed-off-by: Andrew Morton