linux.git/mm/huge_memory.c, branch v6.9

userfaultfd: change src_folio after ensuring it's unpinned in UFFDIO_MOVE

2024-04-16T22:39:48+00:00

Commit d7a08838ab74 ("mm: userfaultfd: fix unexpected change to src_folio
when UFFDIO_MOVE fails") moved the src_folio->{mapping, index} changing to
after clearing the page-table and ensuring that it's not pinned.  This
avoids failure of swapout+migration and possibly memory corruption.

However, the commit missed fixing it in the huge-page case.

Link: https://lkml.kernel.org/r/20240404171726.2302435-1-lokeshgidra@google.com
Fixes: adef440691ba ("userfaultfd: UFFDIO_MOVE uABI")
Signed-off-by: Lokesh Gidra 
Acked-by: David Hildenbrand 
Cc: Andrea Arcangeli 
Cc: Kalesh Singh 
Cc: Lokesh Gidra 
Cc: Nicolas Geoffray 
Cc: Peter Xu 
Cc: Qi Zheng 
Cc: Matthew Wilcox 
Cc: 
Signed-off-by: Andrew Morton

mm/huge_memory: skip invalid debugfs new_order input for folio split

2024-03-12T20:07:17+00:00

User can put arbitrary new_order via debugfs for folio split test. 
Although new_order check is added to split_huge_page_to_list_order() in
the prior commit, these two additional checks can avoid unnecessary folio
locking and split_folio_to_order() calls.

Link: https://lkml.kernel.org/r/20240307181854.138928-2-zi.yan@sent.com
Signed-off-by: Zi Yan 
Reported-by: Dan Carpenter 
Closes: https://lore.kernel.org/linux-mm/7dda9283-b437-4cf8-ab0d-83c330deb9c0@moroto.mountain/
Cc: David Hildenbrand 
Cc: Kirill A. Shutemov 
Cc: Matthew Wilcox (Oracle) 
Cc: Ryan Roberts 
Cc: Yang Shi 
Cc: Yu Zhao 
Signed-off-by: Andrew Morton

mm/huge_memory: check new folio order when split a folio

2024-03-12T20:07:17+00:00

A folio can only be split into lower orders.

Since there are no new_order checks in debugfs, any new_order can be
passed via debugfs into split_huge_page_to_list_to_order().

Check new_order to make sure it is smaller than input folio order.

Link: https://lkml.kernel.org/r/20240307181854.138928-1-zi.yan@sent.com
Fixes: c010d47f107f ("mm: thp: split huge page to any lower order pages")
Signed-off-by: Zi Yan 
Reported-by: Dan Carpenter 
Closes: https://lore.kernel.org/linux-mm/7dda9283-b437-4cf8-ab0d-83c330deb9c0@moroto.mountain/
Cc: David Hildenbrand 
Cc: Kirill A. Shutemov 
Cc: Matthew Wilcox (Oracle) 
Cc: Ryan Roberts 
Cc: Yang Shi 
Cc: Yu Zhao 
Signed-off-by: Andrew Morton

mm: use folio more widely in __split_huge_page

2024-03-05T01:01:27+00:00

We already have a folio; use it instead of the head page where reasonable.
Saves a couple of calls to compound_head() and elimimnates a few
references to page->mapping.

Link: https://lkml.kernel.org/r/20240228164326.1355045-1-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) 
Reviewed-by: Sidhartha Kumar 
Signed-off-by: Andrew Morton

mm: huge_memory: enable debugfs to split huge pages to any order

2024-03-05T01:01:20+00:00

It is used to test split_huge_page_to_list_to_order for pagecache THPs. 
Also add test cases for split_huge_page_to_list_to_order via both debugfs.

[ziy@nvidia.com: fix issue discovered with NFS]
  Link: https://lkml.kernel.org/r/262E4DAA-4A78-4328-B745-1355AE356A07@nvidia.com
Link: https://lkml.kernel.org/r/20240226205534.1603748-9-zi.yan@sent.com
Signed-off-by: Zi Yan 
Tested-by: Aishwarya TCV 
Cc: David Hildenbrand 
Cc: Hugh Dickins 
Cc: Kirill A. Shutemov 
Cc: Luis Chamberlain 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Michal Koutny 
Cc: Roman Gushchin 
Cc: Ryan Roberts 
Cc: Yang Shi 
Cc: Yu Zhao 
Cc: Zach O'Keefe 
Cc: Aishwarya TCV 
Signed-off-by: Andrew Morton

mm: thp: split huge page to any lower order pages

2024-03-05T01:01:20+00:00

To split a THP to any lower order pages, we need to reform THPs on
subpages at given order and add page refcount based on the new page order.
Also we need to reinitialize page_deferred_list after removing the page
from the split_queue, otherwise a subsequent split will see list
corruption when checking the page_deferred_list again.

Note: Anonymous order-1 folio is not supported because _deferred_list,
which is used by partially mapped folios, is stored in subpage 2 and an
order-1 folio only has subpage 0 and 1.  File-backed order-1 folios are
fine, since they do not use _deferred_list.

[ziy@nvidia.com: fixup per discussion with Ryan]
  Link: https://lkml.kernel.org/r/494F48CD-1F0F-4CAD-884E-6D48F40AF990@nvidia.com
Link: https://lkml.kernel.org/r/20240226205534.1603748-8-zi.yan@sent.com
Signed-off-by: Zi Yan 
Cc: David Hildenbrand 
Cc: Hugh Dickins 
Cc: Kirill A. Shutemov 
Cc: Luis Chamberlain 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Michal Koutny 
Cc: Roman Gushchin 
Cc: Ryan Roberts 
Cc: Yang Shi 
Cc: Yu Zhao 
Cc: Zach O'Keefe 
Signed-off-by: Andrew Morton

mm: page_owner: add support for splitting to any order in split page_owner

2024-03-05T01:01:20+00:00

It adds a new_order parameter to set new page order in page owner.  It
prepares for upcoming changes to support split huge page to any lower
order.

Link: https://lkml.kernel.org/r/20240226205534.1603748-7-zi.yan@sent.com
Signed-off-by: Zi Yan 
Cc: David Hildenbrand 
Acked-by: David Hildenbrand 
Cc: Kirill A. Shutemov 
Cc: Luis Chamberlain 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Michal Koutny 
Cc: Roman Gushchin 
Cc: Ryan Roberts 
Cc: Yang Shi 
Cc: Yu Zhao 
Cc: Zach O'Keefe 
Signed-off-by: Andrew Morton

mm: memcg: make memcg huge page split support any order split

2024-03-05T01:01:20+00:00

It sets memcg information for the pages after the split.  A new parameter
new_order is added to tell the order of subpages in the new page, always 0
for now.  It prepares for upcoming changes to support split huge page to
any lower order.

Link: https://lkml.kernel.org/r/20240226205534.1603748-6-zi.yan@sent.com
Signed-off-by: Zi Yan 
Acked-by: David Hildenbrand 
Cc: Hugh Dickins 
Cc: Kirill A. Shutemov 
Cc: Luis Chamberlain 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Michal Koutny 
Cc: Roman Gushchin 
Cc: Ryan Roberts 
Cc: Yang Shi 
Cc: Yu Zhao 
Cc: Zach O'Keefe 
Signed-off-by: Andrew Morton

mm/page_owner: use order instead of nr in split_page_owner()

2024-03-05T01:01:19+00:00

We do not have non power of two pages, using nr is error prone if nr is
not power-of-two.  Use page order instead.

Link: https://lkml.kernel.org/r/20240226205534.1603748-5-zi.yan@sent.com
Signed-off-by: Zi Yan 
Acked-by: David Hildenbrand 
Cc: Hugh Dickins 
Cc: Kirill A. Shutemov 
Cc: Luis Chamberlain 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Michal Koutny 
Cc: Roman Gushchin 
Cc: Ryan Roberts 
Cc: Yang Shi 
Cc: Yu Zhao 
Cc: Zach O'Keefe 
Signed-off-by: Andrew Morton

mm/memcg: use order instead of nr in split_page_memcg()

2024-03-05T01:01:19+00:00

We do not have non power of two pages, using nr is error prone if nr is
not power-of-two.  Use page order instead.

Link: https://lkml.kernel.org/r/20240226205534.1603748-4-zi.yan@sent.com
Signed-off-by: Zi Yan 
Acked-by: David Hildenbrand 
Cc: Hugh Dickins 
Cc: Kirill A. Shutemov 
Cc: Luis Chamberlain 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Michal Koutny 
Cc: Roman Gushchin 
Cc: Ryan Roberts 
Cc: Yang Shi 
Cc: Yu Zhao 
Cc: Zach O'Keefe 
Signed-off-by: Andrew Morton