linux.git/mm/migrate.c, branch v6.17

mm/migrate: fix NULL movable_ops if CONFIG_ZSMALLOC=m

2025-08-19T23:35:57+00:00

After commit 84caf98838a3e5f4bdb34 ("mm: stop storing migration_ops in
page->mapping") we get such an error message if CONFIG_ZSMALLOC=m:

 WARNING: CPU: 3 PID: 42 at mm/migrate.c:142 isolate_movable_ops_page+0xa8/0x1c0
 CPU: 3 UID: 0 PID: 42 Comm: kcompactd0 Not tainted 6.16.0-rc5+ #2133 PREEMPT
 pc 9000000000540bd8 ra 9000000000540b84 tp 9000000100420000 sp 9000000100423a60
 a0 9000000100193a80 a1 000000000000000c a2 000000000000001b a3 ffffffffffffffff
 a4 ffffffffffffffff a5 0000000000000267 a6 0000000000000000 a7 9000000100423ae0
 t0 00000000000000f1 t1 00000000000000f6 t2 0000000000000000 t3 0000000000000001
 t4 ffffff00010eb834 t5 0000000000000040 t6 900000010c89d380 t7 90000000023fcc70
 t8 0000000000000018 u0 0000000000000000 s9 ffffff00010eb800 s0 ffffff00010eb800
 s1 000000000000000c s2 0000000000043ae0 s3 0000800000000000 s4 900000000219cc40
 s5 0000000000000000 s6 ffffff00010eb800 s7 0000000000000001 s8 90000000025b4000
    ra: 9000000000540b84 isolate_movable_ops_page+0x54/0x1c0
   ERA: 9000000000540bd8 isolate_movable_ops_page+0xa8/0x1c0
  CRMD: 000000b0 (PLV0 -IE -DA +PG DACF=CC DACM=CC -WE)
  PRMD: 00000004 (PPLV0 +PIE -PWE)
  EUEN: 00000000 (-FPE -SXE -ASXE -BTE)
  ECFG: 00071c1d (LIE=0,2-4,10-12 VS=7)
 ESTAT: 000c0000 [BRK] (IS= ECode=12 EsubCode=0)
  PRID: 0014c010 (Loongson-64bit, Loongson-3A5000)
 CPU: 3 UID: 0 PID: 42 Comm: kcompactd0 Not tainted 6.16.0-rc5+ #2133 PREEMPT
 Stack : 90000000021fd000 0000000000000000 9000000000247720 9000000100420000
         90000001004236a0 90000001004236a8 0000000000000000 90000001004237e8
         90000001004237e0 90000001004237e0 9000000100423550 0000000000000001
         0000000000000001 90000001004236a8 725a84864a19e2d9 90000000023fcc58
         9000000100420000 90000000024c6848 9000000002416848 0000000000000001
         0000000000000000 000000000000000a 0000000007fe0000 ffffff00010eb800
         0000000000000000 90000000021fd000 0000000000000000 900000000205cf30
         000000000000008e 0000000000000009 ffffff00010eb800 0000000000000001
         90000000025b4000 0000000000000000 900000000024773c 00007ffff103d748
         00000000000000b0 0000000000000004 0000000000000000 0000000000071c1d
         ...
 Call Trace:
 [<900000000024773c>] show_stack+0x5c/0x190
 [<90000000002415e0>] dump_stack_lvl+0x70/0x9c
 [<90000000004abe6c>] isolate_migratepages_block+0x3bc/0x16e0
 [<90000000004af408>] compact_zone+0x558/0x1000
 [<90000000004b0068>] compact_node+0xa8/0x1e0
 [<90000000004b0aa4>] kcompactd+0x394/0x410
 [<90000000002b3c98>] kthread+0x128/0x140
 [<9000000001779148>] ret_from_kernel_thread+0x28/0xc0
 [<9000000000245528>] ret_from_kernel_thread_asm+0x10/0x88

The reason is that defined(CONFIG_ZSMALLOC) evaluates to 1 only when
CONFIG_ZSMALLOC=y, we should use IS_ENABLED(CONFIG_ZSMALLOC) instead.  But
when I use IS_ENABLED(CONFIG_ZSMALLOC), page_movable_ops() cannot access
zsmalloc_mops because zsmalloc_mops is in a module.

To solve this problem, we define a set_movable_ops() interface to register
and unregister offline_movable_ops / zsmalloc_movable_ops in mm/migrate.c,
and call them at mm/balloon_compaction.c & mm/zsmalloc.c.  Since
offline_movable_ops / zsmalloc_movable_ops are always accessible, all
#ifdef / #endif are removed in page_movable_ops().

Link: https://lkml.kernel.org/r/20250817151759.2525174-1-chenhuacai@loongson.cn
Fixes: 84caf98838a3 ("mm: stop storing migration_ops in page->mapping")
Signed-off-by: Huacai Chen 
Acked-by: Zi Yan 
Acked-by: David Hildenbrand 
Cc: Huacai Chen 
Cc: Huacai Chen 
Cc: Lorenzo Stoakes 
Cc: "Michael S. Tsirkin" 
Cc: Minchan Kim 
Cc: Sergey Senozhatsky 
Signed-off-by: Andrew Morton

mm/page_owner: convert set_page_owner_migrate_reason() to folios

2025-07-20T01:59:57+00:00

Both callers of set_page_owner_migrate_reason() use folios.  Convert the
function to take a folio directly and move the &folio->page conversion
inside __set_page_owner_migrate_reason().

Link: https://lkml.kernel.org/r/20250711145910.90135-1-sidhartha.kumar@oracle.com
Signed-off-by: Sidhartha Kumar 
Reviewed-by: Matthew Wilcox (Oracle) 
Acked-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Vishal Moola (Oracle) 
Reviewed-by: Oscar Salvador 
Cc: Muchun Song 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton

mm/migrate: remove the -EEXIST conversion for move_pages()

2025-07-20T01:59:45+00:00

The -EEXIST conversion is introduced in commit 65462462ffb2 ("mm/gup:
follow_pfn_pte(): -EEXIST cleanup"), since follow_page() may call
follow_pfn_pte() which may return -EEXIST.

But after commit 7dff875c9436 ("mm/migrate: convert
add_page_for_migration() from follow_page() to folio_walk"), it use
folio_walk instead.  This limit the error code and won't return -EEXIST.

Remove the error code conversion here.

Link: https://lkml.kernel.org/r/20250707065711.18056-1-richard.weiyang@gmail.com
Signed-off-by: Wei Yang 
Acked-by: David Hildenbrand 
Reviewed-by: Joshua Hahn 
Reviewed-by: Zi Yan 
Cc: John Hubbard 
Cc: Matthew Brost 
Cc: Rakie Kim 
Cc: Byungchul Park 
Cc: Gregory Price 
Cc: Ying Huang 
Cc: Alistair Popple 
Signed-off-by: Andrew Morton

mm: rename PG_isolated to PG_movable_ops_isolated

2025-07-13T23:38:30+00:00

Let's rename the flag to make it clearer where it applies (not folios
...).

While at it, define the flag only with CONFIG_MIGRATION.

Link: https://lkml.kernel.org/r/20250704102524.326966-22-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Lorenzo Stoakes 
Reviewed-by: Harry Yoo 
Cc: Alistair Popple 
Cc: Al Viro 
Cc: Arnd Bergmann 
Cc: Brendan Jackman 
Cc: Byungchul Park 
Cc: Chengming Zhou 
Cc: Christian Brauner 
Cc: Christophe Leroy 
Cc: Eugenio Pé rez 
Cc: Greg Kroah-Hartman 
Cc: Gregory Price 
Cc: "Huang, Ying" 
Cc: Jan Kara 
Cc: Jason Gunthorpe 
Cc: Jason Wang 
Cc: Jerrin Shaji George 
Cc: Johannes Weiner 
Cc: John Hubbard 
Cc: Jonathan Corbet 
Cc: Joshua Hahn 
Cc: Liam Howlett 
Cc: Madhavan Srinivasan 
Cc: Mathew Brost 
Cc: Matthew Wilcox (Oracle) 
Cc: Miaohe Lin 
Cc: Michael Ellerman 
Cc: "Michael S. Tsirkin" 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Minchan Kim 
Cc: Naoya Horiguchi 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Peter Xu 
Cc: Qi Zheng 
Cc: Rakie Kim 
Cc: Rik van Riel 
Cc: Sergey Senozhatsky 
Cc: Shakeel Butt 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Xuan Zhuo 
Cc: xu xin 
Signed-off-by: Andrew Morton

mm: stop storing migration_ops in page->mapping

2025-07-13T23:38:29+00:00

...  instead, look them up statically based on the page type.  Maybe in
the future we want a registration interface?  At least for now, it can be
easily handled using the two page types that actually support page
migration.

The remaining usage of page->mapping is to flag such pages as actually
being movable (having movable_ops), which we will change next.

Link: https://lkml.kernel.org/r/20250704102524.326966-20-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Lorenzo Stoakes 
Reviewed-by: Harry Yoo 
Cc: Alistair Popple 
Cc: Al Viro 
Cc: Arnd Bergmann 
Cc: Brendan Jackman 
Cc: Byungchul Park 
Cc: Chengming Zhou 
Cc: Christian Brauner 
Cc: Christophe Leroy 
Cc: Eugenio Pé rez 
Cc: Greg Kroah-Hartman 
Cc: Gregory Price 
Cc: "Huang, Ying" 
Cc: Jan Kara 
Cc: Jason Gunthorpe 
Cc: Jason Wang 
Cc: Jerrin Shaji George 
Cc: Johannes Weiner 
Cc: John Hubbard 
Cc: Jonathan Corbet 
Cc: Joshua Hahn 
Cc: Liam Howlett 
Cc: Madhavan Srinivasan 
Cc: Mathew Brost 
Cc: Matthew Wilcox (Oracle) 
Cc: Miaohe Lin 
Cc: Michael Ellerman 
Cc: "Michael S. Tsirkin" 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Minchan Kim 
Cc: Naoya Horiguchi 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Peter Xu 
Cc: Qi Zheng 
Cc: Rakie Kim 
Cc: Rik van Riel 
Cc: Sergey Senozhatsky 
Cc: Shakeel Butt 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Xuan Zhuo 
Cc: xu xin 
Signed-off-by: Andrew Morton

mm: remove __folio_test_movable()

2025-07-13T23:38:29+00:00

Convert to page_has_movable_ops().  While at it, cleanup relevant code a
bit.

The data_race() in migrate_folio_unmap() is questionable: we already hold
a page reference, and concurrent modifications can no longer happen (iow:
__ClearPageMovable() no longer exists).  Drop it for now, we'll rework
page_has_movable_ops() soon either way to no longer rely on page->mapping.

Wherever we cast from folio to page now is a clear sign that this code has
to be decoupled.

Link: https://lkml.kernel.org/r/20250704102524.326966-19-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Lorenzo Stoakes 
Reviewed-by: Harry Yoo 
Cc: Alistair Popple 
Cc: Al Viro 
Cc: Arnd Bergmann 
Cc: Brendan Jackman 
Cc: Byungchul Park 
Cc: Chengming Zhou 
Cc: Christian Brauner 
Cc: Christophe Leroy 
Cc: Eugenio Pé rez 
Cc: Greg Kroah-Hartman 
Cc: Gregory Price 
Cc: "Huang, Ying" 
Cc: Jan Kara 
Cc: Jason Gunthorpe 
Cc: Jason Wang 
Cc: Jerrin Shaji George 
Cc: Johannes Weiner 
Cc: John Hubbard 
Cc: Jonathan Corbet 
Cc: Joshua Hahn 
Cc: Liam Howlett 
Cc: Madhavan Srinivasan 
Cc: Mathew Brost 
Cc: Matthew Wilcox (Oracle) 
Cc: Miaohe Lin 
Cc: Michael Ellerman 
Cc: "Michael S. Tsirkin" 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Minchan Kim 
Cc: Naoya Horiguchi 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Peter Xu 
Cc: Qi Zheng 
Cc: Rakie Kim 
Cc: Rik van Riel 
Cc: Sergey Senozhatsky 
Cc: Shakeel Butt 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Xuan Zhuo 
Cc: xu xin 
Signed-off-by: Andrew Morton

mm: rename __PageMovable() to page_has_movable_ops()

2025-07-13T23:38:29+00:00

Let's make it clearer that we are talking about movable_ops pages.

While at it, convert a VM_BUG_ON to a VM_WARN_ON_ONCE_PAGE.

Link: https://lkml.kernel.org/r/20250704102524.326966-17-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Lorenzo Stoakes 
Reviewed-by: Harry Yoo 
Cc: Alistair Popple 
Cc: Al Viro 
Cc: Arnd Bergmann 
Cc: Brendan Jackman 
Cc: Byungchul Park 
Cc: Chengming Zhou 
Cc: Christian Brauner 
Cc: Christophe Leroy 
Cc: Eugenio Pé rez 
Cc: Greg Kroah-Hartman 
Cc: Gregory Price 
Cc: "Huang, Ying" 
Cc: Jan Kara 
Cc: Jason Gunthorpe 
Cc: Jason Wang 
Cc: Jerrin Shaji George 
Cc: Johannes Weiner 
Cc: John Hubbard 
Cc: Jonathan Corbet 
Cc: Joshua Hahn 
Cc: Liam Howlett 
Cc: Madhavan Srinivasan 
Cc: Mathew Brost 
Cc: Matthew Wilcox (Oracle) 
Cc: Miaohe Lin 
Cc: Michael Ellerman 
Cc: "Michael S. Tsirkin" 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Minchan Kim 
Cc: Naoya Horiguchi 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Peter Xu 
Cc: Qi Zheng 
Cc: Rakie Kim 
Cc: Rik van Riel 
Cc: Sergey Senozhatsky 
Cc: Shakeel Butt 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Xuan Zhuo 
Cc: xu xin 
Signed-off-by: Andrew Morton

mm/migration: remove PageMovable()

2025-07-13T23:38:28+00:00

Previously, if __ClearPageMovable() were invoked on a page, this would
cause __PageMovable() to return false, but due to the continued existence
of page movable ops, PageMovable() would have returned true.

With __ClearPageMovable() gone, the two are exactly equivalent.

So we can replace PageMovable() checks by __PageMovable().  In fact,
__PageMovable() cannot change until a page is freed, so we can turn some
PageMovable() into sanity checks for __PageMovable().

Link: https://lkml.kernel.org/r/20250704102524.326966-16-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Lorenzo Stoakes 
Reviewed-by: Harry Yoo 
Cc: Alistair Popple 
Cc: Al Viro 
Cc: Arnd Bergmann 
Cc: Brendan Jackman 
Cc: Byungchul Park 
Cc: Chengming Zhou 
Cc: Christian Brauner 
Cc: Christophe Leroy 
Cc: Eugenio Pé rez 
Cc: Greg Kroah-Hartman 
Cc: Gregory Price 
Cc: "Huang, Ying" 
Cc: Jan Kara 
Cc: Jason Gunthorpe 
Cc: Jason Wang 
Cc: Jerrin Shaji George 
Cc: Johannes Weiner 
Cc: John Hubbard 
Cc: Jonathan Corbet 
Cc: Joshua Hahn 
Cc: Liam Howlett 
Cc: Madhavan Srinivasan 
Cc: Mathew Brost 
Cc: Matthew Wilcox (Oracle) 
Cc: Miaohe Lin 
Cc: Michael Ellerman 
Cc: "Michael S. Tsirkin" 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Minchan Kim 
Cc: Naoya Horiguchi 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Peter Xu 
Cc: Qi Zheng 
Cc: Rakie Kim 
Cc: Rik van Riel 
Cc: Sergey Senozhatsky 
Cc: Shakeel Butt 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Xuan Zhuo 
Cc: xu xin 
Signed-off-by: Andrew Morton

mm/migrate: move movable_ops page handling out of move_to_new_folio()

2025-07-13T23:38:27+00:00

Let's move that handling directly into migrate_folio_move(), so we can
simplify move_to_new_folio().  While at it, fixup the documentation a bit.

Note that unmap_and_move_huge_page() does not care, because it only deals
with actual folios.  (we only support migration of individual movable_ops
pages)

Link: https://lkml.kernel.org/r/20250704102524.326966-12-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Harry Yoo 
Reviewed-by: Lorenzo Stoakes 
Cc: Alistair Popple 
Cc: Al Viro 
Cc: Arnd Bergmann 
Cc: Brendan Jackman 
Cc: Byungchul Park 
Cc: Chengming Zhou 
Cc: Christian Brauner 
Cc: Christophe Leroy 
Cc: Eugenio Pé rez 
Cc: Greg Kroah-Hartman 
Cc: Gregory Price 
Cc: "Huang, Ying" 
Cc: Jan Kara 
Cc: Jason Gunthorpe 
Cc: Jason Wang 
Cc: Jerrin Shaji George 
Cc: Johannes Weiner 
Cc: John Hubbard 
Cc: Jonathan Corbet 
Cc: Joshua Hahn 
Cc: Liam Howlett 
Cc: Madhavan Srinivasan 
Cc: Mathew Brost 
Cc: Matthew Wilcox (Oracle) 
Cc: Miaohe Lin 
Cc: Michael Ellerman 
Cc: "Michael S. Tsirkin" 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Minchan Kim 
Cc: Naoya Horiguchi 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Peter Xu 
Cc: Qi Zheng 
Cc: Rakie Kim 
Cc: Rik van Riel 
Cc: Sergey Senozhatsky 
Cc: Shakeel Butt 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Xuan Zhuo 
Cc: xu xin 
Signed-off-by: Andrew Morton

mm/migrate: factor out movable_ops page handling into migrate_movable_ops_page()

2025-07-13T23:38:27+00:00

Let's factor it out, simplifying the calling code.

Before this change, we would have called flush_dcache_folio() also on
movable_ops pages.  As documented in Documentation/core-api/cachetlb.rst:

	"This routine need only be called for page cache pages which can
	 potentially ever be mapped into the address space of a user
	 process."

So don't do it for movable_ops pages.  If there would ever be such a
movable_ops page user, it should do the flushing itself after performing
the copy.

Note that we can now change folio_mapping_flags() to folio_test_anon() to
make it clearer, because movable_ops pages will never take that path.

[akpm@linux-foundation.org: fix kerneldoc]
Link: https://lkml.kernel.org/r/20250704102524.326966-10-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Zi Yan 
Reviewed-by: Lorenzo Stoakes 
Cc: Alistair Popple 
Cc: Al Viro 
Cc: Arnd Bergmann 
Cc: Brendan Jackman 
Cc: Byungchul Park 
Cc: Chengming Zhou 
Cc: Christian Brauner 
Cc: Christophe Leroy 
Cc: Eugenio Pé rez 
Cc: Greg Kroah-Hartman 
Cc: Gregory Price 
Cc: Harry Yoo 
Cc: "Huang, Ying" 
Cc: Jan Kara 
Cc: Jason Gunthorpe 
Cc: Jason Wang 
Cc: Jerrin Shaji George 
Cc: Johannes Weiner 
Cc: John Hubbard 
Cc: Jonathan Corbet 
Cc: Joshua Hahn 
Cc: Liam Howlett 
Cc: Madhavan Srinivasan 
Cc: Mathew Brost 
Cc: Matthew Wilcox (Oracle) 
Cc: Miaohe Lin 
Cc: Michael Ellerman 
Cc: "Michael S. Tsirkin" 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Minchan Kim 
Cc: Naoya Horiguchi 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Peter Xu 
Cc: Qi Zheng 
Cc: Rakie Kim 
Cc: Rik van Riel 
Cc: Sergey Senozhatsky 
Cc: Shakeel Butt 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Xuan Zhuo 
Cc: xu xin 
Signed-off-by: Andrew Morton