linux.git/mm/huge_memory.c, branch v5.13

mm: thp: replace DEBUG_VM BUG with VM_WARN when unmap fails for split

2021-06-16T16:24:42+00:00

When debugging the bug reported by Wang Yugui [1], try_to_unmap() may
fail, but the first VM_BUG_ON_PAGE() just checks page_mapcount() however
it may miss the failure when head page is unmapped but other subpage is
mapped.  Then the second DEBUG_VM BUG() that check total mapcount would
catch it.  This may incur some confusion.

As this is not a fatal issue, so consolidate the two DEBUG_VM checks
into one VM_WARN_ON_ONCE_PAGE().

[1] https://lore.kernel.org/linux-mm/20210412180659.B9E3.409509F4@e16-tech.com/

Link: https://lkml.kernel.org/r/d0f0db68-98b8-ebfb-16dc-f29df24cf012@google.com
Signed-off-by: Yang Shi 
Reviewed-by: Zi Yan 
Acked-by: Kirill A. Shutemov 
Signed-off-by: Hugh Dickins 
Cc: Alistair Popple 
Cc: Jan Kara 
Cc: Jue Wang 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Miaohe Lin 
Cc: Minchan Kim 
Cc: Naoya Horiguchi 
Cc: Oscar Salvador 
Cc: Peter Xu 
Cc: Ralph Campbell 
Cc: Shakeel Butt 
Cc: Wang Yugui 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/thp: try_to_unmap() use TTU_SYNC for safe splitting

2021-06-16T16:24:42+00:00

Stressing huge tmpfs often crashed on unmap_page()'s VM_BUG_ON_PAGE
(!unmap_success): with dump_page() showing mapcount:1, but then its raw
struct page output showing _mapcount ffffffff i.e.  mapcount 0.

And even if that particular VM_BUG_ON_PAGE(!unmap_success) is removed,
it is immediately followed by a VM_BUG_ON_PAGE(compound_mapcount(head)),
and further down an IS_ENABLED(CONFIG_DEBUG_VM) total_mapcount BUG():
all indicative of some mapcount difficulty in development here perhaps.
But the !CONFIG_DEBUG_VM path handles the failures correctly and
silently.

I believe the problem is that once a racing unmap has cleared pte or
pmd, try_to_unmap_one() may skip taking the page table lock, and emerge
from try_to_unmap() before the racing task has reached decrementing
mapcount.

Instead of abandoning the unsafe VM_BUG_ON_PAGE(), and the ones that
follow, use PVMW_SYNC in try_to_unmap_one() in this case: adding
TTU_SYNC to the options, and passing that from unmap_page().

When CONFIG_DEBUG_VM, or for non-debug too? Consensus is to do the same
for both: the slight overhead added should rarely matter, except perhaps
if splitting sparsely-populated multiply-mapped shmem.  Once confident
that bugs are fixed, TTU_SYNC here can be removed, and the race
tolerated.

Link: https://lkml.kernel.org/r/c1e95853-8bcd-d8fd-55fa-e7f2488e78f@google.com
Fixes: fec89c109f3a ("thp: rewrite freeze_page()/unfreeze_page() with generic rmap walkers")
Signed-off-by: Hugh Dickins 
Cc: Alistair Popple 
Cc: Jan Kara 
Cc: Jue Wang 
Cc: Kirill A. Shutemov 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Miaohe Lin 
Cc: Minchan Kim 
Cc: Naoya Horiguchi 
Cc: Oscar Salvador 
Cc: Peter Xu 
Cc: Ralph Campbell 
Cc: Shakeel Butt 
Cc: Wang Yugui 
Cc: Yang Shi 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/thp: make is_huge_zero_pmd() safe and quicker

2021-06-16T16:24:42+00:00

Most callers of is_huge_zero_pmd() supply a pmd already verified
present; but a few (notably zap_huge_pmd()) do not - it might be a pmd
migration entry, in which the pfn is encoded differently from a present
pmd: which might pass the is_huge_zero_pmd() test (though not on x86,
since L1TF forced us to protect against that); or perhaps even crash in
pmd_page() applied to a swap-like entry.

Make it safe by adding pmd_present() check into is_huge_zero_pmd()
itself; and make it quicker by saving huge_zero_pfn, so that
is_huge_zero_pmd() will not need to do that pmd_page() lookup each time.

__split_huge_pmd_locked() checked pmd_trans_huge() before: that worked,
but is unnecessary now that is_huge_zero_pmd() checks present.

Link: https://lkml.kernel.org/r/21ea9ca-a1f5-8b90-5e88-95fb1c49bbfa@google.com
Fixes: e71769ae5260 ("mm: enable thp migration for shmem thp")
Signed-off-by: Hugh Dickins 
Acked-by: Kirill A. Shutemov 
Reviewed-by: Yang Shi 
Cc: Alistair Popple 
Cc: Jan Kara 
Cc: Jue Wang 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Miaohe Lin 
Cc: Minchan Kim 
Cc: Naoya Horiguchi 
Cc: Oscar Salvador 
Cc: Peter Xu 
Cc: Ralph Campbell 
Cc: Shakeel Butt 
Cc: Wang Yugui 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/thp: fix __split_huge_pmd_locked() on shmem migration entry

2021-06-16T16:24:42+00:00

Patch series "mm/thp: fix THP splitting unmap BUGs and related", v10.

Here is v2 batch of long-standing THP bug fixes that I had not got
around to sending before, but prompted now by Wang Yugui's report
https://lore.kernel.org/linux-mm/20210412180659.B9E3.409509F4@e16-tech.com/

Wang Yugui has tested a rollup of these fixes applied to 5.10.39, and
they have done no harm, but have *not* fixed that issue: something more
is needed and I have no idea of what.

This patch (of 7):

Stressing huge tmpfs page migration racing hole punch often crashed on
the VM_BUG_ON(!pmd_present) in pmdp_huge_clear_flush(), with DEBUG_VM=y
kernel; or shortly afterwards, on a bad dereference in
__split_huge_pmd_locked() when DEBUG_VM=n.  They forgot to allow for pmd
migration entries in the non-anonymous case.

Full disclosure: those particular experiments were on a kernel with more
relaxed mmap_lock and i_mmap_rwsem locking, and were not repeated on the
vanilla kernel: it is conceivable that stricter locking happens to avoid
those cases, or makes them less likely; but __split_huge_pmd_locked()
already allowed for pmd migration entries when handling anonymous THPs,
so this commit brings the shmem and file THP handling into line.

And while there: use old_pmd rather than _pmd, as in the following
blocks; and make it clearer to the eye that the !vma_is_anonymous()
block is self-contained, making an early return after accounting for
unmapping.

Link: https://lkml.kernel.org/r/af88612-1473-2eaa-903-8d1a448b26@google.com
Link: https://lkml.kernel.org/r/dd221a99-efb3-cd1d-6256-7e646af29314@google.com
Fixes: e71769ae5260 ("mm: enable thp migration for shmem thp")
Signed-off-by: Hugh Dickins 
Cc: Kirill A. Shutemov 
Cc: Yang Shi 
Cc: Wang Yugui 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Naoya Horiguchi 
Cc: Alistair Popple 
Cc: Ralph Campbell 
Cc: Zi Yan 
Cc: Miaohe Lin 
Cc: Minchan Kim 
Cc: Jue Wang 
Cc: Peter Xu 
Cc: Jan Kara 
Cc: Shakeel Butt 
Cc: Oscar Salvador 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: fix typos in comments

2021-05-07T07:26:35+00:00

Fix ~94 single-word typos in locking code comments, plus a few
very obvious grammar mistakes.

Link: https://lkml.kernel.org/r/20210322212624.GA1963421@gmail.com
Link: https://lore.kernel.org/r/20210322205203.GB1959563@gmail.com
Signed-off-by: Ingo Molnar 
Reviewed-by: Matthew Wilcox (Oracle) 
Reviewed-by: Randy Dunlap 
Cc: Bhaskar Chowdhury 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: vmscan: consolidate shrinker_maps handling code

2021-05-05T18:27:23+00:00

The shrinker map management is not purely memcg specific, it is at the
intersection between memory cgroup and shrinkers.  It's allocation and
assignment of a structure, and the only memcg bit is the map is being
stored in a memcg structure.  So move the shrinker_maps handling code
into vmscan.c for tighter integration with shrinker code, and remove the
"memcg_" prefix.  There is no functional change.

Link: https://lkml.kernel.org/r/20210311190845.9708-3-shy828301@gmail.com
Signed-off-by: Yang Shi 
Acked-by: Vlastimil Babka 
Acked-by: Kirill Tkhai 
Acked-by: Roman Gushchin 
Reviewed-by: Shakeel Butt 
Cc: Dave Chinner 
Cc: Johannes Weiner 
Cc: Michal Hocko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: huge_memory: debugfs for file-backed THP split

2021-05-05T18:27:21+00:00

Further extend /split_huge_pages to accept
",," for file-backed THP split tests since
tmpfs may have file backed by THP that mapped nowhere.

Update selftest program to test file-backed THP split too.

Link: https://lkml.kernel.org/r/20210331235309.332292-2-zi.yan@sent.com
Signed-off-by: Zi Yan 
Suggested-by: Kirill A. Shutemov 
Reviewed-by: Yang Shi 
Cc: "Kirill A . Shutemov" 
Cc: Shuah Khan 
Cc: John Hubbard 
Cc: Sandipan Das 
Cc: David Hildenbrand 
Cc: Mika Penttila 
Cc: David Rientjes 
Cc: Matthew Wilcox 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: huge_memory: a new debugfs interface for splitting THP tests

2021-05-05T18:27:21+00:00

We did not have a direct user interface of splitting the compound page
backing a THP and there is no need unless we want to expose the THP
implementation details to users.  Make /split_huge_pages accept a
new command to do that.

By writing ",," to
/split_huge_pages, THPs within the given virtual address range
from the process with the given pid are split. It is used to test
split_huge_page function. In addition, a selftest program is added to
tools/testing/selftests/vm to utilize the interface by splitting
PMD THPs and PTE-mapped THPs.

This does not change the old behavior, i.e., writing 1 to the interface
to split all THPs in the system.

Link: https://lkml.kernel.org/r/20210331235309.332292-1-zi.yan@sent.com
Signed-off-by: Zi Yan 
Reviewed-by: Yang Shi 
Cc: David Hildenbrand 
Cc: David Rientjes 
Cc: John Hubbard 
Cc: "Kirill A . Shutemov" 
Cc: Matthew Wilcox 
Cc: Mika Penttila 
Cc: Sandipan Das 
Cc: Shuah Khan 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/huge_memory.c: use helper function migration_entry_to_page()

2021-05-05T18:27:21+00:00

It's more recommended to use helper function migration_entry_to_page()
to get the page via migration entry.  We can also enjoy the PageLocked()
check there.

Link: https://lkml.kernel.org/r/20210318122722.13135-7-linmiaohe@huawei.com
Signed-off-by: Miaohe Lin 
Reviewed-by: Peter Xu 
Cc: Aneesh Kumar K.V 
Cc: Matthew Wilcox 
Cc: Michel Lespinasse 
Cc: Ralph Campbell 
Cc: Thomas Hellstrm (Intel) 
Cc: Vlastimil Babka 
Cc: Wei Yang 
Cc: William Kucharski 
Cc: Yang Shi 
Cc: yuleixzhang 
Cc: Zi Yan 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/huge_memory.c: remove redundant PageCompound() check

2021-05-05T18:27:21+00:00

The !PageCompound() check limits the page must be head or tail while
!PageHead() further limits it to page head only.  So !PageHead() check is
equivalent here.

Link: https://lkml.kernel.org/r/20210318122722.13135-5-linmiaohe@huawei.com
Signed-off-by: Miaohe Lin 
Reviewed-by: Peter Xu 
Cc: Aneesh Kumar K.V 
Cc: Matthew Wilcox 
Cc: Michel Lespinasse 
Cc: Ralph Campbell 
Cc: Thomas Hellstrm (Intel) 
Cc: Vlastimil Babka 
Cc: Wei Yang 
Cc: William Kucharski 
Cc: Yang Shi 
Cc: yuleixzhang 
Cc: Zi Yan 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds