linux-stable.git/mm, branch v5.4.129

mm, futex: fix shared futex pgoff on shmem huge page

2021-06-30T12:47:55+00:00

[ Upstream commit fe19bd3dae3d15d2fbfdb3de8839a6ea0fe94264 ]

If more than one futex is placed on a shmem huge page, it can happen
that waking the second wakes the first instead, and leaves the second
waiting: the key's shared.pgoff is wrong.

When 3.11 commit 13d60f4b6ab5 ("futex: Take hugepages into account when
generating futex_key"), the only shared huge pages came from hugetlbfs,
and the code added to deal with its exceptional page->index was put into
hugetlb source.  Then that was missed when 4.8 added shmem huge pages.

page_to_pgoff() is what others use for this nowadays: except that, as
currently written, it gives the right answer on hugetlbfs head, but
nonsense on hugetlbfs tails.  Fix that by calling hugetlbfs-specific
hugetlb_basepage_index() on PageHuge tails as well as on head.

Yes, it's unconventional to declare hugetlb_basepage_index() there in
pagemap.h, rather than in hugetlb.h; but I do not expect anything but
page_to_pgoff() ever to need it.

[akpm@linux-foundation.org: give hugetlb_basepage_index() prototype the correct scope]

Link: https://lkml.kernel.org/r/b17d946b-d09-326e-b42a-52884c36df32@google.com
Fixes: 800d8c63b2e9 ("shmem: add huge pages support")
Reported-by: Neel Natu 
Signed-off-by: Hugh Dickins 
Reviewed-by: Matthew Wilcox (Oracle) 
Acked-by: Thomas Gleixner 
Cc: "Kirill A. Shutemov" 
Cc: Zhang Yi 
Cc: Mel Gorman 
Cc: Mike Kravetz 
Cc: Ingo Molnar 
Cc: Peter Zijlstra 
Cc: Darren Hart 
Cc: Davidlohr Bueso 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 

Note on stable backport: leave redundant #include 
in kernel/futex.c, to avoid conflict over the header files included.

Signed-off-by: Hugh Dickins 
Signed-off-by: Greg Kroah-Hartman

mm/thp: another PVMW_SYNC fix in page_vma_mapped_walk()

2021-06-30T12:47:55+00:00

Aha! Shouldn't that quick scan over pte_none()s make sure that it holds
ptlock in the PVMW_SYNC case? That too might have been responsible for
BUGs or WARNs in split_huge_page_to_list() or its unmap_page(), though
I've never seen any.

Link: https://lkml.kernel.org/r/1bdf384c-8137-a149-2a1e-475a4791c3c@google.com
Link: https://lore.kernel.org/linux-mm/20210412180659.B9E3.409509F4@e16-tech.com/
Fixes: ace71a19cec5 ("mm: introduce page_vma_mapped_walk()")
Signed-off-by: Hugh Dickins 
Acked-by: Kirill A. Shutemov 
Tested-by: Wang Yugui 
Cc: Alistair Popple 
Cc: Matthew Wilcox 
Cc: Peter Xu 
Cc: Ralph Campbell 
Cc: Will Deacon 
Cc: Yang Shi 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Greg Kroah-Hartman

mm/thp: fix page_vma_mapped_walk() if THP mapped by ptes

2021-06-30T12:47:54+00:00

Running certain tests with a DEBUG_VM kernel would crash within hours,
on the total_mapcount BUG() in split_huge_page_to_list(), while trying
to free up some memory by punching a hole in a shmem huge page: split's
try_to_unmap() was unable to find all the mappings of the page (which,
on a !DEBUG_VM kernel, would then keep the huge page pinned in memory).

Crash dumps showed two tail pages of a shmem huge page remained mapped
by pte: ptes in a non-huge-aligned vma of a gVisor process, at the end
of a long unmapped range; and no page table had yet been allocated for
the head of the huge page to be mapped into.

Although designed to handle these odd misaligned huge-page-mapped-by-pte
cases, page_vma_mapped_walk() falls short by returning false prematurely
when !pmd_present or !pud_present or !p4d_present or !pgd_present: there
are cases when a huge page may span the boundary, with ptes present in
the next.

Restructure page_vma_mapped_walk() as a loop to continue in these cases,
while keeping its layout much as before.  Add a step_forward() helper to
advance pvmw->address across those boundaries: originally I tried to use
mm's standard p?d_addr_end() macros, but hit the same crash 512 times
less often: because of the way redundant levels are folded together, but
folded differently in different configurations, it was just too
difficult to use them correctly; and step_forward() is simpler anyway.

Link: https://lkml.kernel.org/r/fedb8632-1798-de42-f39e-873551d5bc81@google.com
Fixes: ace71a19cec5 ("mm: introduce page_vma_mapped_walk()")
Signed-off-by: Hugh Dickins 
Acked-by: Kirill A. Shutemov 
Cc: Alistair Popple 
Cc: Matthew Wilcox 
Cc: Peter Xu 
Cc: Ralph Campbell 
Cc: Wang Yugui 
Cc: Will Deacon 
Cc: Yang Shi 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Greg Kroah-Hartman

mm: page_vma_mapped_walk(): get vma_address_end() earlier

2021-06-30T12:47:54+00:00

page_vma_mapped_walk() cleanup: get THP's vma_address_end() at the
start, rather than later at next_pte.

It's a little unnecessary overhead on the first call, but makes for a
simpler loop in the following commit.

Link: https://lkml.kernel.org/r/4542b34d-862f-7cb4-bb22-e0df6ce830a2@google.com
Signed-off-by: Hugh Dickins 
Acked-by: Kirill A. Shutemov 
Cc: Alistair Popple 
Cc: Matthew Wilcox 
Cc: Peter Xu 
Cc: Ralph Campbell 
Cc: Wang Yugui 
Cc: Will Deacon 
Cc: Yang Shi 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Greg Kroah-Hartman

mm: page_vma_mapped_walk(): use goto instead of while (1)

2021-06-30T12:47:54+00:00

page_vma_mapped_walk() cleanup: add a label this_pte, matching next_pte,
and use "goto this_pte", in place of the "while (1)" loop at the end.

Link: https://lkml.kernel.org/r/a52b234a-851-3616-2525-f42736e8934@google.com
Signed-off-by: Hugh Dickins 
Acked-by: Kirill A. Shutemov 
Cc: Alistair Popple 
Cc: Matthew Wilcox 
Cc: Peter Xu 
Cc: Ralph Campbell 
Cc: Wang Yugui 
Cc: Will Deacon 
Cc: Yang Shi 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Greg Kroah-Hartman

mm: page_vma_mapped_walk(): add a level of indentation

2021-06-30T12:47:54+00:00

page_vma_mapped_walk() cleanup: add a level of indentation to much of
the body, making no functional change in this commit, but reducing the
later diff when this is all converted to a loop.

[hughd@google.com: : page_vma_mapped_walk(): add a level of indentation fix]
  Link: https://lkml.kernel.org/r/7f817555-3ce1-c785-e438-87d8efdcaf26@google.com

Link: https://lkml.kernel.org/r/efde211-f3e2-fe54-977-ef481419e7f3@google.com
Signed-off-by: Hugh Dickins 
Acked-by: Kirill A. Shutemov 
Cc: Alistair Popple 
Cc: Matthew Wilcox 
Cc: Peter Xu 
Cc: Ralph Campbell 
Cc: Wang Yugui 
Cc: Will Deacon 
Cc: Yang Shi 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Greg Kroah-Hartman

mm: page_vma_mapped_walk(): crossing page table boundary

2021-06-30T12:47:54+00:00

page_vma_mapped_walk() cleanup: adjust the test for crossing page table
boundary - I believe pvmw->address is always page-aligned, but nothing
else here assumed that; and remember to reset pvmw->pte to NULL after
unmapping the page table, though I never saw any bug from that.

Link: https://lkml.kernel.org/r/799b3f9c-2a9e-dfef-5d89-26e9f76fd97@google.com
Signed-off-by: Hugh Dickins 
Acked-by: Kirill A. Shutemov 
Cc: Alistair Popple 
Cc: Matthew Wilcox 
Cc: Peter Xu 
Cc: Ralph Campbell 
Cc: Wang Yugui 
Cc: Will Deacon 
Cc: Yang Shi 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Greg Kroah-Hartman

mm: page_vma_mapped_walk(): prettify PVMW_MIGRATION block

2021-06-30T12:47:54+00:00

page_vma_mapped_walk() cleanup: rearrange the !pmd_present() block to
follow the same "return not_found, return not_found, return true"
pattern as the block above it (note: returning not_found there is never
premature, since existence or prior existence of huge pmd guarantees
good alignment).

Link: https://lkml.kernel.org/r/378c8650-1488-2edf-9647-32a53cf2e21@google.com
Signed-off-by: Hugh Dickins 
Acked-by: Kirill A. Shutemov 
Reviewed-by: Peter Xu 
Cc: Alistair Popple 
Cc: Matthew Wilcox 
Cc: Ralph Campbell 
Cc: Wang Yugui 
Cc: Will Deacon 
Cc: Yang Shi 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Greg Kroah-Hartman

mm: page_vma_mapped_walk(): use pmde for *pvmw->pmd

2021-06-30T12:47:53+00:00

page_vma_mapped_walk() cleanup: re-evaluate pmde after taking lock, then
use it in subsequent tests, instead of repeatedly dereferencing pointer.

Link: https://lkml.kernel.org/r/53fbc9d-891e-46b2-cb4b-468c3b19238e@google.com
Signed-off-by: Hugh Dickins 
Acked-by: Kirill A. Shutemov 
Reviewed-by: Peter Xu 
Cc: Alistair Popple 
Cc: Matthew Wilcox 
Cc: Ralph Campbell 
Cc: Wang Yugui 
Cc: Will Deacon 
Cc: Yang Shi 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Greg Kroah-Hartman

mm: page_vma_mapped_walk(): settle PageHuge on entry

2021-06-30T12:47:53+00:00

page_vma_mapped_walk() cleanup: get the hugetlbfs PageHuge case out of
the way at the start, so no need to worry about it later.

Link: https://lkml.kernel.org/r/e31a483c-6d73-a6bb-26c5-43c3b880a2@google.com
Signed-off-by: Hugh Dickins 
Acked-by: Kirill A. Shutemov 
Reviewed-by: Peter Xu 
Cc: Alistair Popple 
Cc: "Kirill A. Shutemov" 
Cc: Matthew Wilcox 
Cc: Ralph Campbell 
Cc: Wang Yugui 
Cc: Will Deacon 
Cc: Yang Shi 
Cc: Zi Yan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Greg Kroah-Hartman