linux.git/mm/page_alloc.c, branch v4.20-rc2

memblock: stop using implicit alignment to SMP_CACHE_BYTES

2018-10-31T15:54:16+00:00

When a memblock allocation APIs are called with align = 0, the alignment
is implicitly set to SMP_CACHE_BYTES.

Implicit alignment is done deep in the memblock allocator and it can
come as a surprise.  Not that such an alignment would be wrong even
when used incorrectly but it is better to be explicit for the sake of
clarity and the prinicple of the least surprise.

Replace all such uses of memblock APIs with the 'align' parameter
explicitly set to SMP_CACHE_BYTES and stop implicit alignment assignment
in the memblock internal allocation functions.

For the case when memblock APIs are used via helper functions, e.g.  like
iommu_arena_new_node() in Alpha, the helper functions were detected with
Coccinelle's help and then manually examined and updated where
appropriate.

The direct memblock APIs users were updated using the semantic patch below:

@@
expression size, min_addr, max_addr, nid;
@@
(
|
- memblock_alloc_try_nid_raw(size, 0, min_addr, max_addr, nid)
+ memblock_alloc_try_nid_raw(size, SMP_CACHE_BYTES, min_addr, max_addr,
nid)
|
- memblock_alloc_try_nid_nopanic(size, 0, min_addr, max_addr, nid)
+ memblock_alloc_try_nid_nopanic(size, SMP_CACHE_BYTES, min_addr, max_addr,
nid)
|
- memblock_alloc_try_nid(size, 0, min_addr, max_addr, nid)
+ memblock_alloc_try_nid(size, SMP_CACHE_BYTES, min_addr, max_addr, nid)
|
- memblock_alloc(size, 0)
+ memblock_alloc(size, SMP_CACHE_BYTES)
|
- memblock_alloc_raw(size, 0)
+ memblock_alloc_raw(size, SMP_CACHE_BYTES)
|
- memblock_alloc_from(size, 0, min_addr)
+ memblock_alloc_from(size, SMP_CACHE_BYTES, min_addr)
|
- memblock_alloc_nopanic(size, 0)
+ memblock_alloc_nopanic(size, SMP_CACHE_BYTES)
|
- memblock_alloc_low(size, 0)
+ memblock_alloc_low(size, SMP_CACHE_BYTES)
|
- memblock_alloc_low_nopanic(size, 0)
+ memblock_alloc_low_nopanic(size, SMP_CACHE_BYTES)
|
- memblock_alloc_from_nopanic(size, 0, min_addr)
+ memblock_alloc_from_nopanic(size, SMP_CACHE_BYTES, min_addr)
|
- memblock_alloc_node(size, 0, nid)
+ memblock_alloc_node(size, SMP_CACHE_BYTES, nid)
)

[mhocko@suse.com: changelog update]
[akpm@linux-foundation.org: coding-style fixes]
[rppt@linux.ibm.com: fix missed uses of implicit alignment]
  Link: http://lkml.kernel.org/r/20181016133656.GA10925@rapoport-lnx
Link: http://lkml.kernel.org/r/1538687224-17535-1-git-send-email-rppt@linux.vnet.ibm.com
Signed-off-by: Mike Rapoport 
Suggested-by: Michal Hocko 
Acked-by: Paul Burton 	[MIPS]
Acked-by: Michael Ellerman 	[powerpc]
Acked-by: Michal Hocko 
Cc: Catalin Marinas 
Cc: Chris Zankel 
Cc: Geert Uytterhoeven 
Cc: Guan Xuetao 
Cc: Ingo Molnar 
Cc: Matt Turner 
Cc: Michal Simek 
Cc: Richard Weinberger 
Cc: Russell King 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: remove include/linux/bootmem.h

2018-10-31T15:54:16+00:00

Move remaining definitions and declarations from include/linux/bootmem.h
into include/linux/memblock.h and remove the redundant header.

The includes were replaced with the semantic patch below and then
semi-automated removal of duplicated '#include 

@@
@@
- #include 
+ #include 

[sfr@canb.auug.org.au: dma-direct: fix up for the removal of linux/bootmem.h]
  Link: http://lkml.kernel.org/r/20181002185342.133d1680@canb.auug.org.au
[sfr@canb.auug.org.au: powerpc: fix up for removal of linux/bootmem.h]
  Link: http://lkml.kernel.org/r/20181005161406.73ef8727@canb.auug.org.au
[sfr@canb.auug.org.au: x86/kaslr, ACPI/NUMA: fix for linux/bootmem.h removal]
  Link: http://lkml.kernel.org/r/20181008190341.5e396491@canb.auug.org.au
Link: http://lkml.kernel.org/r/1536927045-23536-30-git-send-email-rppt@linux.vnet.ibm.com
Signed-off-by: Mike Rapoport 
Signed-off-by: Stephen Rothwell 
Acked-by: Michal Hocko 
Cc: Catalin Marinas 
Cc: Chris Zankel 
Cc: "David S. Miller" 
Cc: Geert Uytterhoeven 
Cc: Greentime Hu 
Cc: Greg Kroah-Hartman 
Cc: Guan Xuetao 
Cc: Ingo Molnar 
Cc: "James E.J. Bottomley" 
Cc: Jonas Bonn 
Cc: Jonathan Corbet 
Cc: Ley Foon Tan 
Cc: Mark Salter 
Cc: Martin Schwidefsky 
Cc: Matt Turner 
Cc: Michael Ellerman 
Cc: Michal Simek 
Cc: Palmer Dabbelt 
Cc: Paul Burton 
Cc: Richard Kuo 
Cc: Richard Weinberger 
Cc: Rich Felker 
Cc: Russell King 
Cc: Serge Semin 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Cc: Vineet Gupta 
Cc: Yoshinori Sato 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memblock: rename __free_pages_bootmem to memblock_free_pages

2018-10-31T15:54:16+00:00

The conversion is done using

sed -i 's@__free_pages_bootmem@memblock_free_pages@' \
    $(git grep -l __free_pages_bootmem)

Link: http://lkml.kernel.org/r/1536927045-23536-27-git-send-email-rppt@linux.vnet.ibm.com
Signed-off-by: Mike Rapoport 
Acked-by: Michal Hocko 
Cc: Catalin Marinas 
Cc: Chris Zankel 
Cc: "David S. Miller" 
Cc: Geert Uytterhoeven 
Cc: Greentime Hu 
Cc: Greg Kroah-Hartman 
Cc: Guan Xuetao 
Cc: Ingo Molnar 
Cc: "James E.J. Bottomley" 
Cc: Jonas Bonn 
Cc: Jonathan Corbet 
Cc: Ley Foon Tan 
Cc: Mark Salter 
Cc: Martin Schwidefsky 
Cc: Matt Turner 
Cc: Michael Ellerman 
Cc: Michal Simek 
Cc: Palmer Dabbelt 
Cc: Paul Burton 
Cc: Richard Kuo 
Cc: Richard Weinberger 
Cc: Rich Felker 
Cc: Russell King 
Cc: Serge Semin 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Cc: Vineet Gupta 
Cc: Yoshinori Sato 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memblock: rename free_all_bootmem to memblock_free_all

2018-10-31T15:54:16+00:00

The conversion is done using

sed -i 's@free_all_bootmem@memblock_free_all@' \
    $(git grep -l free_all_bootmem)

Link: http://lkml.kernel.org/r/1536927045-23536-26-git-send-email-rppt@linux.vnet.ibm.com
Signed-off-by: Mike Rapoport 
Acked-by: Michal Hocko 
Cc: Catalin Marinas 
Cc: Chris Zankel 
Cc: "David S. Miller" 
Cc: Geert Uytterhoeven 
Cc: Greentime Hu 
Cc: Greg Kroah-Hartman 
Cc: Guan Xuetao 
Cc: Ingo Molnar 
Cc: "James E.J. Bottomley" 
Cc: Jonas Bonn 
Cc: Jonathan Corbet 
Cc: Ley Foon Tan 
Cc: Mark Salter 
Cc: Martin Schwidefsky 
Cc: Matt Turner 
Cc: Michael Ellerman 
Cc: Michal Simek 
Cc: Palmer Dabbelt 
Cc: Paul Burton 
Cc: Richard Kuo 
Cc: Richard Weinberger 
Cc: Rich Felker 
Cc: Russell King 
Cc: Serge Semin 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Cc: Vineet Gupta 
Cc: Yoshinori Sato 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memblock: remove _virt from APIs returning virtual address

2018-10-31T15:54:15+00:00

The conversion is done using

sed -i 's@memblock_virt_alloc@memblock_alloc@g' \
	$(git grep -l memblock_virt_alloc)

Link: http://lkml.kernel.org/r/1536927045-23536-8-git-send-email-rppt@linux.vnet.ibm.com
Signed-off-by: Mike Rapoport 
Cc: Catalin Marinas 
Cc: Chris Zankel 
Cc: "David S. Miller" 
Cc: Geert Uytterhoeven 
Cc: Greentime Hu 
Cc: Greg Kroah-Hartman 
Cc: Guan Xuetao 
Cc: Ingo Molnar 
Cc: "James E.J. Bottomley" 
Cc: Jonas Bonn 
Cc: Jonathan Corbet 
Cc: Ley Foon Tan 
Cc: Mark Salter 
Cc: Martin Schwidefsky 
Cc: Matt Turner 
Cc: Michael Ellerman 
Cc: Michal Hocko 
Cc: Michal Simek 
Cc: Palmer Dabbelt 
Cc: Paul Burton 
Cc: Richard Kuo 
Cc: Richard Weinberger 
Cc: Rich Felker 
Cc: Russell King 
Cc: Serge Semin 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Cc: Vineet Gupta 
Cc: Yoshinori Sato 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: remove CONFIG_HAVE_MEMBLOCK

2018-10-31T15:54:15+00:00

All architecures use memblock for early memory management. There is no need
for the CONFIG_HAVE_MEMBLOCK configuration option.

[rppt@linux.vnet.ibm.com: of/fdt: fixup #ifdefs]
  Link: http://lkml.kernel.org/r/20180919103457.GA20545@rapoport-lnx
[rppt@linux.vnet.ibm.com: csky: fixups after bootmem removal]
  Link: http://lkml.kernel.org/r/20180926112744.GC4628@rapoport-lnx
[rppt@linux.vnet.ibm.com: remove stale #else and the code it protects]
  Link: http://lkml.kernel.org/r/1538067825-24835-1-git-send-email-rppt@linux.vnet.ibm.com
Link: http://lkml.kernel.org/r/1536927045-23536-4-git-send-email-rppt@linux.vnet.ibm.com
Signed-off-by: Mike Rapoport 
Acked-by: Michal Hocko 
Tested-by: Jonathan Cameron 
Cc: Catalin Marinas 
Cc: Chris Zankel 
Cc: "David S. Miller" 
Cc: Geert Uytterhoeven 
Cc: Greentime Hu 
Cc: Greg Kroah-Hartman 
Cc: Guan Xuetao 
Cc: Ingo Molnar 
Cc: "James E.J. Bottomley" 
Cc: Jonas Bonn 
Cc: Jonathan Corbet 
Cc: Ley Foon Tan 
Cc: Mark Salter 
Cc: Martin Schwidefsky 
Cc: Matt Turner 
Cc: Michael Ellerman 
Cc: Michal Simek 
Cc: Palmer Dabbelt 
Cc: Paul Burton 
Cc: Richard Kuo 
Cc: Richard Weinberger 
Cc: Rich Felker 
Cc: Russell King 
Cc: Serge Semin 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Cc: Vineet Gupta 
Cc: Yoshinori Sato 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: return zero_resv_unavail optimization

2018-10-26T23:38:15+00:00

When checking for valid pfns in zero_resv_unavail(), it is not necessary
to verify that pfns within pageblock_nr_pages ranges are valid, only the
first one needs to be checked.  This is because memory for pages are
allocated in contiguous chunks that contain pageblock_nr_pages struct
pages.

Link: http://lkml.kernel.org/r/20181002143821.5112-3-msys.mizuma@gmail.com
Signed-off-by: Pavel Tatashin 
Signed-off-by: Masayoshi Mizuma 
Reviewed-by: Masayoshi Mizuma 
Acked-by: Naoya Horiguchi 
Reviewed-by: Oscar Salvador 
Cc: Ingo Molnar 
Cc: Michal Hocko 
Cc: Thomas Gleixner 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: zero remaining unavailable struct pages

2018-10-26T23:38:15+00:00

Patch series "mm: Fix for movable_node boot option", v3.

This patch series contains a fix for the movable_node boot option issue
which was introduced by commit 124049decbb1 ("x86/e820: put !E820_TYPE_RAM
regions into memblock.reserved").

The commit breaks the option because it changed the memory gap range to
reserved memblock.  So, the node is marked as Normal zone even if the SRAT
has Hot pluggable affinity.

First and second patch fix the original issue which the commit tried to
fix, then revert the commit.

This patch (of 3):

There is a kernel panic that is triggered when reading /proc/kpageflags on
the kernel booted with kernel parameter 'memmap=nn[KMG]!ss[KMG]':

  BUG: unable to handle kernel paging request at fffffffffffffffe
  PGD 9b20e067 P4D 9b20e067 PUD 9b210067 PMD 0
  Oops: 0000 [#1] SMP PTI
  CPU: 2 PID: 1728 Comm: page-types Not tainted 4.17.0-rc6-mm1-v4.17-rc6-180605-0816-00236-g2dfb086ef02c+ #160
  Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.0-2.fc28 04/01/2014
  RIP: 0010:stable_page_flags+0x27/0x3c0
  Code: 00 00 00 0f 1f 44 00 00 48 85 ff 0f 84 a0 03 00 00 41 54 55 49 89 fc 53 48 8b 57 08 48 8b 2f 48 8d 42 ff 83 e2 01 48 0f 44 c7 <48> 8b 00 f6 c4 01 0f 84 10 03 00 00 31 db 49 8b 54 24 08 4c 89 e7
  RSP: 0018:ffffbbd44111fde0 EFLAGS: 00010202
  RAX: fffffffffffffffe RBX: 00007fffffffeff9 RCX: 0000000000000000
  RDX: 0000000000000001 RSI: 0000000000000202 RDI: ffffed1182fff5c0
  RBP: ffffffffffffffff R08: 0000000000000001 R09: 0000000000000001
  R10: ffffbbd44111fed8 R11: 0000000000000000 R12: ffffed1182fff5c0
  R13: 00000000000bffd7 R14: 0000000002fff5c0 R15: ffffbbd44111ff10
  FS:  00007efc4335a500(0000) GS:ffff93a5bfc00000(0000) knlGS:0000000000000000
  CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  CR2: fffffffffffffffe CR3: 00000000b2a58000 CR4: 00000000001406e0
  Call Trace:
   kpageflags_read+0xc7/0x120
   proc_reg_read+0x3c/0x60
   __vfs_read+0x36/0x170
   vfs_read+0x89/0x130
   ksys_pread64+0x71/0x90
   do_syscall_64+0x5b/0x160
   entry_SYSCALL_64_after_hwframe+0x44/0xa9
  RIP: 0033:0x7efc42e75e23
  Code: 09 00 ba 9f 01 00 00 e8 ab 81 f4 ff 66 2e 0f 1f 84 00 00 00 00 00 90 83 3d 29 0a 2d 00 00 75 13 49 89 ca b8 11 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 34 c3 48 83 ec 08 e8 db d3 01 00 48 89 04 24

According to kernel bisection, this problem became visible due to commit
f7f99100d8d9 which changes how struct pages are initialized.

Memblock layout affects the pfn ranges covered by node/zone.  Consider
that we have a VM with 2 NUMA nodes and each node has 4GB memory, and the
default (no memmap= given) memblock layout is like below:

  MEMBLOCK configuration:
   memory size = 0x00000001fff75c00 reserved size = 0x000000000300c000
   memory.cnt  = 0x4
   memory[0x0]     [0x0000000000001000-0x000000000009efff], 0x000000000009e000 bytes on node 0 flags: 0x0
   memory[0x1]     [0x0000000000100000-0x00000000bffd6fff], 0x00000000bfed7000 bytes on node 0 flags: 0x0
   memory[0x2]     [0x0000000100000000-0x000000013fffffff], 0x0000000040000000 bytes on node 0 flags: 0x0
   memory[0x3]     [0x0000000140000000-0x000000023fffffff], 0x0000000100000000 bytes on node 1 flags: 0x0
   ...

If you give memmap=1G!4G (so it just covers memory[0x2]),
the range [0x100000000-0x13fffffff] is gone:

  MEMBLOCK configuration:
   memory size = 0x00000001bff75c00 reserved size = 0x000000000300c000
   memory.cnt  = 0x3
   memory[0x0]     [0x0000000000001000-0x000000000009efff], 0x000000000009e000 bytes on node 0 flags: 0x0
   memory[0x1]     [0x0000000000100000-0x00000000bffd6fff], 0x00000000bfed7000 bytes on node 0 flags: 0x0
   memory[0x2]     [0x0000000140000000-0x000000023fffffff], 0x0000000100000000 bytes on node 1 flags: 0x0
   ...

This causes shrinking node 0's pfn range because it is calculated by the
address range of memblock.memory.  So some of struct pages in the gap
range are left uninitialized.

We have a function zero_resv_unavail() which does zeroing the struct pages
outside memblock.memory, but currently it covers only the reserved
unavailable range (i.e.  memblock.memory && !memblock.reserved).  This
patch extends it to cover all unavailable range, which fixes the reported
issue.

Link: http://lkml.kernel.org/r/20181002143821.5112-2-msys.mizuma@gmail.com
Fixes: f7f99100d8d9 ("mm: stop zeroing memory during allocation in vmemmap")
Signed-off-by: Naoya Horiguchi 
Signed-off-by-by: Masayoshi Mizuma 
Tested-by: Oscar Salvador 
Tested-by: Masayoshi Mizuma 
Reviewed-by: Pavel Tatashin 
Cc: Ingo Molnar 
Cc: Michal Hocko 
Cc: Thomas Gleixner 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: move mirrored memory specific code outside of memmap_init_zone

2018-10-26T23:38:10+00:00

memmap_init_zone, is getting complex, because it is called from different
contexts: hotplug, and during boot, and also because it must handle some
architecture quirks.  One of them is mirrored memory.

Move the code that decides whether to skip mirrored memory outside of
memmap_init_zone, into a separate function.

[pasha.tatashin@oracle.com: uninline overlap_memmap_init()]
  Link: http://lkml.kernel.org/r/20180726193509.3326-4-pasha.tatashin@oracle.com
Link: http://lkml.kernel.org/r/20180724235520.10200-4-pasha.tatashin@oracle.com
Signed-off-by: Pavel Tatashin 
Reviewed-by: Oscar Salvador 
Cc: Pasha Tatashin 
Cc: Abdul Haleem 
Cc: Baoquan He 
Cc: Daniel Jordan 
Cc: Dan Williams 
Cc: Dave Hansen 
Cc: David Rientjes 
Cc: Greg Kroah-Hartman 
Cc: Ingo Molnar 
Cc: Jan Kara 
Cc: Jérôme Glisse 
Cc: Kirill A. Shutemov 
Cc: Michael Ellerman 
Cc: Michal Hocko 
Cc: Souptick Joarder 
Cc: Steven Sistare 
Cc: Vlastimil Babka 
Cc: Wei Yang 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: calculate deferred pages after skipping mirrored memory

2018-10-26T23:26:35+00:00

update_defer_init() should be called only when struct page is about to be
initialized. Because it counts number of initialized struct pages, but
there we may skip struct pages if there is some mirrored memory.

So move, update_defer_init() after checking for mirrored memory.

Also, rename update_defer_init() to defer_init() and reverse the return
boolean to emphasize that this is a boolean function, that tells that the
reset of memmap initialization should be deferred.

Make this function self-contained: do not pass number of already
initialized pages in this zone by using static counters.

I found this bug by reading the code.  The effect is that fewer than
expected struct pages are initialized early in boot, and it is possible
that in some corner cases we may fail to boot when mirrored pages are
used.  The deferred on demand code should somewhat mitigate this.  But
this still brings some inconsistencies compared to when booting without
mirrored pages, so it is better to fix.

[pasha.tatashin@oracle.com: add comment about defer_init's lack of locking]
  Link: http://lkml.kernel.org/r/20180726193509.3326-3-pasha.tatashin@oracle.com
[akpm@linux-foundation.org: make defer_init non-inline, __meminit]
Link: http://lkml.kernel.org/r/20180724235520.10200-3-pasha.tatashin@oracle.com
Signed-off-by: Pavel Tatashin 
Reviewed-by: Oscar Salvador 
Cc: Abdul Haleem 
Cc: Baoquan He 
Cc: Daniel Jordan 
Cc: Dan Williams 
Cc: Dave Hansen 
Cc: David Rientjes 
Cc: Greg Kroah-Hartman 
Cc: Ingo Molnar 
Cc: Jan Kara 
Cc: Jérôme Glisse 
Cc: Kirill A. Shutemov 
Cc: Michael Ellerman 
Cc: Michal Hocko 
Cc: Souptick Joarder 
Cc: Steven Sistare 
Cc: Vlastimil Babka 
Cc: Wei Yang 
Cc: Pasha Tatashin 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds