linux-stable.git/include/linux/memory.h, branch v5.3.17

mm/memory_hotplug: fix try_offline_node()

2019-11-20T15:49:15+00:00

commit 2c91f8fc6c999fe10185d8ad99fda1759f662f70 upstream.

try_offline_node() is pretty much broken right now:

 - The node span is updated when onlining memory, not when adding it. We
   ignore memory that was mever onlined. Bad.

 - We touch possible garbage memmaps. The pfn_to_nid(pfn) can easily
   trigger a kernel panic. Bad for memory that is offline but also bad
   for subsection hotadd with ZONE_DEVICE, whereby the memmap of the
   first PFN of a section might contain garbage.

 - Sections belonging to mixed nodes are not properly considered.

As memory blocks might belong to multiple nodes, we would have to walk
all pageblocks (or at least subsections) within present sections.
However, we don't have a way to identify whether a memmap that is not
online was initialized (relevant for ZONE_DEVICE).  This makes things
more complicated.

Luckily, we can piggy pack on the node span and the nid stored in memory
blocks.  Currently, the node span is grown when calling
move_pfn_range_to_zone() - e.g., when onlining memory, and shrunk when
removing memory, before calling try_offline_node().  Sysfs links are
created via link_mem_sections(), e.g., during boot or when adding
memory.

If the node still spans memory or if any memory block belongs to the
nid, we don't set the node offline.  As memory blocks that span multiple
nodes cannot get offlined, the nid stored in memory blocks is reliable
enough (for such online memory blocks, the node still spans the memory).

Introduce for_each_memory_block() to efficiently walk all memory blocks.

Note: We will soon stop shrinking the ZONE_DEVICE zone and the node span
when removing ZONE_DEVICE memory to fix similar issues (access of
garbage memmaps) - until we have a reliable way to identify whether
these memmaps were properly initialized.  This implies later, that once
a node had ZONE_DEVICE memory, we won't be able to set a node offline -
which should be acceptable.

Since commit f1dd2cd13c4b ("mm, memory_hotplug: do not associate
hotadded memory to zones until online") memory that is added is not
assoziated with a zone/node (memmap not initialized).  The introducing
commit 60a5a19e7419 ("memory-hotplug: remove sysfs file of node")
already missed that we could have multiple nodes for a section and that
the zone/node span is updated when onlining pages, not when adding them.

I tested this by hotplugging two DIMMs to a memory-less and cpu-less
NUMA node.  The node is properly onlined when adding the DIMMs.  When
removing the DIMMs, the node is properly offlined.

Masayoshi Mizuma reported:

: Without this patch, memory hotplug fails as panic:
:
:  BUG: kernel NULL pointer dereference, address: 0000000000000000
:  ...
:  Call Trace:
:   remove_memory_block_devices+0x81/0xc0
:   try_remove_memory+0xb4/0x130
:   __remove_memory+0xa/0x20
:   acpi_memory_device_remove+0x84/0x100
:   acpi_bus_trim+0x57/0x90
:   acpi_bus_trim+0x2e/0x90
:   acpi_device_hotplug+0x2b2/0x4d0
:   acpi_hotplug_work_fn+0x1a/0x30
:   process_one_work+0x171/0x380
:   worker_thread+0x49/0x3f0
:   kthread+0xf8/0x130
:   ret_from_fork+0x35/0x40

[david@redhat.com: v3]
  Link: http://lkml.kernel.org/r/20191102120221.7553-1-david@redhat.com
Link: http://lkml.kernel.org/r/20191028105458.28320-1-david@redhat.com
Fixes: 60a5a19e7419 ("memory-hotplug: remove sysfs file of node")
Fixes: f1dd2cd13c4b ("mm, memory_hotplug: do not associate hotadded memory to zones until online") # visiable after d0dc12e86b319
Signed-off-by: David Hildenbrand 
Tested-by: Masayoshi Mizuma 
Cc: Tang Chen 
Cc: Greg Kroah-Hartman 
Cc: "Rafael J. Wysocki" 
Cc: Keith Busch 
Cc: Jiri Olsa 
Cc: "Peter Zijlstra (Intel)" 
Cc: Jani Nikula 
Cc: Nayna Jain 
Cc: Michal Hocko 
Cc: Oscar Salvador 
Cc: Stephen Rothwell 
Cc: Dan Williams 
Cc: Pavel Tatashin 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Greg Kroah-Hartman

drivers/base/memory.c: get rid of find_memory_block_hinted()

2019-07-19T00:08:07+00:00

No longer needed, let's remove it.  Also, drop the "hint" parameter
completely from "find_memory_block_by_id", as nobody needs it anymore.

[david@redhat.com: v3]
  Link: http://lkml.kernel.org/r/20190620183139.4352-7-david@redhat.com
[david@redhat.com: handle zero-length walks]
  Link: http://lkml.kernel.org/r/1c2edc22-afd7-2211-c4c7-40e54e5007e8@redhat.com
Link: http://lkml.kernel.org/r/20190614100114.311-7-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Andrew Morton 
Tested-by: Qian Cai 
Cc: Greg Kroah-Hartman 
Cc: "Rafael J. Wysocki" 
Cc: David Hildenbrand 
Cc: Stephen Rothwell 
Cc: Pavel Tatashin 
Cc: Andrew Banman 
Cc: Mike Travis 
Cc: Oscar Salvador 
Cc: Michal Hocko 
Cc: Wei Yang 
Cc: Arun KS 
Cc: Stephen Rothwell 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/memory_hotplug: move and simplify walk_memory_blocks()

2019-07-19T00:08:06+00:00

Let's move walk_memory_blocks() to the place where memory block logic
resides and simplify it.  While at it, add a type for the callback
function.

Link: http://lkml.kernel.org/r/20190614100114.311-6-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Andrew Morton 
Cc: Greg Kroah-Hartman 
Cc: "Rafael J. Wysocki" 
Cc: David Hildenbrand 
Cc: Stephen Rothwell 
Cc: Pavel Tatashin 
Cc: Andrew Banman 
Cc: Mike Travis 
Cc: Oscar Salvador 
Cc: Michal Hocko 
Cc: Wei Yang 
Cc: Arun KS 
Cc: Qian Cai 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/memory_hotplug: remove memory block devices before arch_remove_memory()

2019-07-19T00:08:06+00:00

Let's factor out removing of memory block devices, which is only
necessary for memory added via add_memory() and friends that created
memory block devices.  Remove the devices before calling
arch_remove_memory().

This finishes factoring out memory block device handling from
arch_add_memory() and arch_remove_memory().

Link: http://lkml.kernel.org/r/20190527111152.16324-10-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Dan Williams 
Acked-by: Michal Hocko 
Cc: Greg Kroah-Hartman 
Cc: "Rafael J. Wysocki" 
Cc: David Hildenbrand 
Cc: "mike.travis@hpe.com" 
Cc: Andrew Banman 
Cc: Ingo Molnar 
Cc: Alex Deucher 
Cc: "David S. Miller" 
Cc: Mark Brown 
Cc: Chris Wilson 
Cc: Oscar Salvador 
Cc: Jonathan Cameron 
Cc: Arun KS 
Cc: Mathieu Malaterre 
Cc: Andy Lutomirski 
Cc: Anshuman Khandual 
Cc: Ard Biesheuvel 
Cc: Baoquan He 
Cc: Benjamin Herrenschmidt 
Cc: Borislav Petkov 
Cc: Catalin Marinas 
Cc: Chintan Pandya 
Cc: Christophe Leroy 
Cc: Dave Hansen 
Cc: Fenghua Yu 
Cc: Heiko Carstens 
Cc: "H. Peter Anvin" 
Cc: Joonsoo Kim 
Cc: Jun Yao 
Cc: "Kirill A. Shutemov" 
Cc: Logan Gunthorpe 
Cc: Mark Rutland 
Cc: Masahiro Yamada 
Cc: Michael Ellerman 
Cc: Mike Rapoport 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Paul Mackerras 
Cc: Pavel Tatashin 
Cc: Peter Zijlstra 
Cc: Qian Cai 
Cc: Rich Felker 
Cc: Rob Herring 
Cc: Robin Murphy 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Cc: Vasily Gorbik 
Cc: Wei Yang 
Cc: Will Deacon 
Cc: Yoshinori Sato 
Cc: Yu Zhao 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/memory_hotplug: create memory block devices after arch_add_memory()

2019-07-19T00:08:06+00:00

Only memory to be added to the buddy and to be onlined/offlined by user
space using /sys/devices/system/memory/...  needs (and should have!)
memory block devices.

Factor out creation of memory block devices.  Create all devices after
arch_add_memory() succeeded.  We can later drop the want_memblock
parameter, because it is now effectively stale.

Only after memory block devices have been added, memory can be onlined
by user space.  This implies, that memory is not visible to user space
at all before arch_add_memory() succeeded.

While at it
 - use WARN_ON_ONCE instead of BUG_ON in moved unregister_memory()
 - introduce find_memory_block_by_id() to search via block id
 - Use find_memory_block_by_id() in init_memory_block() to catch
   duplicates

Link: http://lkml.kernel.org/r/20190527111152.16324-8-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Pavel Tatashin 
Acked-by: Michal Hocko 
Cc: Greg Kroah-Hartman 
Cc: "Rafael J. Wysocki" 
Cc: David Hildenbrand 
Cc: "mike.travis@hpe.com" 
Cc: Ingo Molnar 
Cc: Andrew Banman 
Cc: Oscar Salvador 
Cc: Qian Cai 
Cc: Wei Yang 
Cc: Arun KS 
Cc: Mathieu Malaterre 
Cc: Alex Deucher 
Cc: Andy Lutomirski 
Cc: Anshuman Khandual 
Cc: Ard Biesheuvel 
Cc: Baoquan He 
Cc: Benjamin Herrenschmidt 
Cc: Borislav Petkov 
Cc: Catalin Marinas 
Cc: Chintan Pandya 
Cc: Christophe Leroy 
Cc: Chris Wilson 
Cc: Dan Williams 
Cc: Dave Hansen 
Cc: "David S. Miller" 
Cc: Fenghua Yu 
Cc: Heiko Carstens 
Cc: "H. Peter Anvin" 
Cc: Jonathan Cameron 
Cc: Joonsoo Kim 
Cc: Jun Yao 
Cc: "Kirill A. Shutemov" 
Cc: Logan Gunthorpe 
Cc: Mark Brown 
Cc: Mark Rutland 
Cc: Masahiro Yamada 
Cc: Michael Ellerman 
Cc: Mike Rapoport 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Cc: Rich Felker 
Cc: Rob Herring 
Cc: Robin Murphy 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Cc: Vasily Gorbik 
Cc: Will Deacon 
Cc: Yoshinori Sato 
Cc: Yu Zhao 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/memory_hotplug: allow arch_remove_memory() without CONFIG_MEMORY_HOTREMOVE

2019-07-19T00:08:06+00:00

We want to improve error handling while adding memory by allowing to use
arch_remove_memory() and __remove_pages() even if
CONFIG_MEMORY_HOTREMOVE is not set to e.g., implement something like:

	arch_add_memory()
	rc = do_something();
	if (rc) {
		arch_remove_memory();
	}

We won't get rid of CONFIG_MEMORY_HOTREMOVE for now, as it will require
quite some dependencies for memory offlining.

Link: http://lkml.kernel.org/r/20190527111152.16324-7-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Pavel Tatashin 
Cc: Tony Luck 
Cc: Fenghua Yu 
Cc: Benjamin Herrenschmidt 
Cc: Paul Mackerras 
Cc: Michael Ellerman 
Cc: Heiko Carstens 
Cc: Yoshinori Sato 
Cc: Rich Felker 
Cc: Dave Hansen 
Cc: Andy Lutomirski 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Borislav Petkov 
Cc: "H. Peter Anvin" 
Cc: Greg Kroah-Hartman 
Cc: "Rafael J. Wysocki" 
Cc: Michal Hocko 
Cc: David Hildenbrand 
Cc: Oscar Salvador 
Cc: "Kirill A. Shutemov" 
Cc: Alex Deucher 
Cc: "David S. Miller" 
Cc: Mark Brown 
Cc: Chris Wilson 
Cc: Christophe Leroy 
Cc: Nicholas Piggin 
Cc: Vasily Gorbik 
Cc: Rob Herring 
Cc: Masahiro Yamada 
Cc: "mike.travis@hpe.com" 
Cc: Andrew Banman 
Cc: Arun KS 
Cc: Qian Cai 
Cc: Mathieu Malaterre 
Cc: Baoquan He 
Cc: Logan Gunthorpe 
Cc: Anshuman Khandual 
Cc: Ard Biesheuvel 
Cc: Catalin Marinas 
Cc: Chintan Pandya 
Cc: Dan Williams 
Cc: Ingo Molnar 
Cc: Jonathan Cameron 
Cc: Joonsoo Kim 
Cc: Jun Yao 
Cc: Mark Rutland 
Cc: Mike Rapoport 
Cc: Oscar Salvador 
Cc: Robin Murphy 
Cc: Wei Yang 
Cc: Will Deacon 
Cc: Yu Zhao 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/memory_hotplug: make unregister_memory_section() never fail

2019-05-14T16:47:49+00:00

Failing while removing memory is mostly ignored and cannot really be
handled.  Let's treat errors in unregister_memory_section() in a nice way,
warning, but continuing.

Link: http://lkml.kernel.org/r/20190409100148.24703-3-david@redhat.com
Signed-off-by: David Hildenbrand 
Cc: Greg Kroah-Hartman 
Cc: "Rafael J. Wysocki" 
Cc: Ingo Molnar 
Cc: Andrew Banman 
Cc: Mike Travis 
Cc: David Hildenbrand 
Cc: Oscar Salvador 
Cc: Michal Hocko 
Cc: Pavel Tatashin 
Cc: Qian Cai 
Cc: Wei Yang 
Cc: Arun KS 
Cc: Mathieu Malaterre 
Cc: Andy Lutomirski 
Cc: Benjamin Herrenschmidt 
Cc: Borislav Petkov 
Cc: Christophe Leroy 
Cc: Dave Hansen 
Cc: Fenghua Yu 
Cc: Geert Uytterhoeven 
Cc: Heiko Carstens 
Cc: "H. Peter Anvin" 
Cc: Ingo Molnar 
Cc: Joonsoo Kim 
Cc: "Kirill A. Shutemov" 
Cc: Martin Schwidefsky 
Cc: Masahiro Yamada 
Cc: Michael Ellerman 
Cc: Mike Rapoport 
Cc: Nicholas Piggin 
Cc: Oscar Salvador 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Cc: Rich Felker 
Cc: Rob Herring 
Cc: Stefan Agner 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Cc: Vasily Gorbik 
Cc: Yoshinori Sato 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

x86/platform/UV: Add adjustable set memory block size function

2018-06-21T14:14:45+00:00

Add a new function to "adjust" the current fixed UV memory block size
of 2GB so it can be changed to a different physical boundary.  This is
out of necessity so arch dependent code can accommodate specific BIOS
requirements which can align these new PMEM modules at less than the
default boundaries.

A "set order" type of function was used to insure that the memory block
size will be a power of two value without requiring a validity check.
64GB was chosen as the upper limit for memory block size values to
accommodate upcoming 4PB systems which have 6 more bits of physical
address space (46 becoming 52).

Signed-off-by: Mike Travis 
Reviewed-by: Andrew Banman 
Cc: Andrew Morton 
Cc: Dimitri Sivanich 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Russ Anderson 
Cc: Thomas Gleixner 
Cc: dan.j.williams@intel.com
Cc: jgross@suse.com
Cc: kirill.shutemov@linux.intel.com
Cc: mhocko@suse.com
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/lkml/20180524201711.609546602@stormcage.americas.sgi.com
Signed-off-by: Ingo Molnar

mm/memory_hotplug: optimize memory hotplug

2018-04-06T04:36:25+00:00

During memory hotplugging we traverse struct pages three times:

1. memset(0) in sparse_add_one_section()
2. loop in __add_section() to set do: set_page_node(page, nid); and
   SetPageReserved(page);
3. loop in memmap_init_zone() to call __init_single_pfn()

This patch removes the first two loops, and leaves only loop 3.  All
struct pages are initialized in one place, the same as it is done during
boot.

The benefits:

 - We improve memory hotplug performance because we are not evicting the
   cache several times and also reduce loop branching overhead.

 - Remove condition from hotpath in __init_single_pfn(), that was added
   in order to fix the problem that was reported by Bharata in the above
   email thread, thus also improve performance during normal boot.

 - Make memory hotplug more similar to the boot memory initialization
   path because we zero and initialize struct pages only in one
   function.

 - Simplifies memory hotplug struct page initialization code, and thus
   enables future improvements, such as multi-threading the
   initialization of struct pages in order to improve hotplug
   performance even further on larger machines.

[pasha.tatashin@oracle.com: v5]
  Link: http://lkml.kernel.org/r/20180228030308.1116-7-pasha.tatashin@oracle.com
Link: http://lkml.kernel.org/r/20180215165920.8570-7-pasha.tatashin@oracle.com
Signed-off-by: Pavel Tatashin 
Reviewed-by: Ingo Molnar 
Cc: Michal Hocko 
Cc: Baoquan He 
Cc: Bharata B Rao 
Cc: Daniel Jordan 
Cc: Dan Williams 
Cc: Greg Kroah-Hartman 
Cc: "H. Peter Anvin" 
Cc: Kirill A. Shutemov 
Cc: Mel Gorman 
Cc: Steven Sistare 
Cc: Thomas Gleixner 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/memory_hotplug: don't read nid from struct page during hotplug

2018-04-06T04:36:25+00:00

During memory hotplugging the probe routine will leave struct pages
uninitialized, the same as it is currently done during boot.  Therefore,
we do not want to access the inside of struct pages before
__init_single_page() is called during onlining.

Because during hotplug we know that pages in one memory block belong to
the same numa node, we can skip the checking.  We should keep checking
for the boot case.

[pasha.tatashin@oracle.com: s/register_new_memory()/hotplug_memory_register()]
  Link: http://lkml.kernel.org/r/20180228030308.1116-6-pasha.tatashin@oracle.com
Link: http://lkml.kernel.org/r/20180215165920.8570-6-pasha.tatashin@oracle.com
Signed-off-by: Pavel Tatashin 
Acked-by: Michal Hocko 
Reviewed-by: Ingo Molnar 
Cc: Baoquan He 
Cc: Bharata B Rao 
Cc: Daniel Jordan 
Cc: Dan Williams 
Cc: Greg Kroah-Hartman 
Cc: "H. Peter Anvin" 
Cc: Kirill A. Shutemov 
Cc: Mel Gorman 
Cc: Steven Sistare 
Cc: Thomas Gleixner 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds