linux.git/drivers/base, branch v5.7-rc4

Merge tag 'driver-core-5.7-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core

2020-04-26T18:04:15+00:00

Pull driver core fixes from Greg KH:
 "Here are some small firmware/driver core/debugfs fixes for 5.7-rc3.

  The debugfs change is now possible as now the last users of
  debugfs_create_u32() have been fixed up in the different trees that
  got merged into 5.7-rc1, and I don't want it creeping back in.

  The firmware changes did cause a regression in linux-next, so the
  final patch here reverts part of that, re-exporting the symbol to
  resolve that issue. All of these patches, with the exception of the
  final one, have been in linux-next with only that one reported issue"

* tag 'driver-core-5.7-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core:
  firmware_loader: revert removal of the fw_fallback_config export
  debugfs: remove return value of debugfs_create_u32()
  firmware_loader: remove unused exports
  firmware: imx: fix compile-testing

firmware_loader: revert removal of the fw_fallback_config export

2020-04-26T08:42:15+00:00

Christoph's patch removed two unsused exported symbols, however, one
symbol is used by the firmware_loader itself.  If CONFIG_FW_LOADER=m so
the firmware_loader is modular but CONFIG_FW_LOADER_USER_HELPER=y we fail
the build at mostpost.

ERROR: modpost: "fw_fallback_config" [drivers/base/firmware_loader/firmware_class.ko] undefined!

This happens because the variable fw_fallback_config is built into the
kernel if CONFIG_FW_LOADER_USER_HELPER=y always, so we need to grant
access to the firmware loader module by exporting it.

Revert only one hunk from his patch.

Fixes: 739604734bd8 ("firmware_loader: remove unused exports")
Reported-by: Randy Dunlap 
Signed-off-by: Luis Chamberlain 
Link: https://lore.kernel.org/r/20200424184916.22843-1-mcgrof@kernel.org
Signed-off-by: Greg Kroah-Hartman

PM: sleep: core: Switch back to async_schedule_dev()

2020-04-22T14:36:18+00:00

Commit 8b9ec6b73277 ("PM core: Use new async_schedule_dev command")
introduced a new function for better performance.

However commit f2a424f6c613 ("PM / core: Introduce dpm_async_fn()
helper") went back to the non-optimized version, async_schedule().

So switch back to the sync_schedule_dev() to improve performance

Fixes: f2a424f6c613 ("PM / core: Introduce dpm_async_fn() helper")
Signed-off-by: Kai-Heng Feng 
Signed-off-by: Rafael J. Wysocki

firmware_loader: remove unused exports

2020-04-17T07:57:44+00:00

Neither fw_fallback_config nor firmware_config_table are used by modules.

Signed-off-by: Christoph Hellwig 
Link: https://lore.kernel.org/r/20200417064146.1086644-3-hch@lst.de
Signed-off-by: Greg Kroah-Hartman

mm/memory_hotplug: allow to specify a default online_type

2020-04-07T17:43:41+00:00

For now, distributions implement advanced udev rules to essentially
- Don't online any hotplugged memory (s390x)
- Online all memory to ZONE_NORMAL (e.g., most virt environments like
  hyperv)
- Online all memory to ZONE_MOVABLE in case the zone imbalance is taken
  care of (e.g., bare metal, special virt environments)

In summary: All memory is usually onlined the same way, however, the
kernel always has to ask user space to come up with the same answer.
E.g., Hyper-V always waits for a memory block to get onlined before
continuing, otherwise it might end up adding memory faster than
onlining it, which can result in strange OOM situations.  This waiting
slows down adding of a bigger amount of memory.

Let's allow to specify a default online_type, not just "online" and
"offline".  This allows distributions to configure the default online_type
when booting up and be done with it.

We can now specify "offline", "online", "online_movable" and
"online_kernel" via
- "memhp_default_state=" on the kernel cmdline
- /sys/devices/system/memory/auto_online_blocks
just like we are able to specify for a single memory block via
/sys/devices/system/memory/memoryX/state

Signed-off-by: David Hildenbrand 
Signed-off-by: Andrew Morton 
Reviewed-by: Wei Yang 
Reviewed-by: Baoquan He 
Acked-by: Michal Hocko 
Acked-by: Pankaj Gupta 
Cc: Greg Kroah-Hartman 
Cc: Oscar Salvador 
Cc: "Rafael J. Wysocki" 
Cc: Wei Yang 
Cc: Benjamin Herrenschmidt 
Cc: Eduardo Habkost 
Cc: Haiyang Zhang 
Cc: Igor Mammedov 
Cc: "K. Y. Srinivasan" 
Cc: Michael Ellerman 
Cc: Paul Mackerras 
Cc: Stephen Hemminger 
Cc: Vitaly Kuznetsov 
Cc: Wei Liu 
Cc: Yumei Huang 
Link: http://lkml.kernel.org/r/20200317104942.11178-9-david@redhat.com
Signed-off-by: Linus Torvalds

mm/memory_hotplug: convert memhp_auto_online to store an online_type

2020-04-07T17:43:40+00:00

...  and rename it to memhp_default_online_type.  This is a preparation
for more detailed default online behavior.

Signed-off-by: David Hildenbrand 
Signed-off-by: Andrew Morton 
Reviewed-by: Wei Yang 
Reviewed-by: Baoquan He 
Acked-by: Michal Hocko 
Acked-by: Pankaj Gupta 
Cc: Greg Kroah-Hartman 
Cc: Oscar Salvador 
Cc: "Rafael J. Wysocki" 
Cc: Wei Yang 
Cc: Benjamin Herrenschmidt 
Cc: Eduardo Habkost 
Cc: Haiyang Zhang 
Cc: Igor Mammedov 
Cc: "K. Y. Srinivasan" 
Cc: Michael Ellerman 
Cc: Paul Mackerras 
Cc: Stephen Hemminger 
Cc: Vitaly Kuznetsov 
Cc: Wei Liu 
Cc: Yumei Huang 
Link: http://lkml.kernel.org/r/20200317104942.11178-8-david@redhat.com
Signed-off-by: Linus Torvalds

drivers/base/memory: store mapping between MMOP_* and string in an array

2020-04-07T17:43:40+00:00

Let's use a simple array which we can reuse soon.  While at it, move the
string->mmop conversion out of the device hotplug lock.

Signed-off-by: David Hildenbrand 
Signed-off-by: Andrew Morton 
Reviewed-by: Wei Yang 
Reviewed-by: Baoquan He 
Acked-by: Michal Hocko 
Acked-by: Pankaj Gupta 
Cc: Greg Kroah-Hartman 
Cc: Oscar Salvador 
Cc: "Rafael J. Wysocki" 
Cc: Wei Yang 
Cc: Benjamin Herrenschmidt 
Cc: Eduardo Habkost 
Cc: Haiyang Zhang 
Cc: Igor Mammedov 
Cc: "K. Y. Srinivasan" 
Cc: Michael Ellerman 
Cc: Paul Mackerras 
Cc: Stephen Hemminger 
Cc: Vitaly Kuznetsov 
Cc: Wei Liu 
Cc: Yumei Huang 
Link: http://lkml.kernel.org/r/20200317104942.11178-4-david@redhat.com
Signed-off-by: Linus Torvalds

drivers/base/memory: map MMOP_OFFLINE to 0

2020-04-07T17:43:40+00:00

Historically, we used the value -1.  Just treat 0 as the special case now.
Clarify a comment (which was wrong, when we come via device_online() the
first time, the online_type would have been 0 / MEM_ONLINE).  The default
is now always MMOP_OFFLINE.  This removes the last user of the manual
"-1", which didn't use the enum value.

This is a preparation to use the online_type as an array index.

Signed-off-by: David Hildenbrand 
Signed-off-by: Andrew Morton 
Reviewed-by: Wei Yang 
Reviewed-by: Baoquan He 
Acked-by: Michal Hocko 
Acked-by: Pankaj Gupta 
Cc: Greg Kroah-Hartman 
Cc: Oscar Salvador 
Cc: "Rafael J. Wysocki" 
Cc: Wei Yang 
Cc: Benjamin Herrenschmidt 
Cc: Eduardo Habkost 
Cc: Haiyang Zhang 
Cc: Igor Mammedov 
Cc: "K. Y. Srinivasan" 
Cc: Michael Ellerman 
Cc: Paul Mackerras 
Cc: Stephen Hemminger 
Cc: Vitaly Kuznetsov 
Cc: Wei Liu 
Cc: Yumei Huang 
Link: http://lkml.kernel.org/r/20200317104942.11178-3-david@redhat.com
Signed-off-by: Linus Torvalds

drivers/base/memory: rename MMOP_ONLINE_KEEP to MMOP_ONLINE

2020-04-07T17:43:40+00:00

Patch series "mm/memory_hotplug: allow to specify a default online_type", v3.

Distributions nowadays use udev rules ([1] [2]) to specify if and how to
online hotplugged memory.  The rules seem to get more complex with many
special cases.  Due to the various special cases,
CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE cannot be used.  All memory hotplug
is handled via udev rules.

Every time we hotplug memory, the udev rule will come to the same
conclusion.  Especially Hyper-V (but also soon virtio-mem) add a lot of
memory in separate memory blocks and wait for memory to get onlined by
user space before continuing to add more memory blocks (to not add memory
faster than it is getting onlined).  This of course slows down the whole
memory hotplug process.

To make the job of distributions easier and to avoid udev rules that get
more and more complicated, let's extend the mechanism provided by
- /sys/devices/system/memory/auto_online_blocks
- "memhp_default_state=" on the kernel cmdline
to be able to specify also "online_movable" as well as "online_kernel"

=== Example /usr/libexec/config-memhotplug ===

#!/bin/bash

VIRT=`systemd-detect-virt --vm`
ARCH=`uname -p`

sense_virtio_mem() {
  if [ -d "/sys/bus/virtio/drivers/virtio_mem/" ]; then
    DEVICES=`find /sys/bus/virtio/drivers/virtio_mem/ -maxdepth 1 -type l | wc -l`
    if [ $DEVICES != "0" ]; then
        return 0
    fi
  fi
  return 1
}

if [ ! -e "/sys/devices/system/memory/auto_online_blocks" ]; then
  echo "Memory hotplug configuration support missing in the kernel"
  exit 1
fi

if grep "memhp_default_state=" /proc/cmdline > /dev/null; then
  echo "Memory hotplug configuration overridden in kernel cmdline (memhp_default_state=)"
  exit 1
fi

if [ $VIRT == "microsoft" ]; then
  echo "Detected Hyper-V on $ARCH"
  # Hyper-V wants all memory in ZONE_NORMAL
  ONLINE_TYPE="online_kernel"
elif sense_virtio_mem; then
  echo "Detected virtio-mem on $ARCH"
  # virtio-mem wants all memory in ZONE_NORMAL
  ONLINE_TYPE="online_kernel"
elif [ $ARCH == "s390x" ] || [ $ARCH == "s390" ]; then
  echo "Detected $ARCH"
  # standby memory should not be onlined automatically
  ONLINE_TYPE="offline"
elif [ $ARCH == "ppc64" ] || [ $ARCH == "ppc64le" ]; then
  echo "Detected" $ARCH
  # PPC64 onlines all hotplugged memory right from the kernel
  ONLINE_TYPE="offline"
elif [ $VIRT == "none" ]; then
  echo "Detected bare-metal on $ARCH"
  # Bare metal users expect hotplugged memory to be unpluggable. We assume
  # that ZONE imbalances on such enterpise servers cannot happen and is
  # properly documented
  ONLINE_TYPE="online_movable"
else
  # TODO: Hypervisors that want to unplug DIMMs and can guarantee that ZONE
  # imbalances won't happen
  echo "Detected $VIRT on $ARCH"
  # Usually, ballooning is used in virtual environments, so memory should go to
  # ZONE_NORMAL. However, sometimes "movable_node" is relevant.
  ONLINE_TYPE="online"
fi

echo "Selected online_type:" $ONLINE_TYPE

# Configure what to do with memory that will be hotplugged in the future
echo $ONLINE_TYPE 2>/dev/null > /sys/devices/system/memory/auto_online_blocks
if [ $? != "0" ]; then
  echo "Memory hotplug cannot be configured (e.g., old kernel or missing permissions)"
  # A backup udev rule should handle old kernels if necessary
  exit 1
fi

# Process all already pluggedd blocks (e.g., DIMMs, but also Hyper-V or virtio-mem)
if [ $ONLINE_TYPE != "offline" ]; then
  for MEMORY in /sys/devices/system/memory/memory*; do
    STATE=`cat $MEMORY/state`
    if [ $STATE == "offline" ]; then
        echo $ONLINE_TYPE > $MEMORY/state
    fi
  done
fi

=== Example /usr/lib/systemd/system/config-memhotplug.service ===

[Unit]
Description=Configure memory hotplug behavior
DefaultDependencies=no
Conflicts=shutdown.target
Before=sysinit.target shutdown.target
After=systemd-modules-load.service
ConditionPathExists=|/sys/devices/system/memory/auto_online_blocks

[Service]
ExecStart=/usr/libexec/config-memhotplug
Type=oneshot
TimeoutSec=0
RemainAfterExit=yes

[Install]
WantedBy=sysinit.target

=== Example modification to the 40-redhat.rules [2] ===

: diff --git a/40-redhat.rules b/40-redhat.rules-new
: index 2c690e5..168fd03 100644
: --- a/40-redhat.rules
: +++ b/40-redhat.rules-new
: @@ -6,6 +6,9 @@ SUBSYSTEM=="cpu", ACTION=="add", TEST=="online", ATTR{online}=="0", ATTR{online}
:  # Memory hotadd request
:  SUBSYSTEM!="memory", GOTO="memory_hotplug_end"
:  ACTION!="add", GOTO="memory_hotplug_end"
: +# memory hotplug behavior configured
: +PROGRAM=="grep online /sys/devices/system/memory/auto_online_blocks", GOTO="memory_hotplug_end"
: +
:  PROGRAM="/bin/uname -p", RESULT=="s390*", GOTO="memory_hotplug_end"
:
:  ENV{.state}="online"

===

[1] https://github.com/lnykryn/systemd-rhel/pull/281
[2] https://github.com/lnykryn/systemd-rhel/blob/staging/rules/40-redhat.rules

This patch (of 8):

The name is misleading and it's not really clear what is "kept".  Let's
just name it like the online_type name we expose to user space ("online").

Add some documentation to the types.

Signed-off-by: David Hildenbrand 
Signed-off-by: Andrew Morton 
Reviewed-by: Wei Yang 
Reviewed-by: Baoquan He 
Acked-by: Pankaj Gupta 
Cc: Greg Kroah-Hartman 
Cc: Michal Hocko 
Cc: Oscar Salvador 
Cc: "Rafael J. Wysocki" 
Cc: Wei Yang 
Cc: Vitaly Kuznetsov 
Cc: Yumei Huang 
Cc: Igor Mammedov 
Cc: Eduardo Habkost 
Cc: Benjamin Herrenschmidt 
Cc: Haiyang Zhang 
Cc: K. Y. Srinivasan 
Cc: Michael Ellerman  (powerpc)
Cc: Paul Mackerras 
Cc: Stephen Hemminger 
Cc: Wei Liu 
Link: http://lkml.kernel.org/r/20200319131221.14044-1-david@redhat.com
Link: http://lkml.kernel.org/r/20200317104942.11178-2-david@redhat.com
Signed-off-by: Linus Torvalds

drivers/base/memory.c: drop pages_correctly_probed()

2020-04-07T17:43:40+00:00

pages_correctly_probed() is a leftover from ancient times.  It dates back
to commit 3947be1969a9 ("[PATCH] memory hotplug: sysfs and add/remove
functions"), where Pg_reserved checks were added as a sfety net:

	/*
	 * The probe routines leave the pages reserved, just
	 * as the bootmem code does.  Make sure they're still
	 * that way.
	 */

The checks were refactored quite a bit over the years, especially in
commit b77eab7079d9 ("mm/memory_hotplug: optimize probe routine"), where
checks for present, valid, and online sections were added.

Hotplugged memory is added via add_memory(), which will create the full
memmap for the hotplugged memory, and mark all sections valid and present.

Only full memory blocks are onlined/offlined, so we also cannot have an
inconsistency in that regard (especially, memory blocks with some sections
being online and some being offline).

1. Boot memory always starts online.  Since commit c5e79ef561b0
   ("mm/memory_hotplug.c: don't allow to online/offline memory blocks with
   holes") we disallow to offline any memory with holes.  Therefore, we
   never online memory with holes.  Present and validity checks are
   superfluous.

2. Only complete memory blocks are onlined/offlined (and especially,
   the state - online or offline - is stored for whole memory blocks).
   Besides the core, only arch/powerpc/platforms/powernv/memtrace.c
   manually calls offline_pages() and fiddels with memory block states.
   But it also only offlines complete memory blocks.

3. To make any of these conditions trigger, something would have to be
   terribly messed up in the core.  (e.g., online/offline only some
   sections of a memory block).

4. Memory unplug properly makes sure that all sysfs attributes were
   removed (and therefore, that all threads left the sysfs handlers).  We
   don't have to worry about zombie devices at this point.

5. The valid_section_nr(section_nr) check is actually dead code, as it
   would never have been reached due to the WARN_ON_ONCE(!pfn_valid(pfn)).

No wonder we haven't seen any of these errors in a long time (or even
   ever, according to my search).  Let's just get rid of them.  Now, all
   checks that could hinder onlining and offlining are completely
   contained in online_pages()/offline_pages().

Signed-off-by: David Hildenbrand 
Signed-off-by: Andrew Morton 
Cc: Greg Kroah-Hartman 
Cc: "Rafael J. Wysocki" 
Cc: Andrew Morton 
Cc: Michal Hocko 
Cc: Dan Williams 
Cc: Pavel Tatashin 
Cc: Anshuman Khandual 
Link: http://lkml.kernel.org/r/20200127110424.5757-3-david@redhat.com
Signed-off-by: Linus Torvalds