linux-stable.git/drivers/pci, branch v4.9.65 Linux kernel stable tree PCI: mvebu: Handle changes to the bridge windows while enabled 2017-11-15T14:53:11+00:00 Jason Gunthorpe jgunthorpe@obsidianresearch.com 2016-12-12T18:30:20+00:00 7ac8a10c8a50a73bb739c6f23a067857fe474d36 [ Upstream commit d9bf28e2650fe3eeefed7e34841aea07d10c6543 ] The PCI core will write to the bridge window config multiple times while they are enabled. This can lead to mbus failures like this: mvebu_mbus: cannot add window '4:e8', conflicts with another window mvebu-pcie mbus:pex@e0000000: Could not create MBus window at [mem 0xe0000000-0xe00fffff]: -22 For me this is happening during a hotplug cycle. The PCI core is not changing the values, just writing them twice while active. The patch addresses the general case of any change to an active window, but not atomically. The code is slightly refactored so io and mem can share more of the window logic. Signed-off-by: Jason Gunthorpe <jgunthorpe@obsidianresearch.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Jason Cooper <jason@lakedaemon.net> Signed-off-by: Sasha Levin <alexander.levin@verizon.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
[ Upstream commit d9bf28e2650fe3eeefed7e34841aea07d10c6543 ]

The PCI core will write to the bridge window config multiple times while
they are enabled.  This can lead to mbus failures like this:

 mvebu_mbus: cannot add window '4:e8', conflicts with another window
 mvebu-pcie mbus:pex@e0000000: Could not create MBus window at [mem 0xe0000000-0xe00fffff]: -22

For me this is happening during a hotplug cycle.  The PCI core is not
changing the values, just writing them twice while active.

The patch addresses the general case of any change to an active window, but
not atomically.  The code is slightly refactored so io and mem can share
more of the window logic.

Signed-off-by: Jason Gunthorpe <jgunthorpe@obsidianresearch.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Jason Cooper <jason@lakedaemon.net>
Signed-off-by: Sasha Levin <alexander.levin@verizon.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
PCI: Avoid possible deadlock on pci_lock and p->pi_lock 2017-11-08T09:08:37+00:00 Bjorn Helgaas bhelgaas@google.com 2017-10-07T22:37:34+00:00 df0c2d409e9d3159724b206da7e7e697fb5874eb [ Upstream commit cdcb33f9824429a926b971bf041a6cec238f91ff ] pci_lock is an IRQ-safe spinlock that protects all accesses to PCI configuration space (see PCI_OP_READ() and PCI_OP_WRITE() in pci/access.c). The pci_cfg_access_unlock() path acquires pci_lock, then p->pi_lock (inside wake_up_all()). According to lockdep, there is a possible path involving snbep_uncore_pci_read_counter() that could acquire them in the reverse order: acquiring p->pi_lock, then pci_lock, which could result in a deadlock. Lockdep details are in the bugzilla below. Avoid the possible deadlock by dropping pci_lock before waking up any config access waiters. Link: https://bugzilla.kernel.org/show_bug.cgi?id=192901 Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Sasha Levin <alexander.levin@verizon.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
[ Upstream commit cdcb33f9824429a926b971bf041a6cec238f91ff ]

pci_lock is an IRQ-safe spinlock that protects all accesses to PCI
configuration space (see PCI_OP_READ() and PCI_OP_WRITE() in pci/access.c).

The pci_cfg_access_unlock() path acquires pci_lock, then p->pi_lock (inside
wake_up_all()).  According to lockdep, there is a possible path involving
snbep_uncore_pci_read_counter() that could acquire them in the reverse
order: acquiring p->pi_lock, then pci_lock, which could result in a
deadlock.  Lockdep details are in the bugzilla below.

Avoid the possible deadlock by dropping pci_lock before waking up any
config access waiters.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=192901
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Sasha Levin <alexander.levin@verizon.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
PCI/MSI: Return failure when msix_setup_entries() fails 2017-11-08T09:08:34+00:00 Christophe JAILLET christophe.jaillet@wanadoo.fr 2017-10-07T22:36:49+00:00 c16283db12e179e34c159941f5bec0aa279529ba [ Upstream commit 3adfb572f2978a980b250a9e1a56f84f3a031001 ] If alloc_msi_entry() fails, we free resources and set ret = -ENOMEM. However, msix_setup_entries() returns 0 unconditionally. Return the error code instead. Fixes: e75eafb9b039 ("genirq/msi: Switch to new irq spreading infrastructure") Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Sasha Levin <alexander.levin@verizon.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
[ Upstream commit 3adfb572f2978a980b250a9e1a56f84f3a031001 ]

If alloc_msi_entry() fails, we free resources and set ret = -ENOMEM.

However, msix_setup_entries() returns 0 unconditionally.  Return the error
code instead.

Fixes: e75eafb9b039 ("genirq/msi: Switch to new irq spreading infrastructure")
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Sasha Levin <alexander.levin@verizon.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
PCI: Fix race condition with driver_override 2017-10-05T07:44:03+00:00 Nicolai Stange nstange@suse.de 2017-09-11T07:45:40+00:00 bb1e06d281a82c75487fb7ddf25e540b82db4306 commit 9561475db680f7144d2223a409dd3d7e322aca03 upstream. The driver_override implementation is susceptible to a race condition when different threads are reading vs. storing a different driver override. Add locking to avoid the race condition. This is in close analogy to commit 6265539776a0 ("driver core: platform: fix race condition with driver_override") from Adrian Salido. Fixes: 782a985d7af2 ("PCI: Introduce new device binding path using pci_dev.driver_override") Signed-off-by: Nicolai Stange <nstange@suse.de> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit 9561475db680f7144d2223a409dd3d7e322aca03 upstream.

The driver_override implementation is susceptible to a race condition when
different threads are reading vs. storing a different driver override.  Add
locking to avoid the race condition.

This is in close analogy to commit 6265539776a0 ("driver core: platform:
fix race condition with driver_override") from Adrian Salido.

Fixes: 782a985d7af2 ("PCI: Introduce new device binding path using pci_dev.driver_override")
Signed-off-by: Nicolai Stange <nstange@suse.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
PCI: pciehp: Report power fault only once until we clear it 2017-09-27T12:39:24+00:00 Keith Busch keith.busch@intel.com 2017-08-01T07:11:52+00:00 2fd62929c88f4cd0e4a8d920b509504249ff67aa commit 7612b3b28c0b900dcbcdf5e9b9747cc20a1e2455 upstream. When a power fault occurs, the power controller sets Power Fault Detected in the Slot Status register, and pciehp_isr() queues an INT_POWER_FAULT event to handle it. It also clears Power Fault Detected, but since nothing has yet changed to correct the power fault, the power controller will likely set it again immediately, which may cause an infinite loop when pcie_isr() rechecks Slot Status. Fix that by masking off Power Fault Detected from new events if the driver hasn't seen the power fault clear from the previous handling attempt. Fixes: fad214b0aa72 ("PCI: pciehp: Process all hotplug events before looking for new ones") Signed-off-by: Keith Busch <keith.busch@intel.com> [bhelgaas: changelog, pull test out and add comment] Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Mayurkumar Patel <mayurkumar.patel@intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit 7612b3b28c0b900dcbcdf5e9b9747cc20a1e2455 upstream.

When a power fault occurs, the power controller sets Power Fault Detected
in the Slot Status register, and pciehp_isr() queues an INT_POWER_FAULT
event to handle it.

It also clears Power Fault Detected, but since nothing has yet changed to
correct the power fault, the power controller will likely set it again
immediately, which may cause an infinite loop when pcie_isr() rechecks
Slot Status.

Fix that by masking off Power Fault Detected from new events if the driver
hasn't seen the power fault clear from the previous handling attempt.

Fixes: fad214b0aa72 ("PCI: pciehp: Process all hotplug events before looking for new ones")
Signed-off-by: Keith Busch <keith.busch@intel.com>
[bhelgaas: changelog, pull test out and add comment]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Mayurkumar Patel <mayurkumar.patel@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
PCI: shpchp: Enable bridge bus mastering if MSI is enabled 2017-09-27T12:39:23+00:00 Aleksandr Bezzubikov zuban32s@gmail.com 2017-07-18T14:12:25+00:00 998a9f51bc74933713b6308b676b941c7b1ce3aa commit 48b79a14505349a29b3e20f03619ada9b33c4b17 upstream. An SHPC may generate MSIs to notify software about slot or controller events (SHPC spec r1.0, sec 4.7). A PCI device can only generate an MSI if it has bus mastering enabled. Enable bus mastering if the bridge contains an SHPC that uses MSI for event notifications. Signed-off-by: Aleksandr Bezzubikov <zuban32s@gmail.com> [bhelgaas: changelog] Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Marcel Apfelbaum <marcel@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit 48b79a14505349a29b3e20f03619ada9b33c4b17 upstream.

An SHPC may generate MSIs to notify software about slot or controller
events (SHPC spec r1.0, sec 4.7).  A PCI device can only generate an MSI if
it has bus mastering enabled.

Enable bus mastering if the bridge contains an SHPC that uses MSI for event
notifications.

Signed-off-by: Aleksandr Bezzubikov <zuban32s@gmail.com>
[bhelgaas: changelog]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
PCI/PM: Restore the status of PCI devices across hibernation 2017-07-27T22:08:00+00:00 Chen Yu yu.c.chen@intel.com 2017-05-25T08:49:07+00:00 33780512d9fe64872b3c46596d2c6d812be98fc9 commit e60514bd4485c0c7c5a7cf779b200ce0b95c70d6 upstream. Currently we saw a lot of "No irq handler" errors during hibernation, which caused the system hang finally: ata4.00: qc timeout (cmd 0xec) ata4.00: failed to IDENTIFY (I/O error, err_mask=0x4) ata4.00: revalidation failed (errno=-5) ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300) do_IRQ: 31.151 No irq handler for vector According to above logs, there is an interrupt triggered and it is dispatched to CPU31 with a vector number 151, but there is no handler for it, thus this IRQ will not get acked and will cause an IRQ flood which kills the system. To be more specific, the 31.151 is an interrupt from the AHCI host controller. After some investigation, the reason why this issue is triggered is because the thaw_noirq() function does not restore the MSI/MSI-X settings across hibernation. The scenario is illustrated below: 1. Before hibernation, IRQ 34 is the handler for the AHCI device, which is bound to CPU31. 2. Hibernation starts, the AHCI device is put into low power state. 3. All the nonboot CPUs are put offline, so IRQ 34 has to be migrated to the last alive one - CPU0. 4. After the snapshot has been created, all the nonboot CPUs are brought up again; IRQ 34 remains bound to CPU0. 5. AHCI devices are put into D0. 6. The snapshot is written to the disk. The issue is triggered in step 6. The AHCI interrupt should be delivered to CPU0, however it is delivered to the original CPU31 instead, which causes the "No irq handler" issue. Ying Huang has provided a clue that, in step 3 it is possible that writing to the register might not take effect as the PCI devices have been suspended. In step 3, the IRQ 34 affinity should be modified from CPU31 to CPU0, but in fact it is not. In __pci_write_msi_msg(), if the device is already in low power state, the low level MSI message entry will not be updated but cached. During the device restore process after a normal suspend/resume, pci_restore_msi_state() writes the cached MSI back to the hardware. But this is not the case for hibernation. pci_restore_msi_state() is not currently called in pci_pm_thaw_noirq(), although pci_save_state() has saved the necessary PCI cached information in pci_pm_freeze_noirq(). Restore the PCI status for the device during hibernation. Otherwise the status might be lost across hibernation (for example, settings for MSI, MSI-X, ATS, ACS, IOV, etc.), which might cause problems during hibernation. Suggested-by: Ying Huang <ying.huang@intel.com> Suggested-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Signed-off-by: Chen Yu <yu.c.chen@intel.com> [bhelgaas: changelog] Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Cc: Len Brown <len.brown@intel.com> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Rui Zhang <rui.zhang@intel.com> Cc: Ying Huang <ying.huang@intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit e60514bd4485c0c7c5a7cf779b200ce0b95c70d6 upstream.

Currently we saw a lot of "No irq handler" errors during hibernation, which
caused the system hang finally:

  ata4.00: qc timeout (cmd 0xec)
  ata4.00: failed to IDENTIFY (I/O error, err_mask=0x4)
  ata4.00: revalidation failed (errno=-5)
  ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
  do_IRQ: 31.151 No irq handler for vector

According to above logs, there is an interrupt triggered and it is
dispatched to CPU31 with a vector number 151, but there is no handler for
it, thus this IRQ will not get acked and will cause an IRQ flood which
kills the system.  To be more specific, the 31.151 is an interrupt from the
AHCI host controller.

After some investigation, the reason why this issue is triggered is because
the thaw_noirq() function does not restore the MSI/MSI-X settings across
hibernation.

The scenario is illustrated below:

  1. Before hibernation, IRQ 34 is the handler for the AHCI device, which
     is bound to CPU31.

  2. Hibernation starts, the AHCI device is put into low power state.

  3. All the nonboot CPUs are put offline, so IRQ 34 has to be migrated to
     the last alive one - CPU0.

  4. After the snapshot has been created, all the nonboot CPUs are brought
     up again; IRQ 34 remains bound to CPU0.

  5. AHCI devices are put into D0.

  6. The snapshot is written to the disk.

The issue is triggered in step 6.  The AHCI interrupt should be delivered
to CPU0, however it is delivered to the original CPU31 instead, which
causes the "No irq handler" issue.

Ying Huang has provided a clue that, in step 3 it is possible that writing
to the register might not take effect as the PCI devices have been
suspended.

In step 3, the IRQ 34 affinity should be modified from CPU31 to CPU0, but
in fact it is not.  In __pci_write_msi_msg(), if the device is already in
low power state, the low level MSI message entry will not be updated but
cached.  During the device restore process after a normal suspend/resume,
pci_restore_msi_state() writes the cached MSI back to the hardware.

But this is not the case for hibernation.  pci_restore_msi_state() is not
currently called in pci_pm_thaw_noirq(), although pci_save_state() has
saved the necessary PCI cached information in pci_pm_freeze_noirq().

Restore the PCI status for the device during hibernation.  Otherwise the
status might be lost across hibernation (for example, settings for MSI,
MSI-X, ATS, ACS, IOV, etc.), which might cause problems during hibernation.

Suggested-by: Ying Huang <ying.huang@intel.com>
Suggested-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
[bhelgaas: changelog]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Len Brown <len.brown@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Rui Zhang <rui.zhang@intel.com>
Cc: Ying Huang <ying.huang@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
PCI: rockchip: Use normal register bank for config accessors 2017-07-27T22:08:00+00:00 Shawn Lin shawn.lin@rock-chips.com 2017-07-03T09:21:02+00:00 f257f4bf6f0766725fd8bcb243ad9271aee7a106 commit dc8cca5ef25ac4cb0dfc37467521a759767ff361 upstream. Rockchip's RC has two banks of registers for the root port: a normal bank that is strictly compatible with the PCIe spec, and a privileged bank that can be used to change RO bits of root port registers. When probing the RC driver, we use the privileged bank to do some basic setup work as some RO bits are hw-inited to wrong value. But we didn't change to the normal bank after probing the driver. This leads to a serious problem when the PME code tries to clear the PME status by writing PCI_EXP_RTSTA_PME to the register of PCI_EXP_RTSTA. Per PCIe 3.0 spec, section 7.8.14, the PME status bit is RW1C. So the PME code is doing the right thing to clear the PME status but we find the RC doesn't clear it but actually setting it to one. So finally the system trap in pcie_pme_work_fn() as PCI_EXP_RTSTA_PME is true now forever. This issue can be reproduced by booting kernel with pci=nomsi. Use the normal register bank for the PCI config accessors. The privileged bank is used only internally by this driver. Fixes: e77f847d ("PCI: rockchip: Add Rockchip PCIe controller support") Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Jeffy Chen <jeffy.chen@rock-chips.com> Cc: Brian Norris <briannorris@chromium.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit dc8cca5ef25ac4cb0dfc37467521a759767ff361 upstream.

Rockchip's RC has two banks of registers for the root port: a normal bank
that is strictly compatible with the PCIe spec, and a privileged bank that
can be used to change RO bits of root port registers.

When probing the RC driver, we use the privileged bank to do some basic
setup work as some RO bits are hw-inited to wrong value.  But we didn't
change to the normal bank after probing the driver.

This leads to a serious problem when the PME code tries to clear the PME
status by writing PCI_EXP_RTSTA_PME to the register of PCI_EXP_RTSTA.  Per
PCIe 3.0 spec, section 7.8.14, the PME status bit is RW1C.  So the PME code
is doing the right thing to clear the PME status but we find the RC doesn't
clear it but actually setting it to one.  So finally the system trap in
pcie_pme_work_fn() as PCI_EXP_RTSTA_PME is true now forever.  This issue
can be reproduced by booting kernel with pci=nomsi.

Use the normal register bank for the PCI config accessors.  The privileged
bank is used only internally by this driver.

Fixes: e77f847d ("PCI: rockchip: Add Rockchip PCIe controller support")
Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jeffy Chen <jeffy.chen@rock-chips.com>
Cc: Brian Norris <briannorris@chromium.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
PCI/PM: Add needs_resume flag to avoid suspend complete optimization 2017-06-17T04:41:48+00:00 Imre Deak imre.deak@intel.com 2017-05-23T19:18:17+00:00 b372d35a522617967911b72e209d35a0c4161fe6 commit 4d071c3238987325b9e50e33051a40d1cce311cc upstream. Some drivers - like i915 - may not support the system suspend direct complete optimization due to differences in their runtime and system suspend sequence. Add a flag that when set resumes the device before calling the driver's system suspend handlers which effectively disables the optimization. Needed by a future patch fixing suspend/resume on i915. Suggested by Rafael. Signed-off-by: Imre Deak <imre.deak@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Cc: stable@vger.kernel.org (rebased on v4.8, added kernel version to commit message stable tag) Signed-off-by: Imre Deak <imre.deak@intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit 4d071c3238987325b9e50e33051a40d1cce311cc upstream.

Some drivers - like i915 - may not support the system suspend direct
complete optimization due to differences in their runtime and system
suspend sequence.  Add a flag that when set resumes the device before
calling the driver's system suspend handlers which effectively disables
the optimization.

Needed by a future patch fixing suspend/resume on i915.

Suggested by Rafael.

Signed-off-by: Imre Deak <imre.deak@intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: stable@vger.kernel.org
(rebased on v4.8, added kernel version to commit message stable tag)
Signed-off-by: Imre Deak <imre.deak@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
PCI: Freeze PME scan before suspending devices 2017-05-25T13:44:48+00:00 Lukas Wunner lukas@wunner.de 2017-04-18T18:44:30+00:00 db663641619558c9a1b8b77f5bfd351e131b57a7 commit ea00353f36b64375518662a8ad15e39218a1f324 upstream. Laurent Pinchart reported that the Renesas R-Car H2 Lager board (r8a7790) crashes during suspend tests. Geert Uytterhoeven managed to reproduce the issue on an M2-W Koelsch board (r8a7791): It occurs when the PME scan runs, once per second. During PME scan, the PCI host bridge (rcar-pci) registers are accessed while its module clock has already been disabled, leading to the crash. One reproducer is to configure s2ram to use "s2idle" instead of "deep" suspend: # echo 0 > /sys/module/printk/parameters/console_suspend # echo s2idle > /sys/power/mem_sleep # echo mem > /sys/power/state Another reproducer is to write either "platform" or "processors" to /sys/power/pm_test. It does not (or is less likely) to happen during full system suspend ("core" or "none") because system suspend also disables timers, and thus the workqueue handling PME scans no longer runs. Geert believes the issue may still happen in the small window between disabling module clocks and disabling timers: # echo 0 > /sys/module/printk/parameters/console_suspend # echo platform > /sys/power/pm_test # Or "processors" # echo mem > /sys/power/state (Make sure CONFIG_PCI_RCAR_GEN2 and CONFIG_USB_OHCI_HCD_PCI are enabled.) Rafael Wysocki agrees that PME scans should be suspended before the host bridge registers become inaccessible. To that end, queue the task on a workqueue that gets frozen before devices suspend. Rafael notes however that as a result, some wakeup events may be missed if they are delivered via PME from a device without working IRQ (which hence must be polled) and occur after the workqueue has been frozen. If that turns out to be an issue in practice, it may be possible to solve it by calling pci_pme_list_scan() once directly from one of the host bridge's pm_ops callbacks. Stacktrace for posterity: PM: Syncing filesystems ... [ 38.566237] done. PM: Preparing system for sleep (mem) Freezing user space processes ... [ 38.579813] (elapsed 0.001 seconds) done. Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. PM: Suspending system (mem) PM: suspend of devices complete after 152.456 msecs PM: late suspend of devices complete after 2.809 msecs PM: noirq suspend of devices complete after 29.863 msecs suspend debug: Waiting for 5 second(s). Unhandled fault: asynchronous external abort (0x1211) at 0x00000000 pgd = c0003000 [00000000] *pgd=80000040004003, *pmd=00000000 Internal error: : 1211 [#1] SMP ARM Modules linked in: CPU: 1 PID: 20 Comm: kworker/1:1 Not tainted 4.9.0-rc1-koelsch-00011-g68db9bc814362e7f #3383 Hardware name: Generic R8A7791 (Flattened Device Tree) Workqueue: events pci_pme_list_scan task: eb56e140 task.stack: eb58e000 PC is at pci_generic_config_read+0x64/0x6c LR is at rcar_pci_cfg_base+0x64/0x84 pc : [<c041d7b4>] lr : [<c04309a0>] psr: 600d0093 sp : eb58fe98 ip : c041d750 fp : 00000008 r10: c0e2283c r9 : 00000000 r8 : 600d0013 r7 : 00000008 r6 : eb58fed6 r5 : 00000002 r4 : eb58feb4 r3 : 00000000 r2 : 00000044 r1 : 00000008 r0 : 00000000 Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user Control: 30c5387d Table: 6a9f6c80 DAC: 55555555 Process kworker/1:1 (pid: 20, stack limit = 0xeb58e210) Stack: (0xeb58fe98 to 0xeb590000) fe80: 00000002 00000044 fea0: eb6f5800 c041d9b0 eb58feb4 00000008 00000044 00000000 eb78a000 eb78a000 fec0: 00000044 00000000 eb9aff00 c0424bf0 eb78a000 00000000 eb78a000 c0e22830 fee0: ea8a6fc0 c0424c5c eaae79c0 c0424ce0 eb55f380 c0e22838 eb9a9800 c0235fbc ff00: eb55f380 c0e22838 eb55f380 eb9a9800 eb9a9800 eb58e000 eb9a9824 c0e02100 ff20: eb55f398 c02366c4 eb56e140 eb5631c0 00000000 eb55f380 c023641c 00000000 ff40: 00000000 00000000 00000000 c023a928 cd105598 00000000 40506a34 eb55f380 ff60: 00000000 00000000 dead4ead ffffffff ffffffff eb58ff74 eb58ff74 00000000 ff80: 00000000 dead4ead ffffffff ffffffff eb58ff90 eb58ff90 eb58ffac eb5631c0 ffa0: c023a844 00000000 00000000 c0206d68 00000000 00000000 00000000 00000000 ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 ffe0: 00000000 00000000 00000000 00000000 00000013 00000000 3a81336c 10ccd1dd [<c041d7b4>] (pci_generic_config_read) from [<c041d9b0>] (pci_bus_read_config_word+0x58/0x80) [<c041d9b0>] (pci_bus_read_config_word) from [<c0424bf0>] (pci_check_pme_status+0x34/0x78) [<c0424bf0>] (pci_check_pme_status) from [<c0424c5c>] (pci_pme_wakeup+0x28/0x54) [<c0424c5c>] (pci_pme_wakeup) from [<c0424ce0>] (pci_pme_list_scan+0x58/0xb4) [<c0424ce0>] (pci_pme_list_scan) from [<c0235fbc>] (process_one_work+0x1bc/0x308) [<c0235fbc>] (process_one_work) from [<c02366c4>] (worker_thread+0x2a8/0x3e0) [<c02366c4>] (worker_thread) from [<c023a928>] (kthread+0xe4/0xfc) [<c023a928>] (kthread) from [<c0206d68>] (ret_from_fork+0x14/0x2c) Code: ea000000 e5903000 f57ff04f e3a00000 (e5843000) ---[ end trace 667d43ba3aa9e589 ]--- Fixes: df17e62e5bff ("PCI: Add support for polling PME state on suspended legacy PCI devices") Reported-and-tested-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com> Reported-and-tested-by: Geert Uytterhoeven <geert+renesas@glider.be> Signed-off-by: Lukas Wunner <lukas@wunner.de> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Cc: Mika Westerberg <mika.westerberg@linux.intel.com> Cc: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se> Cc: Simon Horman <horms+renesas@verge.net.au> Cc: Yinghai Lu <yinghai@kernel.org> Cc: Matthew Garrett <mjg59@srcf.ucam.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit ea00353f36b64375518662a8ad15e39218a1f324 upstream.

Laurent Pinchart reported that the Renesas R-Car H2 Lager board (r8a7790)
crashes during suspend tests.  Geert Uytterhoeven managed to reproduce the
issue on an M2-W Koelsch board (r8a7791):

  It occurs when the PME scan runs, once per second.  During PME scan, the
  PCI host bridge (rcar-pci) registers are accessed while its module clock
  has already been disabled, leading to the crash.

One reproducer is to configure s2ram to use "s2idle" instead of "deep"
suspend:

  # echo 0 > /sys/module/printk/parameters/console_suspend
  # echo s2idle > /sys/power/mem_sleep
  # echo mem > /sys/power/state

Another reproducer is to write either "platform" or "processors" to
/sys/power/pm_test.  It does not (or is less likely) to happen during full
system suspend ("core" or "none") because system suspend also disables
timers, and thus the workqueue handling PME scans no longer runs.  Geert
believes the issue may still happen in the small window between disabling
module clocks and disabling timers:

  # echo 0 > /sys/module/printk/parameters/console_suspend
  # echo platform > /sys/power/pm_test    # Or "processors"
  # echo mem > /sys/power/state

(Make sure CONFIG_PCI_RCAR_GEN2 and CONFIG_USB_OHCI_HCD_PCI are enabled.)

Rafael Wysocki agrees that PME scans should be suspended before the host
bridge registers become inaccessible.  To that end, queue the task on a
workqueue that gets frozen before devices suspend.

Rafael notes however that as a result, some wakeup events may be missed if
they are delivered via PME from a device without working IRQ (which hence
must be polled) and occur after the workqueue has been frozen.  If that
turns out to be an issue in practice, it may be possible to solve it by
calling pci_pme_list_scan() once directly from one of the host bridge's
pm_ops callbacks.

Stacktrace for posterity:

  PM: Syncing filesystems ... [   38.566237] done.
  PM: Preparing system for sleep (mem)
  Freezing user space processes ... [   38.579813] (elapsed 0.001 seconds) done.
  Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
  PM: Suspending system (mem)
  PM: suspend of devices complete after 152.456 msecs
  PM: late suspend of devices complete after 2.809 msecs
  PM: noirq suspend of devices complete after 29.863 msecs
  suspend debug: Waiting for 5 second(s).
  Unhandled fault: asynchronous external abort (0x1211) at 0x00000000
  pgd = c0003000
  [00000000] *pgd=80000040004003, *pmd=00000000
  Internal error: : 1211 [#1] SMP ARM
  Modules linked in:
  CPU: 1 PID: 20 Comm: kworker/1:1 Not tainted
  4.9.0-rc1-koelsch-00011-g68db9bc814362e7f #3383
  Hardware name: Generic R8A7791 (Flattened Device Tree)
  Workqueue: events pci_pme_list_scan
  task: eb56e140 task.stack: eb58e000
  PC is at pci_generic_config_read+0x64/0x6c
  LR is at rcar_pci_cfg_base+0x64/0x84
  pc : [<c041d7b4>]    lr : [<c04309a0>]    psr: 600d0093
  sp : eb58fe98  ip : c041d750  fp : 00000008
  r10: c0e2283c  r9 : 00000000  r8 : 600d0013
  r7 : 00000008  r6 : eb58fed6  r5 : 00000002  r4 : eb58feb4
  r3 : 00000000  r2 : 00000044  r1 : 00000008  r0 : 00000000
  Flags: nZCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment user
  Control: 30c5387d  Table: 6a9f6c80  DAC: 55555555
  Process kworker/1:1 (pid: 20, stack limit = 0xeb58e210)
  Stack: (0xeb58fe98 to 0xeb590000)
  fe80:                                                       00000002 00000044
  fea0: eb6f5800 c041d9b0 eb58feb4 00000008 00000044 00000000 eb78a000 eb78a000
  fec0: 00000044 00000000 eb9aff00 c0424bf0 eb78a000 00000000 eb78a000 c0e22830
  fee0: ea8a6fc0 c0424c5c eaae79c0 c0424ce0 eb55f380 c0e22838 eb9a9800 c0235fbc
  ff00: eb55f380 c0e22838 eb55f380 eb9a9800 eb9a9800 eb58e000 eb9a9824 c0e02100
  ff20: eb55f398 c02366c4 eb56e140 eb5631c0 00000000 eb55f380 c023641c 00000000
  ff40: 00000000 00000000 00000000 c023a928 cd105598 00000000 40506a34 eb55f380
  ff60: 00000000 00000000 dead4ead ffffffff ffffffff eb58ff74 eb58ff74 00000000
  ff80: 00000000 dead4ead ffffffff ffffffff eb58ff90 eb58ff90 eb58ffac eb5631c0
  ffa0: c023a844 00000000 00000000 c0206d68 00000000 00000000 00000000 00000000
  ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
  ffe0: 00000000 00000000 00000000 00000000 00000013 00000000 3a81336c 10ccd1dd
  [<c041d7b4>] (pci_generic_config_read) from [<c041d9b0>]
  (pci_bus_read_config_word+0x58/0x80)
  [<c041d9b0>] (pci_bus_read_config_word) from [<c0424bf0>]
  (pci_check_pme_status+0x34/0x78)
  [<c0424bf0>] (pci_check_pme_status) from [<c0424c5c>] (pci_pme_wakeup+0x28/0x54)
  [<c0424c5c>] (pci_pme_wakeup) from [<c0424ce0>] (pci_pme_list_scan+0x58/0xb4)
  [<c0424ce0>] (pci_pme_list_scan) from [<c0235fbc>]
  (process_one_work+0x1bc/0x308)
  [<c0235fbc>] (process_one_work) from [<c02366c4>] (worker_thread+0x2a8/0x3e0)
  [<c02366c4>] (worker_thread) from [<c023a928>] (kthread+0xe4/0xfc)
  [<c023a928>] (kthread) from [<c0206d68>] (ret_from_fork+0x14/0x2c)
  Code: ea000000 e5903000 f57ff04f e3a00000 (e5843000)
  ---[ end trace 667d43ba3aa9e589 ]---

Fixes: df17e62e5bff ("PCI: Add support for polling PME state on suspended legacy PCI devices")
Reported-and-tested-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Reported-and-tested-by: Geert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: Lukas Wunner <lukas@wunner.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
Cc: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Cc: Simon Horman <horms+renesas@verge.net.au>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>