linux-stable.git/drivers/pci/remove.c, branch linux-3.4.y Linux kernel stable tree PCI/PM: Clean up PME state when removing a device 2013-02-17T18:49:26+00:00 Rafael J. Wysocki rjw@sisk.pl 2013-02-11T19:49:49+00:00 15295d7dc33491a0a015a0f48d04bbaa696b802d commit 249bfb83cf8ba658955f0245ac3981d941f746ee upstream. Devices are added to pci_pme_list when drivers use pci_enable_wake() or pci_wake_from_d3(), but they aren't removed from the list unless the driver explicitly disables wakeup. Many drivers never disable wakeup, so their devices remain on the list even after they are removed, e.g., via hotplug. A subsequent PME poll will oops when it tries to touch the device. This patch disables PME# on a device before removing it, which removes the device from pci_pme_list. This is safe even if the device never had PME# enabled. This oops can be triggered by unplugging a Thunderbolt ethernet adapter on a Macbook Pro, as reported by Daniel below. [bhelgaas: changelog] Reference: http://lkml.kernel.org/r/CAMVG2svG21yiM1wkH4_2pen2n+cr2-Zv7TbH3Gj+8MwevZjDbw@mail.gmail.com Reported-and-tested-by: Daniel J Blueman <daniel@quora.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
commit 249bfb83cf8ba658955f0245ac3981d941f746ee upstream.

Devices are added to pci_pme_list when drivers use pci_enable_wake()
or pci_wake_from_d3(), but they aren't removed from the list unless
the driver explicitly disables wakeup.  Many drivers never disable
wakeup, so their devices remain on the list even after they are
removed, e.g., via hotplug.  A subsequent PME poll will oops when
it tries to touch the device.

This patch disables PME# on a device before removing it, which removes
the device from pci_pme_list.  This is safe even if the device never
had PME# enabled.

This oops can be triggered by unplugging a Thunderbolt ethernet adapter
on a Macbook Pro, as reported by Daniel below.

[bhelgaas: changelog]
Reference: http://lkml.kernel.org/r/CAMVG2svG21yiM1wkH4_2pen2n+cr2-Zv7TbH3Gj+8MwevZjDbw@mail.gmail.com
Reported-and-tested-by: Daniel J Blueman <daniel@quora.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
PCI: export __pci_remove_bus_device 2012-02-27T20:16:55+00:00 Yinghai Lu yinghai@kernel.org 2012-02-25T21:54:21+00:00 6b22cf3f35fd332e4cc2c1b27056920b3643667a Don't switch to pci_remove_bus_device yet, keep the __ prefix for now (the behavior is still the same: remove without stopping first). This allows other out of tree users or pending patches to get notified from compiler warning. Signed-off-by: Yinghai Lu <yinghai@kernel.org> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> 
Don't switch to pci_remove_bus_device yet, keep the __ prefix for now
(the behavior is still the same: remove without stopping first).

This allows other out of tree users or pending patches to get notified
from compiler warning.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
PCI: Rename pci_remove_behind_bridge to pci_stop_and_remove_behind_bridge 2012-02-27T20:14:55+00:00 Yinghai Lu yinghai@kernel.org 2012-02-25T21:54:22+00:00 6754b9e9c33502223db066de50dda8a876f70c2c The old pci_remove_behind_bridge actually do stop and remove. Make the name reflect that to reduce confusion. Suggested-by: Jesse Barnes <jbarnes@virtuousgeek.org> Signed-off-by: Yinghai Lu <yinghai@kernel.org> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> 
The old pci_remove_behind_bridge actually do stop and remove.

Make the name reflect that to reduce confusion.

Suggested-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
PCI: Rename pci_remove_bus_device to pci_stop_and_remove_bus_device 2012-02-27T20:12:18+00:00 Yinghai Lu yinghai@kernel.org 2012-02-25T21:54:20+00:00 210647af897af8ef2d00828aa2a6b1b42206aae6 The old pci_remove_bus_device actually did stop and remove. Make the name reflect that to reduce confusion. This patch is done by sed scripts and changes back some incorrect __pci_remove_bus_device changes. Suggested-by: Jesse Barnes <jbarnes@virtuousgeek.org> Signed-off-by: Yinghai Lu <yinghai@kernel.org> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> 
The old pci_remove_bus_device actually did stop and remove.

Make the name reflect that to reduce confusion.

This patch is done by sed scripts and changes back some incorrect
__pci_remove_bus_device changes.

Suggested-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
PCI: make sriov work with hotplug remove 2012-02-14T16:44:59+00:00 Yinghai Lu yinghai@kernel.org 2012-01-27T18:55:09+00:00 ac205b7bb72fa4227d2e79979bbe2b4687cdf44d When hot removing a pci express module that has a pcie switch and supports SRIOV, we got: [ 5918.610127] pciehp 0000:80:02.2:pcie04: pcie_isr: intr_loc 1 [ 5918.615779] pciehp 0000:80:02.2:pcie04: Attention button interrupt received [ 5918.622730] pciehp 0000:80:02.2:pcie04: Button pressed on Slot(3) [ 5918.629002] pciehp 0000:80:02.2:pcie04: pciehp_get_power_status: SLOTCTRL a8 value read 1f9 [ 5918.637416] pciehp 0000:80:02.2:pcie04: PCI slot #3 - powering off due to button press. [ 5918.647125] pciehp 0000:80:02.2:pcie04: pcie_isr: intr_loc 10 [ 5918.653039] pciehp 0000:80:02.2:pcie04: pciehp_green_led_blink: SLOTCTRL a8 write cmd 200 [ 5918.661229] pciehp 0000:80:02.2:pcie04: pciehp_set_attention_status: SLOTCTRL a8 write cmd c0 [ 5924.667627] pciehp 0000:80:02.2:pcie04: Disabling domain:bus:device=0000:b0:00 [ 5924.674909] pciehp 0000:80:02.2:pcie04: pciehp_get_power_status: SLOTCTRL a8 value read 2f9 [ 5924.683262] pciehp 0000:80:02.2:pcie04: pciehp_unconfigure_device: domain:bus:dev = 0000:b0:00 [ 5924.693976] libfcoe_device_notification: NETDEV_UNREGISTER eth6 [ 5924.764979] libfcoe_device_notification: NETDEV_UNREGISTER eth14 [ 5924.873539] libfcoe_device_notification: NETDEV_UNREGISTER eth15 [ 5924.995209] libfcoe_device_notification: NETDEV_UNREGISTER eth16 [ 5926.114407] sxge 0000:b2:00.0: PCI INT A disabled [ 5926.119342] BUG: unable to handle kernel NULL pointer dereference at (null) [ 5926.127189] IP: [<ffffffff81353a3b>] pci_stop_bus_device+0x33/0x83 [ 5926.133377] PGD 0 [ 5926.135402] Oops: 0000 [#1] SMP [ 5926.138659] CPU 2 [ 5926.140499] Modules linked in: ... [ 5926.143754] [ 5926.275823] Call Trace: [ 5926.278267] [<ffffffff81353a38>] pci_stop_bus_device+0x30/0x83 [ 5926.284180] [<ffffffff81353af4>] pci_remove_bus_device+0x1a/0xba [ 5926.290264] [<ffffffff81366311>] pciehp_unconfigure_device+0x110/0x17b [ 5926.296866] [<ffffffff81365dd9>] ? pciehp_disable_slot+0x188/0x188 [ 5926.303123] [<ffffffff81365d6f>] pciehp_disable_slot+0x11e/0x188 [ 5926.309206] [<ffffffff81365e68>] pciehp_power_thread+0x8f/0xe0 ... +-[0000:80]-+-00.0-[81-8f]-- | +-01.0-[90-9f]-- | +-02.0-[a0-af]-- | +-02.2-[b0-bf]----00.0-[b1-b3]--+-02.0-[b2]--+-00.0 Device | | | +-00.1 Device | | | +-00.2 Device | | | \-00.3 Device | | \-03.0-[b3]--+-00.0 Device | | +-00.1 Device | | +-00.2 Device | | \-00.3 Device root complex: 80:02.2 pci express modules: have pcie switch and are listed as b0:00.0, b1:02.0 and b1:03.0. end devices are b2:00.0 and b3.00.0. VFs are: b2:00.1,... b2:00.3, and b3:00.1,...,b3:00.3 Root cause: when doing pci_stop_bus_device() with phys fn, it will stop virt fn and remove the fn, so list_for_each_safe(l, n, &bus->devices) will have problem to refer freed n that is pointed to vf entry. Solution is just replacing list_for_each_safe() with list_for_each_prev_safe(). This will make sure we can get valid n pointer to PF instead of the freed VF pointer (because newly added devices are inserted to the bus->devices list tail). During reviewing the patch, Bjorn said: | The PCI hot-remove path calls pci_stop_bus_devices() via | pci_remove_bus_device(). | | pci_stop_bus_devices() traverses the bus->devices list (point A below), | stopping each device in turn, which calls the driver remove() method. When | the device is an SR-IOV PF, the driver calls pci_disable_sriov(), which | also uses pci_remove_bus_device() to remove the VF devices from the | bus->devices list (point B). | | pci_remove_bus_device | pci_stop_bus_device | pci_stop_bus_devices(subordinate) | list_for_each(bus->devices) <-- A | pci_stop_bus_device(PF) | ... | driver->remove | pci_disable_sriov | ... | pci_remove_bus_device(VF) | <remove from bus_list> <-- B | | At B, we're changing the same list we're iterating through at A, so when | the driver remove() method returns, the pci_stop_bus_devices() iterator has | a pointer to a list entry that has already been freed. Discussion thread can be found : https://lkml.org/lkml/2011/10/15/141 https://lkml.org/lkml/2012/1/23/360 -v5: According to Linus to make remove more robust, Change to list_for_each_prev_safe instead. That is more reasonable, because those devices are added to tail of the list before. Signed-off-by: Yinghai Lu <yinghai@kernel.org> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> 
When hot removing a pci express module that has a pcie switch and supports
SRIOV, we got:

[ 5918.610127] pciehp 0000:80:02.2:pcie04: pcie_isr: intr_loc 1
[ 5918.615779] pciehp 0000:80:02.2:pcie04: Attention button interrupt received
[ 5918.622730] pciehp 0000:80:02.2:pcie04: Button pressed on Slot(3)
[ 5918.629002] pciehp 0000:80:02.2:pcie04: pciehp_get_power_status: SLOTCTRL a8 value read 1f9
[ 5918.637416] pciehp 0000:80:02.2:pcie04: PCI slot #3 - powering off due to button press.
[ 5918.647125] pciehp 0000:80:02.2:pcie04: pcie_isr: intr_loc 10
[ 5918.653039] pciehp 0000:80:02.2:pcie04: pciehp_green_led_blink: SLOTCTRL a8 write cmd 200
[ 5918.661229] pciehp 0000:80:02.2:pcie04: pciehp_set_attention_status: SLOTCTRL a8 write cmd c0
[ 5924.667627] pciehp 0000:80:02.2:pcie04: Disabling domain:bus:device=0000:b0:00
[ 5924.674909] pciehp 0000:80:02.2:pcie04: pciehp_get_power_status: SLOTCTRL a8 value read 2f9
[ 5924.683262] pciehp 0000:80:02.2:pcie04: pciehp_unconfigure_device: domain:bus:dev = 0000:b0:00
[ 5924.693976] libfcoe_device_notification: NETDEV_UNREGISTER eth6
[ 5924.764979] libfcoe_device_notification: NETDEV_UNREGISTER eth14
[ 5924.873539] libfcoe_device_notification: NETDEV_UNREGISTER eth15
[ 5924.995209] libfcoe_device_notification: NETDEV_UNREGISTER eth16
[ 5926.114407] sxge 0000:b2:00.0: PCI INT A disabled
[ 5926.119342] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 5926.127189] IP: [<ffffffff81353a3b>] pci_stop_bus_device+0x33/0x83
[ 5926.133377] PGD 0
[ 5926.135402] Oops: 0000 [#1] SMP
[ 5926.138659] CPU 2
[ 5926.140499] Modules linked in:
...
[ 5926.143754]
[ 5926.275823] Call Trace:
[ 5926.278267]  [<ffffffff81353a38>] pci_stop_bus_device+0x30/0x83
[ 5926.284180]  [<ffffffff81353af4>] pci_remove_bus_device+0x1a/0xba
[ 5926.290264]  [<ffffffff81366311>] pciehp_unconfigure_device+0x110/0x17b
[ 5926.296866]  [<ffffffff81365dd9>] ? pciehp_disable_slot+0x188/0x188
[ 5926.303123]  [<ffffffff81365d6f>] pciehp_disable_slot+0x11e/0x188
[ 5926.309206]  [<ffffffff81365e68>] pciehp_power_thread+0x8f/0xe0
...

 +-[0000:80]-+-00.0-[81-8f]--
 |           +-01.0-[90-9f]--
 |           +-02.0-[a0-af]--
 |           +-02.2-[b0-bf]----00.0-[b1-b3]--+-02.0-[b2]--+-00.0 Device
 |           |                               |            +-00.1 Device
 |           |                               |            +-00.2 Device
 |           |                               |            \-00.3 Device
 |           |                               \-03.0-[b3]--+-00.0 Device
 |           |                                            +-00.1 Device
 |           |                                            +-00.2 Device
 |           |                                            \-00.3 Device

root complex: 80:02.2
pci express modules: have pcie switch and are listed as b0:00.0, b1:02.0 and b1:03.0.
end devices  are b2:00.0 and b3.00.0.
VFs are: b2:00.1,... b2:00.3, and b3:00.1,...,b3:00.3

Root cause: when doing pci_stop_bus_device() with phys fn, it will stop
virt fn and remove the fn, so
	list_for_each_safe(l, n, &bus->devices)
will have problem to refer freed n that is pointed to vf entry.

Solution is just replacing list_for_each_safe() with
list_for_each_prev_safe().  This will make sure we can get valid n pointer
to PF instead of the freed VF pointer (because newly added devices are
inserted to the bus->devices list tail).

During reviewing the patch, Bjorn said:
|   The PCI hot-remove path calls pci_stop_bus_devices() via
|   pci_remove_bus_device().
|
|   pci_stop_bus_devices() traverses the bus->devices list (point A below),
|   stopping each device in turn, which calls the driver remove() method.  When
|   the device is an SR-IOV PF, the driver calls pci_disable_sriov(), which
|   also uses pci_remove_bus_device() to remove the VF devices from the
|   bus->devices list (point B).
|
|       pci_remove_bus_device
|         pci_stop_bus_device
|           pci_stop_bus_devices(subordinate)
|             list_for_each(bus->devices)             <-- A
|               pci_stop_bus_device(PF)
|                 ...
|                   driver->remove
|                     pci_disable_sriov
|                       ...
|                         pci_remove_bus_device(VF)
|                             <remove from bus_list>  <-- B
|
|   At B, we're changing the same list we're iterating through at A, so when
|   the driver remove() method returns, the pci_stop_bus_devices() iterator has
|   a pointer to a list entry that has already been freed.

Discussion thread can be found : https://lkml.org/lkml/2011/10/15/141
				 https://lkml.org/lkml/2012/1/23/360

-v5: According to Linus to make remove more robust, Change to
     list_for_each_prev_safe instead. That is more reasonable, because
     those devices are added to tail of the list before.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
PCI: Fix pci cardbus removal 2012-02-10T20:19:31+00:00 Yinghai Lu yinghai@kernel.org 2012-02-05T06:55:00+00:00 3682a3946d2b0bad621db871e3bead83e523a238 During test busn_res allocation with cardbus, found pci card removal is not working anymore, and it turns out it is broken by: |commit 79cc9601c3e42b4f0650fe7e69132ebce7ab48f9 |Date: Tue Nov 22 21:06:53 2011 -0800 | | PCI: Only call pci_stop_bus_device() one time for child devices at remove The above changed the behavior of pci_remove_behind_bridge that yenta_cardbus depended on. So restore the old behavoir of pci_remove_behind_bridge (which requires stopping and removing of all devices) by: 1. rename pci_remove_behind_bridge to __pci_remove_behind_bridge, and let __pci_remove_bus_device() call it instead. 2. add pci_stop_behind_bridge that will stop devices behind a bridge 3. add back pci_remove_behind_bridge that will stop and remove devices under bridge. -v2: update commit description a little bit. Tested-by: Dominik Brodowski <linux@dominikbrodowski.net> Signed-off-by: Yinghai Lu <yinghai@kernel.org> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> 
During test busn_res allocation with cardbus, found pci card removal is not
working anymore, and it turns out it is broken by:

|commit 79cc9601c3e42b4f0650fe7e69132ebce7ab48f9
|Date:   Tue Nov 22 21:06:53 2011 -0800
|
|    PCI: Only call pci_stop_bus_device() one time for child devices at remove

The above changed the behavior of pci_remove_behind_bridge that
yenta_cardbus depended on.  So restore the old behavoir of
pci_remove_behind_bridge (which requires stopping and removing of all
devices) by:

1. rename pci_remove_behind_bridge to __pci_remove_behind_bridge, and let
   __pci_remove_bus_device() call it instead.
2. add pci_stop_behind_bridge that will stop devices behind a bridge
3. add back pci_remove_behind_bridge that will stop and remove devices
   under bridge.

-v2: update commit description a little bit.

Tested-by: Dominik Brodowski <linux@dominikbrodowski.net>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
PCI: Only call pci_stop_bus_device() one time for child devices at remove 2012-01-06T20:10:48+00:00 Yinghai Lu yinghai.lu@oracle.com 2011-11-23T05:06:53+00:00 79cc9601c3e42b4f0650fe7e69132ebce7ab48f9 During debugging pcie hotplug with SRIOV with pcie switch, I found pci_stop_bus_device() is called several times for some child devices. So change original pci_remove_bus_device() to __pci_remove_bus_device(), and make it only do remove work, and add a new pci_remove_bus_device that calls pci_stop_bus_device() one time, and then call __pci_remove_bus_device(). Signed-off-by: Yinghai Lu <yinghai@kernel.org> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> 
During debugging pcie hotplug with SRIOV with pcie switch, I found
pci_stop_bus_device() is called several times for some child devices.

So change original pci_remove_bus_device() to __pci_remove_bus_device(),
and make it only do remove work, and add a new pci_remove_bus_device
that calls pci_stop_bus_device() one time, and then call
__pci_remove_bus_device().

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
PCI/sysfs: move bus cpuaffinity to class dev_attrs 2011-05-21T19:17:13+00:00 Yinghai Lu yinghai@kernel.org 2011-05-13T00:11:40+00:00 dc2c2c9dd513dec6c17df04e8abff795e20a5271 Requested by Greg KH to fix a race condition in the creating of PCI bus cpuaffinity files. Acked-by: Greg Kroah-Hartman <gregkh@suse.de> Signed-off-by: Yinghai Lu <yinghai@kernel.org> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> 
Requested by Greg KH to fix a race condition in the creating of PCI bus
cpuaffinity files.

Acked-by: Greg Kroah-Hartman <gregkh@suse.de>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
PCI: eliminate redundant pci_stop_dev() call from pci_destroy_dev() 2009-06-11T19:04:19+00:00 Alex Chiang achiang@hp.com 2009-05-19T01:02:38+00:00 af4c5f985afd8d4cfdf402aaa03677f2cb96e37c We always call pci_stop_bus_device before calling pci_destroy_dev. Since pci_stop_bus_device calls pci_stop_dev, there is no need for pci_destroy_dev to repeat the call. Signed-off-by: Alex Chiang <achiang@hp.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> 
We always call pci_stop_bus_device before calling pci_destroy_dev.

Since pci_stop_bus_device calls pci_stop_dev, there is no need
for pci_destroy_dev to repeat the call.

Signed-off-by: Alex Chiang <achiang@hp.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
PCI: fix kernel oops on bridge removal 2009-03-26T22:55:18+00:00 Kenji Kaneshige kaneshige.kenji@jp.fujitsu.com 2009-03-26T07:49:52+00:00 7ae0567fd3f4f51d55c4c638ecc6836347992de2 Fix the following kernel oops problem that happens when removing PCI bridge with pciehp loaded. It should also occur with other hotplug driver that is implemented as a bridge's driver. [ 459.997257] pciehp 0000:2f:04.0:pcie24: unloading service driver pciehp [ 459.997495] general protection fault: 0000 [#1] SMP [ 459.997737] last sysfs file: /sys/devices/pci0000:00/0000:00:04.0/0000:2e:00.0/0000:2f:04.0/remove [ 459.997964] CPU 4 [ 459.998129] Modules linked in: pciehp ipv6 autofs4 hidp rfcomm l2cap bluetooth sunrpc cpufreq_ondemand acpi_cpufreq dm_mirror dm_region_hash dm_log dm_multipath scsi_dh dm_mod sbs sbshc battery ac parport_pc lp parport mptspi mptscsih mptbase scsi_transport_spi e1000e sg sr_mod cdrom button serio_raw i2c_i801 i2c_core shpchp pcspkr ata_piix libata megaraid_sas sd_mod scsi_mod crc_t10dif ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode] [ 459.998129] Pid: 56, comm: events/4 Not tainted 2.6.29-rc8-kk #1 PRIMERGY [ 459.998129] RIP: 0010:[<ffffffff803bf047>] [<ffffffff803bf047>] pci_slot_release+0x37/0x100 [ 459.998129] RSP: 0018:ffff88083b3bf9e0 EFLAGS: 00010246 [ 459.998129] RAX: ffff88083adc5158 RBX: ffff880836c1bc80 RCX: 6b6b6b6b6b6b6b6b [ 459.998129] RDX: 0000000000000000 RSI: ffffffff803a77f0 RDI: ffff880836c1bc48 [ 459.998129] RBP: ffff88083b3bfa00 R08: 0000000000000002 R09: 0000000000000000 [ 459.998129] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880836c1bc48 [ 459.998129] R13: ffff880836c1bc20 R14: ffff880836c1bc48 R15: ffff880836d1ec38 [ 459.998129] FS: 0000000000000000(0000) GS:ffff88083ccc3770(0000) knlGS:0000000000000000 [ 459.998129] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b [ 459.998129] CR2: 00007f1562f1d558 CR3: 0000000838090000 CR4: 00000000000006e0 [ 459.998129] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 459.998129] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 459.998129] Process events/4 (pid: 56, threadinfo ffff88083b3be000, task ffff88083b3b3e40) [ 459.998129] Stack: [ 459.998129] ffff880836c1bc80 ffff880836c1bc48 ffffffff80793320 ffff88083b0d0960 [ 459.998129] ffff88083b3bfa30 ffffffff803a788a ffff880836c1bc80 ffffffff803a77f0 [ 459.998129] ffff880836c1bc20 ffff880836d1ec38 ffff88083b3bfa50 ffffffff803a8ce7 [ 459.998129] Call Trace: [ 459.998129] [<ffffffff803a788a>] kobject_release+0x9a/0x290 [ 459.998129] [<ffffffff803a77f0>] ? kobject_release+0x0/0x290 [ 459.998129] [<ffffffff803a8ce7>] kref_put+0x37/0x80 [ 459.998129] [<ffffffff803a76f7>] kobject_put+0x27/0x60 [ 459.998129] [<ffffffff803bebcc>] ? pci_destroy_slot+0x3c/0xc0 [ 459.998129] [<ffffffff803bebd5>] pci_destroy_slot+0x45/0xc0 [ 459.998129] [<ffffffff803c797d>] pci_hp_deregister+0x13d/0x210 [ 459.998129] [<ffffffffa031141d>] cleanup_slots+0x2d/0x80 [pciehp] [ 459.998129] [<ffffffffa0311735>] pciehp_remove+0x15/0x30 [pciehp] [ 459.998129] [<ffffffff803c4c99>] pcie_port_remove_service+0x69/0x90 [ 459.998129] [<ffffffff80441da9>] __device_release_driver+0x59/0x90 [ 459.998129] [<ffffffff80441edb>] device_release_driver+0x2b/0x40 [ 459.998129] [<ffffffff804419d6>] bus_remove_device+0xa6/0x120 [ 459.998129] [<ffffffff8043e46b>] device_del+0x12b/0x190 [ 459.998129] [<ffffffff803c4d90>] ? remove_iter+0x0/0x40 [ 459.998129] [<ffffffff8043e4f6>] device_unregister+0x26/0x70 [ 459.998129] [<ffffffff803c4dbf>] remove_iter+0x2f/0x40 [ 459.998129] [<ffffffff8043ddf3>] device_for_each_child+0x33/0x60 [ 459.998129] [<ffffffff8033ee30>] ? sysfs_schedule_callback_work+0x0/0x50 [ 459.998129] [<ffffffff803c4d30>] pcie_port_device_remove+0x30/0x80 [ 459.998129] [<ffffffff803c55a1>] pcie_portdrv_remove+0x11/0x20 [ 459.998129] [<ffffffff803bfeb2>] pci_device_remove+0x32/0x70 [ 459.998129] [<ffffffff80441da9>] __device_release_driver+0x59/0x90 [ 459.998129] [<ffffffff80441edb>] device_release_driver+0x2b/0x40 [ 459.998129] [<ffffffff804419d6>] bus_remove_device+0xa6/0x120 [ 459.998129] [<ffffffff8043e46b>] device_del+0x12b/0x190 [ 459.998129] [<ffffffff8043e4f6>] device_unregister+0x26/0x70 [ 459.998129] [<ffffffff803ba969>] pci_stop_dev+0x49/0x60 [ 459.998129] [<ffffffff803baab0>] pci_remove_bus_device+0x40/0xc0 [ 459.998129] [<ffffffff803c10d9>] remove_callback+0x29/0x40 [ 459.998129] [<ffffffff8033ee4f>] sysfs_schedule_callback_work+0x1f/0x50 [ 459.998129] [<ffffffff8025769a>] run_workqueue+0x15a/0x230 [ 459.998129] [<ffffffff80257648>] ? run_workqueue+0x108/0x230 [ 459.998129] [<ffffffff8025846f>] worker_thread+0x9f/0x100 [ 459.998129] [<ffffffff8025bce0>] ? autoremove_wake_function+0x0/0x40 [ 459.998129] [<ffffffff802583d0>] ? worker_thread+0x0/0x100 [ 459.998129] [<ffffffff8025b89d>] kthread+0x4d/0x80 [ 459.998129] [<ffffffff8020d4ba>] child_rip+0xa/0x20 [ 459.998129] [<ffffffff8020cebc>] ? restore_args+0x0/0x30 [ 459.998129] [<ffffffff8025b850>] ? kthread+0x0/0x80 [ 459.998129] [<ffffffff8020d4b0>] ? child_rip+0x0/0x20 [ 459.998129] Code: 56 49 89 fe 41 55 4c 8d 6f d8 41 54 53 74 09 f6 05 b8 05 c7 00 08 75 72 49 8b 45 00 48 8b 48 28 eb 05 66 90 48 89 f1 49 8b 45 00 <48> 8b 31 48 83 c0 28 0f 18 0e 48 39 c1 74 1c 8b 41 38 41 0f b6 [ 459.998129] RIP [<ffffffff803bf047>] pci_slot_release+0x37/0x100 [ 459.998129] RSP <ffff88083b3bf9e0> [ 460.018595] ---[ end trace 5a08d2095374aedc ]--- The pci_remove_bus_device() removes all buses and devices under the bridge, and then removes the bridge. So the remove() callback of the hotplug drivers implemented as a bridge's driver is executed after the struct pci_bus of the bridge's secondary bus is removed. The remove() callback of those driver unregisters the slot using pci_destroy_slot(), and slot's release callback refers to the the struct pci_bus that was already freed. This is the cause of the kernel oops. This patch solves the problem by stopping bus drivers before removing the bridge and its child bus and devices. Acked-by: Alex Chiang <achiang@hp.com> Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> 
Fix the following kernel oops problem that happens when removing PCI
bridge with pciehp loaded. It should also occur with other hotplug
driver that is implemented as a bridge's driver.

[  459.997257] pciehp 0000:2f:04.0:pcie24: unloading service driver pciehp
[  459.997495] general protection fault: 0000 [#1] SMP
[  459.997737] last sysfs file: /sys/devices/pci0000:00/0000:00:04.0/0000:2e:00.0/0000:2f:04.0/remove
[  459.997964] CPU 4
[  459.998129] Modules linked in: pciehp ipv6 autofs4 hidp rfcomm l2cap bluetooth sunrpc cpufreq_ondemand acpi_cpufreq dm_mirror dm_region_hash dm_log dm_multipath scsi_dh dm_mod sbs sbshc battery ac parport_pc lp parport mptspi mptscsih mptbase scsi_transport_spi e1000e sg sr_mod cdrom button serio_raw i2c_i801 i2c_core shpchp pcspkr ata_piix libata megaraid_sas sd_mod scsi_mod crc_t10dif ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
[  459.998129] Pid: 56, comm: events/4 Not tainted 2.6.29-rc8-kk #1 PRIMERGY
[  459.998129] RIP: 0010:[<ffffffff803bf047>]  [<ffffffff803bf047>] pci_slot_release+0x37/0x100
[  459.998129] RSP: 0018:ffff88083b3bf9e0  EFLAGS: 00010246
[  459.998129] RAX: ffff88083adc5158 RBX: ffff880836c1bc80 RCX: 6b6b6b6b6b6b6b6b
[  459.998129] RDX: 0000000000000000 RSI: ffffffff803a77f0 RDI: ffff880836c1bc48
[  459.998129] RBP: ffff88083b3bfa00 R08: 0000000000000002 R09: 0000000000000000
[  459.998129] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880836c1bc48
[  459.998129] R13: ffff880836c1bc20 R14: ffff880836c1bc48 R15: ffff880836d1ec38
[  459.998129] FS:  0000000000000000(0000) GS:ffff88083ccc3770(0000) knlGS:0000000000000000
[  459.998129] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
[  459.998129] CR2: 00007f1562f1d558 CR3: 0000000838090000 CR4: 00000000000006e0
[  459.998129] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  459.998129] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  459.998129] Process events/4 (pid: 56, threadinfo ffff88083b3be000, task ffff88083b3b3e40)
[  459.998129] Stack:
[  459.998129]  ffff880836c1bc80 ffff880836c1bc48 ffffffff80793320 ffff88083b0d0960
[  459.998129]  ffff88083b3bfa30 ffffffff803a788a ffff880836c1bc80 ffffffff803a77f0
[  459.998129]  ffff880836c1bc20 ffff880836d1ec38 ffff88083b3bfa50 ffffffff803a8ce7
[  459.998129] Call Trace:
[  459.998129]  [<ffffffff803a788a>] kobject_release+0x9a/0x290
[  459.998129]  [<ffffffff803a77f0>] ? kobject_release+0x0/0x290
[  459.998129]  [<ffffffff803a8ce7>] kref_put+0x37/0x80
[  459.998129]  [<ffffffff803a76f7>] kobject_put+0x27/0x60
[  459.998129]  [<ffffffff803bebcc>] ? pci_destroy_slot+0x3c/0xc0
[  459.998129]  [<ffffffff803bebd5>] pci_destroy_slot+0x45/0xc0
[  459.998129]  [<ffffffff803c797d>] pci_hp_deregister+0x13d/0x210
[  459.998129]  [<ffffffffa031141d>] cleanup_slots+0x2d/0x80 [pciehp]
[  459.998129]  [<ffffffffa0311735>] pciehp_remove+0x15/0x30 [pciehp]
[  459.998129]  [<ffffffff803c4c99>] pcie_port_remove_service+0x69/0x90
[  459.998129]  [<ffffffff80441da9>] __device_release_driver+0x59/0x90
[  459.998129]  [<ffffffff80441edb>] device_release_driver+0x2b/0x40
[  459.998129]  [<ffffffff804419d6>] bus_remove_device+0xa6/0x120
[  459.998129]  [<ffffffff8043e46b>] device_del+0x12b/0x190
[  459.998129]  [<ffffffff803c4d90>] ? remove_iter+0x0/0x40
[  459.998129]  [<ffffffff8043e4f6>] device_unregister+0x26/0x70
[  459.998129]  [<ffffffff803c4dbf>] remove_iter+0x2f/0x40
[  459.998129]  [<ffffffff8043ddf3>] device_for_each_child+0x33/0x60
[  459.998129]  [<ffffffff8033ee30>] ? sysfs_schedule_callback_work+0x0/0x50
[  459.998129]  [<ffffffff803c4d30>] pcie_port_device_remove+0x30/0x80
[  459.998129]  [<ffffffff803c55a1>] pcie_portdrv_remove+0x11/0x20
[  459.998129]  [<ffffffff803bfeb2>] pci_device_remove+0x32/0x70
[  459.998129]  [<ffffffff80441da9>] __device_release_driver+0x59/0x90
[  459.998129]  [<ffffffff80441edb>] device_release_driver+0x2b/0x40
[  459.998129]  [<ffffffff804419d6>] bus_remove_device+0xa6/0x120
[  459.998129]  [<ffffffff8043e46b>] device_del+0x12b/0x190
[  459.998129]  [<ffffffff8043e4f6>] device_unregister+0x26/0x70
[  459.998129]  [<ffffffff803ba969>] pci_stop_dev+0x49/0x60
[  459.998129]  [<ffffffff803baab0>] pci_remove_bus_device+0x40/0xc0
[  459.998129]  [<ffffffff803c10d9>] remove_callback+0x29/0x40
[  459.998129]  [<ffffffff8033ee4f>] sysfs_schedule_callback_work+0x1f/0x50
[  459.998129]  [<ffffffff8025769a>] run_workqueue+0x15a/0x230
[  459.998129]  [<ffffffff80257648>] ? run_workqueue+0x108/0x230
[  459.998129]  [<ffffffff8025846f>] worker_thread+0x9f/0x100
[  459.998129]  [<ffffffff8025bce0>] ? autoremove_wake_function+0x0/0x40
[  459.998129]  [<ffffffff802583d0>] ? worker_thread+0x0/0x100
[  459.998129]  [<ffffffff8025b89d>] kthread+0x4d/0x80
[  459.998129]  [<ffffffff8020d4ba>] child_rip+0xa/0x20
[  459.998129]  [<ffffffff8020cebc>] ? restore_args+0x0/0x30
[  459.998129]  [<ffffffff8025b850>] ? kthread+0x0/0x80
[  459.998129]  [<ffffffff8020d4b0>] ? child_rip+0x0/0x20
[  459.998129] Code: 56 49 89 fe 41 55 4c 8d 6f d8 41 54 53 74 09 f6 05 b8 05 c7 00 08 75 72 49 8b 45 00 48 8b 48 28 eb 05 66 90 48 89 f1 49 8b 45 00 <48> 8b 31 48 83 c0 28 0f 18 0e 48 39 c1 74 1c 8b 41 38 41 0f b6
[  459.998129] RIP  [<ffffffff803bf047>] pci_slot_release+0x37/0x100
[  459.998129]  RSP <ffff88083b3bf9e0>
[  460.018595] ---[ end trace 5a08d2095374aedc ]---

The pci_remove_bus_device() removes all buses and devices under the
bridge, and then removes the bridge. So the remove() callback of the
hotplug drivers implemented as a bridge's driver is executed after the
struct pci_bus of the bridge's secondary bus is removed. The remove()
callback of those driver unregisters the slot using pci_destroy_slot(),
and slot's release callback refers to the the struct pci_bus that was
already freed. This is the cause of the kernel oops.

This patch solves the problem by stopping bus drivers before removing the
bridge and its child bus and devices.

Acked-by: Alex Chiang <achiang@hp.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>