diff options
| author | Denis V. Lunev <den@openvz.org> | 2026-07-26 12:43:11 +0200 |
|---|---|---|
| committer | Paolo Abeni <pabeni@redhat.com> | 2026-07-30 15:16:38 +0200 |
| commit | 451c9075d6c53f2438d110addbeeeea6fac18567 (patch) | |
| tree | 33acfb44ac676a010d9d23aea9561b81e1c08bf0 /tools/perf/scripts/python/bin | |
| parent | c92922c02caf2b9bb506a59f32f9187a21ab0be9 (diff) | |
qede: sync udp_tunnel ports outside qede_lock in the recovery path
A TX timeout on a qede NIC that has VXLAN/GENEVE tunnel ports
configured wedges the rtnetlink control plane of the whole machine:
NETDEV WATCHDOG: ens6f1 (qede): transmit queue 2 timed out 10226 ms
[qede_tx_timeout:586(ens6f1)]TX timeout on queue 2!
[qede_recovery_handler:2665(ens6f0)]Starting a recovery process
The recovery path deadlocks on the driver's own mutex:
qede_sp_task
rtnl_lock()
mutex_lock(&edev->qede_lock) <- taken
qede_recovery_handler
qede_load
udp_tunnel_nic_reset_ntf
__udp_tunnel_nic_device_sync
info->sync_table == qede_udp_tunnel_sync
mutex_lock(&edev->qede_lock) <- same task: deadlock
The mutex is not recursive, so the kworker blocks on itself with
rtnl_lock held, and neither lock is ever released. Every task that
calls rtnl_lock() afterwards (ip, ovs-vswitchd, lldpad, IPv6
addrconf, sshd) blocks forever while the node still answers ping.
In a vmcore from an affected production node rtnl_mutex.owner
decodes to the very kworker blocked at the innermost mutex_lock()
above.
Re-sync the tunnel ports from qede_sp_task() after the internal lock
is dropped, still under rtnl_lock as the udp_tunnel API requires.
This mirrors qede_open(), which calls udp_tunnel_nic_reset_ntf()
under rtnl without the internal lock.
qede_recovery_handler() now returns whether it has successfully
reloaded an open device, and the caller re-syncs the ports only in
that case. This keeps the old gating exactly: a device that was down
or a failed recovery returns false, as those paths never reached the
udp_tunnel_nic_reset_ntf() call before either.
This was the only user of the qede_lock()/qede_unlock() helpers, so
remove them.
Fixes: 8cd160a29415 ("qede: convert to new udp_tunnel_nic infra")
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Andrew Lunn <andrew+netdev@lunn.ch>
CC: "David S. Miller" <davem@davemloft.net>
CC: Eric Dumazet <edumazet@google.com>
CC: Jakub Kicinski <kuba@kernel.org>
CC: Paolo Abeni <pabeni@redhat.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20260726104311.1782900-1-den@openvz.org
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Diffstat (limited to 'tools/perf/scripts/python/bin')
0 files changed, 0 insertions, 0 deletions
