linux-stable.git/drivers/infiniband/ulp/ipoib, branch linux-3.18.y

IB/ipoib: Avoid a race condition between start_xmit and cm_rep_handler

2018-09-26T06:33:57+00:00

commit 816e846c2eb9129a3e0afa5f920c8bbc71efecaa upstream.

Inside of start_xmit() the call to check if the connection is up and the
queueing of the packets for later transmission is not atomic which leaves
a window where cm_rep_handler can run, set the connection up, dequeue
pending packets and leave the subsequently queued packets by start_xmit()
sitting on neigh->queue until they're dropped when the connection is torn
down. This only applies to connected mode. These dropped packets can
really upset TCP, for example, and cause multi-minute delays in
transmission for open connections.

Here's the code in start_xmit where we check to see if the connection is
up:

       if (ipoib_cm_get(neigh)) {
               if (ipoib_cm_up(neigh)) {
                       ipoib_cm_send(dev, skb, ipoib_cm_get(neigh));
                       goto unref;
               }
       }

The race occurs if cm_rep_handler execution occurs after the above
connection check (specifically if it gets to the point where it acquires
priv->lock to dequeue pending skb's) but before the below code snippet in
start_xmit where packets are queued.

       if (skb_queue_len(&neigh->queue) < IPOIB_MAX_PATH_REC_QUEUE) {
               push_pseudo_header(skb, phdr->hwaddr);
               spin_lock_irqsave(&priv->lock, flags);
               __skb_queue_tail(&neigh->queue, skb);
               spin_unlock_irqrestore(&priv->lock, flags);
       } else {
               ++dev->stats.tx_dropped;
               dev_kfree_skb_any(skb);
       }

The patch acquires the netif tx lock in cm_rep_handler for the section
where it sets the connection up and dequeues and retransmits deferred
skb's.

Fixes: 839fcaba355a ("IPoIB: Connected mode experimental support")
Cc: stable@vger.kernel.org
Signed-off-by: Aaron Knister 
Tested-by: Ira Weiny 
Reviewed-by: Ira Weiny 
Signed-off-by: Jason Gunthorpe 
Signed-off-by: Greg Kroah-Hartman

IB/ipoib: Fix for potential no-carrier state

2018-05-30T05:47:31+00:00

[ Upstream commit 1029361084d18cc270f64dfd39529fafa10cfe01 ]

On reboot SM can program port pkey table before ipoib registered its
event handler, which could result in missing pkey event and leave root
interface with initial pkey value from index 0.

Since OPA port starts with invalid pkey in index 0, root interface will
fail to initialize and stay down with no-carrier flag.

For IB ipoib interface may end up with pkey different from value
opensm put in pkey table idx 0, resulting in connectivity issues
(different mcast groups, for example).

Close the window by calling event handler after registration
to make sure ipoib pkey is in sync with port pkey table.

Reviewed-by: Mike Marciniszyn 
Reviewed-by: Ira Weiny 
Signed-off-by: Alex Estrin 
Signed-off-by: Dennis Dalessandro 
Signed-off-by: Jason Gunthorpe 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

IB/ipoib: Avoid memory leak if the SA returns a different DGID

2018-03-24T09:57:35+00:00

[ Upstream commit 439000892ee17a9c92f1e4297818790ef8bb4ced ]

The ipoib path database is organized around DGIDs from the LLADDR, but the
SA is free to return a different GID when asked for path. This causes a
bug because the SA's modified DGID is copied into the database key, even
though it is no longer the correct lookup key, causing a memory leak and
other malfunctions.

Ensure the database key does not change after the SA query completes.

Demonstration of the bug is as  follows
ipoib wants to send to GID fe80:0000:0000:0000:0002:c903:00ef:5ee2, it
creates new record in the DB with that gid as a key, and issues a new
request to the SM.
Now, the SM from some reason returns path-record with other SGID (for
example, 2001:0000:0000:0000:0002:c903:00ef:5ee2 that contains the local
subnet prefix) now ipoib will overwrite the current entry with the new
one, and if new request to the original GID arrives ipoib  will not find
it in the DB (was overwritten) and will create new record that in its
turn will also be overwritten by the response from the SM, and so on
till the driver eats all the device memory.

Signed-off-by: Erez Shitrit 
Signed-off-by: Leon Romanovsky 
Signed-off-by: Jason Gunthorpe 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

IB/ipoib: Update broadcast object if PKey value was changed in index 0

2018-03-24T09:57:33+00:00

[ Upstream commit 9a9b8112699d78e7f317019b37f377e90023f3ed ]

Update the broadcast address in the priv->broadcast object when the
Pkey value changes in index 0, otherwise the multicast GID value will
keep the previous value of the PKey, and will not be updated.
This leads to interface state down because the interface will keep the
old PKey value.

For example, in SR-IOV environment, if the PF changes the value of PKey
index 0 for one of the VFs, then the VF receives PKey change event that
triggers heavy flush. This flush calls update_parent_pkey that update the
broadcast object and its relevant members. If in this case the multicast
GID will not be updated, the interface state will be down.

Fixes: c2904141696e ("IPoIB: Fix pkey change flow for virtualization environments")
Signed-off-by: Feras Daoud 
Signed-off-by: Erez Shitrit 
Reviewed-by: Alex Vesker 
Signed-off-by: Leon Romanovsky 
Signed-off-by: Doug Ledford 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

IB/ipoib: Change list_del to list_del_init in the tx object

2017-11-15T09:04:12+00:00

[ Upstream commit 27d41d29c7f093f6f77843624fbb080c1b4a8b9c ]

Since ipoib_cm_tx_start function and ipoib_cm_tx_reap function
belong to different work queues, they can run in parallel.
In this case if ipoib_cm_tx_reap calls list_del and release the
lock, ipoib_cm_tx_start may acquire it and call list_del_init
on the already deleted object.
Changing list_del to list_del_init in ipoib_cm_tx_reap fixes the problem.

Fixes: 839fcaba355a ("IPoIB: Connected mode experimental support")
Signed-off-by: Feras Daoud 
Signed-off-by: Erez Shitrit 
Reviewed-by: Alex Vesker 
Signed-off-by: Leon Romanovsky 
Reviewed-by: Yuval Shaia 
Signed-off-by: Doug Ledford 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

IB/ipoib: Replace list_del of the neigh->list with list_del_init

2017-10-08T08:11:19+00:00

[ Upstream commit c586071d1dc8227a7182179b8e50ee92cc43f6d2 ]

In order to resolve a situation where a few process delete
the same list element in sequence and cause panic, list_del
is replaced with list_del_init. In this case if the first
process that calls list_del releases the lock before acquiring
it again, other processes who can acquire the lock will call
list_del_init.

Fixes: b63b70d87741 ("IPoIB: Use a private hash table for path lookup")
Signed-off-by: Feras Daoud 
Signed-off-by: Erez Shitrit 
Reviewed-by: Alex Vesker 
Signed-off-by: Leon Romanovsky 
Reviewed-by: Yuval Shaia 
Signed-off-by: Doug Ledford 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

IB/ipoib: rtnl_unlock can not come after free_netdev

2017-10-08T08:11:19+00:00

[ Upstream commit 89a3987ab7a923c047c6dec008e60ad6f41fac22 ]

The ipoib_vlan_add function calls rtnl_unlock after free_netdev,
rtnl_unlock not only releases the lock, but also calls netdev_run_todo.
The latter function browses the net_todo_list array and completes the
unregistration of all its net_device instances. If we call free_netdev
before rtnl_unlock, then netdev_run_todo call over the freed device causes
panic.
To fix, move rtnl_unlock call before free_netdev call.

Fixes: 9baa0b036410 ("IB/ipoib: Add rtnl_link_ops support")
Cc: Or Gerlitz 
Signed-off-by: Feras Daoud 
Signed-off-by: Erez Shitrit 
Reviewed-by: Yuval Shaia 
Signed-off-by: Leon Romanovsky 
Signed-off-by: Doug Ledford 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

IB/ipoib: Fix deadlock over vlan_mutex

2017-10-08T08:11:19+00:00

[ Upstream commit 1c3098cdb05207e740715857df7b0998e372f527 ]

This patch fixes Deadlock while executing ipoib_vlan_delete.

The function takes the vlan_rwsem semaphore and calls
unregister_netdevice. The later function calls
ipoib_mcast_stop_thread that cause workqueue flush.

When the queue has one of the ipoib_ib_dev_flush_xxx events,
a deadlock occur because these events also tries to catch the
same vlan_rwsem semaphore.

To fix, unregister_netdevice should be called after releasing
the semaphore.

Fixes: cbbe1efa4972 ("IPoIB: Fix deadlock between ipoib_open() and child interface create")
Signed-off-by: Feras Daoud 
Signed-off-by: Erez Shitrit 
Reviewed-by: Alex Vesker 
Signed-off-by: Leon Romanovsky 
Signed-off-by: Doug Ledford 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

IB/IPoIB: ibX: failed to create mcg debug file

2017-05-20T12:18:41+00:00

commit 771a52584096c45e4565e8aabb596eece9d73d61 upstream.

When udev renames the netdev devices, ipoib debugfs entries does not
get renamed. As a result, if subsequent probe of ipoib device reuse the
name then creating a debugfs entry for the new device would fail.

Also, moved ipoib_create_debug_files and ipoib_delete_debug_files as part
of ipoib event handling in order to avoid any race condition between these.

Fixes: 1732b0ef3b3a ([IPoIB] add path record information in debugfs)
Signed-off-by: Vijay Kumar 
Signed-off-by: Shamir Rabinovitch 
Reviewed-by: Mark Bloch 
Signed-off-by: Doug Ledford 
Signed-off-by: Greg Kroah-Hartman

IB/ipoib: Fix deadlock between rmmod and set_mode

2017-04-18T05:55:49+00:00

commit 0a0007f28304cb9fc87809c86abb80ec71317f20 upstream.

When calling set_mode from sys/fs, the call flow locks the sys/fs lock
first and then tries to lock rtnl_lock (when calling ipoib_set_mod).
On the other hand, the rmmod call flow takes the rtnl_lock first
(when calling unregister_netdev) and then tries to take the sys/fs
lock. Deadlock a->b, b->a.

The problem starts when ipoib_set_mod frees it's rtnl_lck and tries
to get it after that.

    set_mod:
    [] ? check_preempt_curr+0x6d/0x90
    [] __mutex_lock_slowpath+0x13e/0x180
    [] ? __rtnl_unlock+0x15/0x20
    [] mutex_lock+0x2b/0x50
    [] rtnl_lock+0x15/0x20
    [] ipoib_set_mode+0x97/0x160 [ib_ipoib]
    [] set_mode+0x3b/0x80 [ib_ipoib]
    [] dev_attr_store+0x20/0x30
    [] sysfs_write_file+0xe5/0x170
    [] vfs_write+0xb8/0x1a0
    [] sys_write+0x51/0x90
    [] system_call_fastpath+0x16/0x1b

    rmmod:
    [] ? put_dec+0x10c/0x110
    [] ? number+0x2ee/0x320
    [] schedule_timeout+0x215/0x2e0
    [] ? vsnprintf+0x484/0x5f0
    [] ? string+0x40/0x100
    [] wait_for_common+0x123/0x180
    [] ? default_wake_function+0x0/0x20
    [] ? ifind_fast+0x5e/0xb0
    [] wait_for_completion+0x1d/0x20
    [] sysfs_addrm_finish+0x228/0x270
    [] sysfs_remove_dir+0xa3/0xf0
    [] kobject_del+0x16/0x40
    [] device_del+0x184/0x1e0
    [] netdev_unregister_kobject+0xab/0xc0
    [] rollback_registered+0xae/0x130
    [] unregister_netdevice+0x22/0x70
    [] unregister_netdev+0x1e/0x30
    [] ipoib_remove_one+0xe0/0x120 [ib_ipoib]
    [] ib_unregister_device+0x4f/0x100 [ib_core]
    [] mlx4_ib_remove+0x41/0x180 [mlx4_ib]
    [] mlx4_remove_device+0x71/0x90 [mlx4_core]

Fixes: 862096a8bbf8 ("IB/ipoib: Add more rtnl_link_ops callbacks")
Cc: Or Gerlitz 
Signed-off-by: Feras Daoud 
Signed-off-by: Erez Shitrit 
Signed-off-by: Leon Romanovsky 
Signed-off-by: Doug Ledford 
Signed-off-by: Greg Kroah-Hartman