[ Upstream commit f32f19339596b214c208c0dba716f4b6cc4f6958 ]
syzbot managed to crash a host by creating a bond
with a GRE device.
For non Ethernet device, bonding calls bond_setup_by_slave()
instead of ether_setup(), and unfortunately dev->needed_headroom
was not copied from the new added member.
[ 171.243095] skbuff: skb_under_panic: text:ffffffffa184b9ea len:116 put:20 head:ffff883f84012dc0 data:ffff883f84012dbc tail:0x70 end:0xd00 dev:bond0
[ 171.243111] ------------[ cut here ]------------
[ 171.243112] kernel BUG at net/core/skbuff.c:112!
[ 171.243117] invalid opcode: 0000 [#1] SMP KASAN PTI
[ 171.243469] gsmi: Log Shutdown Reason 0x03
[ 171.243505] Call Trace:
[ 171.243506] <IRQ>
[ 171.243512] [<ffffffffa171be59>] skb_push+0x49/0x50
[ 171.243516] [<ffffffffa184b9ea>] ipgre_header+0x2a/0xf0
[ 171.243520] [<ffffffffa17452d7>] neigh_connected_output+0xb7/0x100
[ 171.243524] [<ffffffffa186f1d3>] ip6_finish_output2+0x383/0x490
[ 171.243528] [<ffffffffa186ede2>] __ip6_finish_output+0xa2/0x110
[ 171.243531] [<ffffffffa186acbc>] ip6_finish_output+0x2c/0xa0
[ 171.243534] [<ffffffffa186abe9>] ip6_output+0x69/0x110
[ 171.243537] [<ffffffffa186ac90>] ? ip6_output+0x110/0x110
[ 171.243541] [<ffffffffa189d952>] mld_sendpack+0x1b2/0x2d0
[ 171.243544] [<ffffffffa189d290>] ? mld_send_report+0xf0/0xf0
[ 171.243548] [<ffffffffa189c797>] mld_ifc_timer_expire+0x2d7/0x3b0
[ 171.243551] [<ffffffffa189c4c0>] ? mld_gq_timer_expire+0x50/0x50
[ 171.243556] [<ffffffffa0fea270>] call_timer_fn+0x30/0x130
[ 171.243559] [<ffffffffa0fea17c>] expire_timers+0x4c/0x110
[ 171.243563] [<ffffffffa0fea0e3>] __run_timers+0x213/0x260
[ 171.243566] [<ffffffffa0fecb7d>] ? ktime_get+0x3d/0xa0
[ 171.243570] [<ffffffffa0ff9c4e>] ? clockevents_program_event+0x7e/0xe0
[ 171.243574] [<ffffffffa0f7e5d5>] ? sched_clock_cpu+0x15/0x190
[ 171.243577] [<ffffffffa0fe973d>] run_timer_softirq+0x1d/0x40
[ 171.243581] [<ffffffffa1c00152>] __do_softirq+0x152/0x2f0
[ 171.243585] [<ffffffffa0f44e1f>] irq_exit+0x9f/0xb0
[ 171.243588] [<ffffffffa1a02e1d>] smp_apic_timer_interrupt+0xfd/0x1a0
[ 171.243591] [<ffffffffa1a01ea6>] apic_timer_interrupt+0x86/0x90
Fixes: f5184d267c1a ("net: Allow netdevices to specify needed head/tailroom")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Reported-by: syzbot <syzkaller@googlegroups.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit 0410d07190961ac526f05085765a8d04d926545b ]
When the ARP monitor is used for link detection, ARP replies are
validated for all slaves (arp_validate=3) and fail_over_mac is set to
active, two slaves of an active-backup bond may get stuck in a state
where both of them are active and pass packets that they receive to
the bond. This state makes IPv6 duplicate address detection fail. The
state is reached thus:
1. The current active slave goes down because the ARP target
is not reachable.
2. The current ARP slave is chosen and made active.
3. A new slave is enslaved. This new slave becomes the current active
slave and can reach the ARP target.
As a result, the current ARP slave stays active after the enslave
action has finished and the log is littered with "PROBE BAD" messages:
> bond0: PROBE: c_arp ens10 && cas ens11 BAD
The workaround is to remove the slave with "going back" status from
the bond and re-enslave it. This issue was encountered when DPDK PMD
interfaces were being enslaved to an active-backup bond.
I would be possible to fix the issue in bond_enslave() or
bond_change_active_slave() but the ARP monitor was fixed instead to
keep most of the actions changing the current ARP slave in the ARP
monitor code. The current ARP slave is set as inactive and backup
during the commit phase. A new state, BOND_LINK_FAIL, has been
introduced for slaves in the context of the ARP monitor. This allows
administrators to see how slaves are rotated for sending ARP requests
and attempts are made to find a new active slave.
Fixes: b2220cad583c9 ("bonding: refactor ARP active-backup monitor")
Signed-off-by: Jiri Wiesner <jwiesner@suse.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit 832707021666411d04795c564a4adea5d6b94f17 ]
When we tear down a network namespace, we unregister all
the netdevices within it. So we may queue a slave device
and a bonding device together in the same unregister queue.
If the only slave device is non-ethernet, it would
automatically unregister the bonding device as well. Thus,
we may end up unregistering the bonding device twice.
Workaround this special case by checking reg_state.
Fixes: 9b5e383c11b0 ("net: Introduce unregister_netdevice_many()")
Reported-by: syzbot+af23e7f3e0a7e10c8b67@syzkaller.appspotmail.com
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Andy Gospodarek <andy@greyhouse.net>
Cc: Jay Vosburgh <j.vosburgh@gmail.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit 4ca0d9ac3fd8f9f90b72a15d8da2aca3ffb58418 ]
Broadcast mode bonds transmit a copy of all traffic simultaneously out of
all interfaces, so the "speed" of the bond isn't really the aggregate of
all interfaces, but rather, the speed of the slowest active interface.
Also, the type of the speed field is u32, not unsigned long, so adjust
that accordingly, as required to make min() function here without
complaining about mismatching types.
Fixes: bb5b052f751b ("bond: add support to read speed and duplex via ethtool")
CC: Jay Vosburgh <j.vosburgh@gmail.com>
CC: Veaceslav Falico <vfalico@gmail.com>
CC: Andy Gospodarek <andy@greyhouse.net>
CC: "David S. Miller" <davem@davemloft.net>
CC: netdev@vger.kernel.org
Acked-by: Jay Vosburgh <jay.vosburgh@canonical.com>
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit c75d1d5248c0c97996051809ad0e9f154ba5d76e ]
Very similar to commit 544f287b8495
("bonding: check error value of register_netdevice() immediately"),
we should immediately check the return value of register_netdevice()
before doing anything else.
Fixes: 005db31d5f5f ("bonding: set carrier off for devices created through netlink")
Reported-and-tested-by: syzbot+bbc3a11c4da63c1b74d6@syzkaller.appspotmail.com
Cc: Beniamino Galvani <bgalvani@redhat.com>
Cc: Taehee Yoo <ap420073@gmail.com>
Cc: Jay Vosburgh <j.vosburgh@gmail.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit 544f287b84959203367cd29e16e772717612fab4 ]
If register_netdevice() is failed, net_device should not be used
because variables are uninitialized or freed.
So, the routine should be stopped immediately.
But, bond_create() doesn't check return value of register_netdevice()
immediately. That will result in a panic because of using uninitialized
or freed memory.
Test commands:
modprobe netdev-notifier-error-inject
echo -22 > /sys/kernel/debug/notifier-error-inject/netdev/\
actions/NETDEV_REGISTER/error
modprobe bonding max_bonds=3
Splat looks like:
[ 375.028492][ T193] general protection fault, probably for non-canonical address 0x6b6b6b6b6b6b6b6b: 0000 [#1] SMP DEBUG_PAGEALLOC PTI
[ 375.033207][ T193] CPU: 2 PID: 193 Comm: kworker/2:2 Not tainted 5.8.0-rc4+ #645
[ 375.036068][ T193] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1ubuntu1 04/01/2014
[ 375.039673][ T193] Workqueue: events linkwatch_event
[ 375.041557][ T193] RIP: 0010:dev_activate+0x4a/0x340
[ 375.043381][ T193] Code: 40 a8 04 0f 85 db 00 00 00 8b 83 08 04 00 00 85 c0 0f 84 0d 01 00 00 31 d2 89 d0 48 8d 04 40 48 c1 e0 07 48 03 83 00 04 00 00 <48> 8b 48 10 f6 41 10 01 75 08 f0 80 a1 a0 01 00 00 fd 48 89 48 08
[ 375.050267][ T193] RSP: 0018:ffff9f8facfcfdd8 EFLAGS: 00010202
[ 375.052410][ T193] RAX: 6b6b6b6b6b6b6b6b RBX: ffff9f8fae6ea000 RCX: 0000000000000006
[ 375.055178][ T193] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9f8fae6ea000
[ 375.057762][ T193] RBP: ffff9f8fae6ea000 R08: 0000000000000000 R09: 0000000000000000
[ 375.059810][ T193] R10: 0000000000000001 R11: 0000000000000000 R12: ffff9f8facfcfe08
[ 375.061892][ T193] R13: ffffffff883587e0 R14: 0000000000000000 R15: ffff9f8fae6ea580
[ 375.063931][ T193] FS: 0000000000000000(0000) GS:ffff9f8fbae00000(0000) knlGS:0000000000000000
[ 375.066239][ T193] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 375.067841][ T193] CR2: 00007f2f542167a0 CR3: 000000012cee6002 CR4: 00000000003606e0
[ 375.069657][ T193] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 375.071471][ T193] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 375.073269][ T193] Call Trace:
[ 375.074005][ T193] linkwatch_do_dev+0x4d/0x50
[ 375.075052][ T193] __linkwatch_run_queue+0x10b/0x200
[ 375.076244][ T193] linkwatch_event+0x21/0x30
[ 375.077274][ T193] process_one_work+0x252/0x600
[ 375.078379][ T193] ? process_one_work+0x600/0x600
[ 375.079518][ T193] worker_thread+0x3c/0x380
[ 375.080534][ T193] ? process_one_work+0x600/0x600
[ 375.081668][ T193] kthread+0x139/0x150
[ 375.082567][ T193] ? kthread_park+0x90/0x90
[ 375.083567][ T193] ret_from_fork+0x22/0x30
Fixes: e826eafa65c6 ("bonding: Call netif_carrier_off after register_netdevice")
Signed-off-by: Taehee Yoo <ap420073@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
commit a068aab42258e25094bc2c159948d263ed7d7a77 upstream.
kobject_init_and_add() takes reference even when it fails.
If this function returns an error, kobject_put() must be called to
properly clean up the memory associated with the object. Previous
commit "b8eb718348b8" fixed a similar problem.
Fixes: 07699f9a7c8d ("bonding: add sysfs /slave dir for bond slave devices.")
Signed-off-by: Qiushi Wu <wu000273@umn.edu>
Acked-by: Jay Vosburgh <jay.vosburgh@canonical.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
commit b7469e83d2add567e4e0b063963db185f3167cea upstream.
Similar to commit 38f88c454042 ("bonding/alb: properly access headers
in bond_alb_xmit()"), we need to make sure arp header was pulled
in skb->head before blindly accessing it in rlb_arp_xmit().
Remove arp_pkt() private helper, since it is more readable/obvious
to have the following construct back to back :
if (!pskb_network_may_pull(skb, sizeof(*arp)))
return NULL;
arp = (struct arp_pkt *)skb_network_header(skb);
syzbot reported :
BUG: KMSAN: uninit-value in bond_slave_has_mac_rx include/net/bonding.h:704 [inline]
BUG: KMSAN: uninit-value in rlb_arp_xmit drivers/net/bonding/bond_alb.c:662 [inline]
BUG: KMSAN: uninit-value in bond_alb_xmit+0x575/0x25e0 drivers/net/bonding/bond_alb.c:1477
CPU: 0 PID: 12743 Comm: syz-executor.4 Not tainted 5.6.0-rc2-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1c9/0x220 lib/dump_stack.c:118
kmsan_report+0xf7/0x1e0 mm/kmsan/kmsan_report.c:118
__msan_warning+0x58/0xa0 mm/kmsan/kmsan_instr.c:215
bond_slave_has_mac_rx include/net/bonding.h:704 [inline]
rlb_arp_xmit drivers/net/bonding/bond_alb.c:662 [inline]
bond_alb_xmit+0x575/0x25e0 drivers/net/bonding/bond_alb.c:1477
__bond_start_xmit drivers/net/bonding/bond_main.c:4257 [inline]
bond_start_xmit+0x85d/0x2f70 drivers/net/bonding/bond_main.c:4282
__netdev_start_xmit include/linux/netdevice.h:4524 [inline]
netdev_start_xmit include/linux/netdevice.h:4538 [inline]
xmit_one net/core/dev.c:3470 [inline]
dev_hard_start_xmit+0x531/0xab0 net/core/dev.c:3486
__dev_queue_xmit+0x37de/0x4220 net/core/dev.c:4063
dev_queue_xmit+0x4b/0x60 net/core/dev.c:4096
packet_snd net/packet/af_packet.c:2967 [inline]
packet_sendmsg+0x8347/0x93b0 net/packet/af_packet.c:2992
sock_sendmsg_nosec net/socket.c:652 [inline]
sock_sendmsg net/socket.c:672 [inline]
__sys_sendto+0xc1b/0xc50 net/socket.c:1998
__do_sys_sendto net/socket.c:2010 [inline]
__se_sys_sendto+0x107/0x130 net/socket.c:2006
__x64_sys_sendto+0x6e/0x90 net/socket.c:2006
do_syscall_64+0xb8/0x160 arch/x86/entry/common.c:296
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x45c479
Code: ad b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 7b b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007fc77ffbbc78 EFLAGS: 00000246 ORIG_RAX: 000000000000002c
RAX: ffffffffffffffda RBX: 00007fc77ffbc6d4 RCX: 000000000045c479
RDX: 000000000000000e RSI: 00000000200004c0 RDI: 0000000000000003
RBP: 000000000076bf20 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 0000000000000a04 R14: 00000000004cc7b0 R15: 000000000076bf2c
Uninit was created at:
kmsan_save_stack_with_flags mm/kmsan/kmsan.c:144 [inline]
kmsan_internal_poison_shadow+0x66/0xd0 mm/kmsan/kmsan.c:127
kmsan_slab_alloc+0x8a/0xe0 mm/kmsan/kmsan_hooks.c:82
slab_alloc_node mm/slub.c:2793 [inline]
__kmalloc_node_track_caller+0xb40/0x1200 mm/slub.c:4401
__kmalloc_reserve net/core/skbuff.c:142 [inline]
__alloc_skb+0x2fd/0xac0 net/core/skbuff.c:210
alloc_skb include/linux/skbuff.h:1051 [inline]
alloc_skb_with_frags+0x18c/0xa70 net/core/skbuff.c:5766
sock_alloc_send_pskb+0xada/0xc60 net/core/sock.c:2242
packet_alloc_skb net/packet/af_packet.c:2815 [inline]
packet_snd net/packet/af_packet.c:2910 [inline]
packet_sendmsg+0x66a0/0x93b0 net/packet/af_packet.c:2992
sock_sendmsg_nosec net/socket.c:652 [inline]
sock_sendmsg net/socket.c:672 [inline]
__sys_sendto+0xc1b/0xc50 net/socket.c:1998
__do_sys_sendto net/socket.c:2010 [inline]
__se_sys_sendto+0x107/0x130 net/socket.c:2006
__x64_sys_sendto+0x6e/0x90 net/socket.c:2006
do_syscall_64+0xb8/0x160 arch/x86/entry/common.c:296
entry_SYSCALL_64_after_hwframe+0x44/0xa9
Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Reported-by: syzbot <syzkaller@googlegroups.com>
Cc: Jay Vosburgh <j.vosburgh@gmail.com>
Cc: Veaceslav Falico <vfalico@gmail.com>
Cc: Andy Gospodarek <andy@greyhouse.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
[ Upstream commit b3e80d44f5b1b470dd9e2dbc6816e63a5c519709 ]
In the "struct bonding", there is stats_lock.
This lock protects "bond_stats" in the "struct bonding".
bond_stats is updated in the bond_get_stats() and this function would be
executed concurrently. So, the lock is needed.
Bonding interfaces would be nested.
So, either stats_lock should use dynamic lockdep class key or stats_lock
should be used by spin_lock_nested(). In the current code, stats_lock is
using a dynamic lockdep class key.
But there is no updating stats_lock_key routine So, lockdep warning
will occur.
Test commands:
ip link add bond0 type bond
ip link add bond1 type bond
ip link set bond0 master bond1
ip link set bond0 nomaster
ip link set bond1 master bond0
Splat looks like:
[ 38.420603][ T957] 5.5.0+ #394 Not tainted
[ 38.421074][ T957] ------------------------------------------------------
[ 38.421837][ T957] ip/957 is trying to acquire lock:
[ 38.422399][ T957] ffff888063262cd8 (&bond->stats_lock_key#2){+.+.}, at: bond_get_stats+0x90/0x4d0 [bonding]
[ 38.423528][ T957]
[ 38.423528][ T957] but task is already holding lock:
[ 38.424526][ T957] ffff888065fd2cd8 (&bond->stats_lock_key){+.+.}, at: bond_get_stats+0x90/0x4d0 [bonding]
[ 38.426075][ T957]
[ 38.426075][ T957] which lock already depends on the new lock.
[ 38.426075][ T957]
[ 38.428536][ T957]
[ 38.428536][ T957] the existing dependency chain (in reverse order) is:
[ 38.429475][ T957]
[ 38.429475][ T957] -> #1 (&bond->stats_lock_key){+.+.}:
[ 38.430273][ T957] _raw_spin_lock+0x30/0x70
[ 38.430812][ T957] bond_get_stats+0x90/0x4d0 [bonding]
[ 38.431451][ T957] dev_get_stats+0x1ec/0x270
[ 38.432088][ T957] bond_get_stats+0x1a5/0x4d0 [bonding]
[ 38.432767][ T957] dev_get_stats+0x1ec/0x270
[ 38.433322][ T957] rtnl_fill_stats+0x44/0xbe0
[ 38.433866][ T957] rtnl_fill_ifinfo+0xeb2/0x3720
[ 38.434474][ T957] rtmsg_ifinfo_build_skb+0xca/0x170
[ 38.435081][ T957] rtmsg_ifinfo_event.part.33+0x1b/0xb0
[ 38.436848][ T957] rtnetlink_event+0xcd/0x120
[ 38.437455][ T957] notifier_call_chain+0x90/0x160
[ 38.438067][ T957] netdev_change_features+0x74/0xa0
[ 38.438708][ T957] bond_compute_features.isra.45+0x4e6/0x6f0 [bonding]
[ 38.439522][ T957] bond_enslave+0x3639/0x47b0 [bonding]
[ 38.440225][ T957] do_setlink+0xaab/0x2ef0
[ 38.440786][ T957] __rtnl_newlink+0x9c5/0x1270
[ 38.441463][ T957] rtnl_newlink+0x65/0x90
[ 38.442075][ T957] rtnetlink_rcv_msg+0x4a8/0x890
[ 38.442774][ T957] netlink_rcv_skb+0x121/0x350
[ 38.443451][ T957] netlink_unicast+0x42e/0x610
[ 38.444282][ T957] netlink_sendmsg+0x65a/0xb90
[ 38.444992][ T957] ____sys_sendmsg+0x5ce/0x7a0
[ 38.445679][ T957] ___sys_sendmsg+0x10f/0x1b0
[ 38.446365][ T957] __sys_sendmsg+0xc6/0x150
[ 38.447007][ T957] do_syscall_64+0x99/0x4f0
[ 38.447668][ T957] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 38.448538][ T957]
[ 38.448538][ T957] -> #0 (&bond->stats_lock_key#2){+.+.}:
[ 38.449554][ T957] __lock_acquire+0x2d8d/0x3de0
[ 38.450148][ T957] lock_acquire+0x164/0x3b0
[ 38.450711][ T957] _raw_spin_lock+0x30/0x70
[ 38.451292][ T957] bond_get_stats+0x90/0x4d0 [bonding]
[ 38.451950][ T957] dev_get_stats+0x1ec/0x270
[ 38.452425][ T957] bond_get_stats+0x1a5/0x4d0 [bonding]
[ 38.453362][ T957] dev_get_stats+0x1ec/0x270
[ 38.453825][ T957] rtnl_fill_stats+0x44/0xbe0
[ 38.454390][ T957] rtnl_fill_ifinfo+0xeb2/0x3720
[ 38.456257][ T957] rtmsg_ifinfo_build_skb+0xca/0x170
[ 38.456998][ T957] rtmsg_ifinfo_event.part.33+0x1b/0xb0
[ 38.459351][ T957] rtnetlink_event+0xcd/0x120
[ 38.460086][ T957] notifier_call_chain+0x90/0x160
[ 38.460829][ T957] netdev_change_features+0x74/0xa0
[ 38.461752][ T957] bond_compute_features.isra.45+0x4e6/0x6f0 [bonding]
[ 38.462705][ T957] bond_enslave+0x3639/0x47b0 [bonding]
[ 38.463476][ T957] do_setlink+0xaab/0x2ef0
[ 38.464141][ T957] __rtnl_newlink+0x9c5/0x1270
[ 38.464897][ T957] rtnl_newlink+0x65/0x90
[ 38.465522][ T957] rtnetlink_rcv_msg+0x4a8/0x890
[ 38.466215][ T957] netlink_rcv_skb+0x121/0x350
[ 38.466895][ T957] netlink_unicast+0x42e/0x610
[ 38.467583][ T957] netlink_sendmsg+0x65a/0xb90
[ 38.468285][ T957] ____sys_sendmsg+0x5ce/0x7a0
[ 38.469202][ T957] ___sys_sendmsg+0x10f/0x1b0
[ 38.469884][ T957] __sys_sendmsg+0xc6/0x150
[ 38.470587][ T957] do_syscall_64+0x99/0x4f0
[ 38.471245][ T957] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 38.472093][ T957]
[ 38.472093][ T957] other info that might help us debug this:
[ 38.472093][ T957]
[ 38.473438][ T957] Possible unsafe locking scenario:
[ 38.473438][ T957]
[ 38.474898][ T957] CPU0 CPU1
[ 38.476234][ T957] ---- ----
[ 38.480171][ T957] lock(&bond->stats_lock_key);
[ 38.480808][ T957] lock(&bond->stats_lock_key#2);
[ 38.481791][ T957] lock(&bond->stats_lock_key);
[ 38.482754][ T957] lock(&bond->stats_lock_key#2);
[ 38.483416][ T957]
[ 38.483416][ T957] *** DEADLOCK ***
[ 38.483416][ T957]
[ 38.484505][ T957] 3 locks held by ip/957:
[ 38.485048][ T957] #0: ffffffffbccf6230 (rtnl_mutex){+.+.}, at: rtnetlink_rcv_msg+0x457/0x890
[ 38.486198][ T957] #1: ffff888065fd2cd8 (&bond->stats_lock_key){+.+.}, at: bond_get_stats+0x90/0x4d0 [bonding]
[ 38.487625][ T957] #2: ffffffffbc9254c0 (rcu_read_lock){....}, at: bond_get_stats+0x5/0x4d0 [bonding]
[ 38.488897][ T957]
[ 38.488897][ T957] stack backtrace:
[ 38.489646][ T957] CPU: 1 PID: 957 Comm: ip Not tainted 5.5.0+ #394
[ 38.490497][ T957] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[ 38.492810][ T957] Call Trace:
[ 38.493219][ T957] dump_stack+0x96/0xdb
[ 38.493709][ T957] check_noncircular+0x371/0x450
[ 38.494344][ T957] ? lookup_address+0x60/0x60
[ 38.494923][ T957] ? print_circular_bug.isra.35+0x310/0x310
[ 38.495699][ T957] ? hlock_class+0x130/0x130
[ 38.496334][ T957] ? __lock_acquire+0x2d8d/0x3de0
[ 38.496979][ T957] __lock_acquire+0x2d8d/0x3de0
[ 38.497607][ T957] ? register_lock_class+0x14d0/0x14d0
[ 38.498333][ T957] ? check_chain_key+0x236/0x5d0
[ 38.499003][ T957] lock_acquire+0x164/0x3b0
[ 38.499800][ T957] ? bond_get_stats+0x90/0x4d0 [bonding]
[ 38.500706][ T957] _raw_spin_lock+0x30/0x70
[ 38.501435][ T957] ? bond_get_stats+0x90/0x4d0 [bonding]
[ 38.502311][ T957] bond_get_stats+0x90/0x4d0 [bonding]
[ ... ]
But, there is another problem.
The dynamic lockdep class key is protected by RTNL, but bond_get_stats()
would be called outside of RTNL.
So, it would use an invalid dynamic lockdep class key.
In order to fix this issue, stats_lock uses spin_lock_nested() instead of
a dynamic lockdep key.
The bond_get_stats() calls bond_get_lowest_level_rcu() to get the correct
nest level value, which will be used by spin_lock_nested().
The "dev->lower_level" indicates lower nest level value, but this value
is invalid outside of RTNL.
So, bond_get_lowest_level_rcu() returns valid lower nest level value in
the RCU critical section.
bond_get_lowest_level_rcu() will be work only when LOCKDEP is enabled.
Fixes: 089bca2caed0 ("bonding: use dynamic lockdep key instead of subclass")
Signed-off-by: Taehee Yoo <ap420073@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
[ Upstream commit 064ff66e2bef84f1153087612032b5b9eab005bd ]
After bond_release(), netdev_update_lockdep_key() should be called.
But both ioctl path and attribute path don't call
netdev_update_lockdep_key().
This patch adds missing netdev_update_lockdep_key().
Test commands:
ip link add bond0 type bond
ip link add bond1 type bond
ifenslave bond0 bond1
ifenslave -d bond0 bond1
ifenslave bond1 bond0
Splat looks like:
[ 29.501182][ T1046] WARNING: possible circular locking dependency detected
[ 29.501945][ T1039] hardirqs last disabled at (1962): [<ffffffffac6c807f>] handle_mm_fault+0x13f/0x700
[ 29.503442][ T1046] 5.5.0+ #322 Not tainted
[ 29.503447][ T1046] ------------------------------------------------------
[ 29.504277][ T1039] softirqs last enabled at (1180): [<ffffffffade00678>] __do_softirq+0x678/0x981
[ 29.505443][ T1046] ifenslave/1046 is trying to acquire lock:
[ 29.505886][ T1039] softirqs last disabled at (1169): [<ffffffffac19c18a>] irq_exit+0x17a/0x1a0
[ 29.509997][ T1046] ffff88805d5da280 (&dev->addr_list_lock_key#3){+...}, at: dev_mc_sync_multiple+0x95/0x120
[ 29.511243][ T1046]
[ 29.511243][ T1046] but task is already holding lock:
[ 29.512192][ T1046] ffff8880460f2280 (&dev->addr_list_lock_key#4){+...}, at: bond_enslave+0x4482/0x47b0 [bonding]
[ 29.514124][ T1046]
[ 29.514124][ T1046] which lock already depends on the new lock.
[ 29.514124][ T1046]
[ 29.517297][ T1046]
[ 29.517297][ T1046] the existing dependency chain (in reverse order) is:
[ 29.518231][ T1046]
[ 29.518231][ T1046] -> #1 (&dev->addr_list_lock_key#4){+...}:
[ 29.519076][ T1046] _raw_spin_lock+0x30/0x70
[ 29.519588][ T1046] dev_mc_sync_multiple+0x95/0x120
[ 29.520208][ T1046] bond_enslave+0x448d/0x47b0 [bonding]
[ 29.520862][ T1046] bond_option_slaves_set+0x1a3/0x370 [bonding]
[ 29.521640][ T1046] __bond_opt_set+0x1ff/0xbb0 [bonding]
[ 29.522438][ T1046] __bond_opt_set_notify+0x2b/0xf0 [bonding]
[ 29.523251][ T1046] bond_opt_tryset_rtnl+0x92/0xf0 [bonding]
[ 29.524082][ T1046] bonding_sysfs_store_option+0x8a/0xf0 [bonding]
[ 29.524959][ T1046] kernfs_fop_write+0x276/0x410
[ 29.525620][ T1046] vfs_write+0x197/0x4a0
[ 29.526218][ T1046] ksys_write+0x141/0x1d0
[ 29.526818][ T1046] do_syscall_64+0x99/0x4f0
[ 29.527430][ T1046] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 29.528265][ T1046]
[ 29.528265][ T1046] -> #0 (&dev->addr_list_lock_key#3){+...}:
[ 29.529272][ T1046] __lock_acquire+0x2d8d/0x3de0
[ 29.529935][ T1046] lock_acquire+0x164/0x3b0
[ 29.530638][ T1046] _raw_spin_lock+0x30/0x70
[ 29.531187][ T1046] dev_mc_sync_multiple+0x95/0x120
[ 29.531790][ T1046] bond_enslave+0x448d/0x47b0 [bonding]
[ 29.532451][ T1046] bond_option_slaves_set+0x1a3/0x370 [bonding]
[ 29.533163][ T1046] __bond_opt_set+0x1ff/0xbb0 [bonding]
[ 29.533789][ T1046] __bond_opt_set_notify+0x2b/0xf0 [bonding]
[ 29.534595][ T1046] bond_opt_tryset_rtnl+0x92/0xf0 [bonding]
[ 29.535500][ T1046] bonding_sysfs_store_option+0x8a/0xf0 [bonding]
[ 29.536379][ T1046] kernfs_fop_write+0x276/0x410
[ 29.537057][ T1046] vfs_write+0x197/0x4a0
[ 29.537640][ T1046] ksys_write+0x141/0x1d0
[ 29.538251][ T1046] do_syscall_64+0x99/0x4f0
[ 29.538870][ T1046] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 29.539659][ T1046]
[ 29.539659][ T1046] other info that might help us debug this:
[ 29.539659][ T1046]
[ 29.540953][ T1046] Possible unsafe locking scenario:
[ 29.540953][ T1046]
[ 29.541883][ T1046] CPU0 CPU1
[ 29.542540][ T1046] ---- ----
[ 29.543209][ T1046] lock(&dev->addr_list_lock_key#4);
[ 29.543880][ T1046] lock(&dev->addr_list_lock_key#3);
[ 29.544873][ T1046] lock(&dev->addr_list_lock_key#4);
[ 29.545863][ T1046] lock(&dev->addr_list_lock_key#3);
[ 29.546525][ T1046]
[ 29.546525][ T1046] *** DEADLOCK ***
[ 29.546525][ T1046]
[ 29.547542][ T1046] 5 locks held by ifenslave/1046:
[ 29.548196][ T1046] #0: ffff88806044c478 (sb_writers#5){.+.+}, at: vfs_write+0x3bb/0x4a0
[ 29.549248][ T1046] #1: ffff88805af00890 (&of->mutex){+.+.}, at: kernfs_fop_write+0x1cf/0x410
[ 29.550343][ T1046] #2: ffff88805b8b54b0 (kn->count#157){.+.+}, at: kernfs_fop_write+0x1f2/0x410
[ 29.551575][ T1046] #3: ffffffffaecf4cf0 (rtnl_mutex){+.+.}, at: bond_opt_tryset_rtnl+0x5f/0xf0 [bonding]
[ 29.552819][ T1046] #4: ffff8880460f2280 (&dev->addr_list_lock_key#4){+...}, at: bond_enslave+0x4482/0x47b0 [bonding]
[ 29.554175][ T1046]
[ 29.554175][ T1046] stack backtrace:
[ 29.554907][ T1046] CPU: 0 PID: 1046 Comm: ifenslave Not tainted 5.5.0+ #322
[ 29.555854][ T1046] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[ 29.557064][ T1046] Call Trace:
[ 29.557504][ T1046] dump_stack+0x96/0xdb
[ 29.558054][ T1046] check_noncircular+0x371/0x450
[ 29.558723][ T1046] ? print_circular_bug.isra.35+0x310/0x310
[ 29.559486][ T1046] ? hlock_class+0x130/0x130
[ 29.560100][ T1046] ? __lock_acquire+0x2d8d/0x3de0
[ 29.560761][ T1046] __lock_acquire+0x2d8d/0x3de0
[ 29.561366][ T1046] ? register_lock_class+0x14d0/0x14d0
[ 29.562045][ T1046] ? find_held_lock+0x39/0x1d0
[ 29.562641][ T1046] lock_acquire+0x164/0x3b0
[ 29.563199][ T1046] ? dev_mc_sync_multiple+0x95/0x120
[ 29.563872][ T1046] _raw_spin_lock+0x30/0x70
[ 29.564464][ T1046] ? dev_mc_sync_multiple+0x95/0x120
[ 29.565146][ T1046] dev_mc_sync_multiple+0x95/0x120
[ 29.565793][ T1046] bond_enslave+0x448d/0x47b0 [bonding]
[ 29.566487][ T1046] ? bond_update_slave_arr+0x940/0x940 [bonding]
[ 29.567279][ T1046] ? bstr_printf+0xc20/0xc20
[ 29.567857][ T1046] ? stack_trace_consume_entry+0x160/0x160
[ 29.568614][ T1046] ? deactivate_slab.isra.77+0x2c5/0x800
[ 29.569320][ T1046] ? check_chain_key+0x236/0x5d0
[ 29.569939][ T1046] ? sscanf+0x93/0xc0
[ 29.570442][ T1046] ? vsscanf+0x1e20/0x1e20
[ 29.571003][ T1046] bond_option_slaves_set+0x1a3/0x370 [bonding]
[ ... ]
Fixes: ab92d68fc22f ("net: core: add generic lockdep keys")
Signed-off-by: Taehee Yoo <ap420073@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>