linux-stable.git/net/xdp, branch linux-5.7.y

xdp: Fix xsk_generic_xmit errno

2020-06-24T15:49:11+00:00

[ Upstream commit aa2cad0600ed2ca6a0ab39948d4db1666b6c962b ]

Propagate sock_alloc_send_skb error code, not set it to
EAGAIN unconditionally, when fail to allocate skb, which
might cause that user space unnecessary loops.

Fixes: 35fcde7f8deb ("xsk: support for Tx")
Signed-off-by: Li RongQing 
Signed-off-by: Daniel Borkmann 
Acked-by: Björn Töpel 
Link: https://lore.kernel.org/bpf/1591852266-24017-1-git-send-email-lirongqing@baidu.com
Signed-off-by: Sasha Levin

xsk: Add overflow check for u64 division, stored into u32

2020-05-25T22:06:00+00:00

The npgs member of struct xdp_umem is an u32 entity, and stores the
number of pages the UMEM consumes. The calculation of npgs

  npgs = size / PAGE_SIZE

can overflow.

To avoid overflow scenarios, the division is now first stored in a
u64, and the result is verified to fit into 32b.

An alternative would be storing the npgs as a u64, however, this
wastes memory and is an unrealisticly large packet area.

Fixes: c0c77d8fb787 ("xsk: add user memory registration support sockopt")
Reported-by: "Minh Bùi Quang" 
Signed-off-by: Björn Töpel 
Signed-off-by: Daniel Borkmann 
Acked-by: Jonathan Lemon 
Link: https://lore.kernel.org/bpf/CACtPs=GGvV-_Yj6rbpzTVnopgi5nhMoCcTkSkYrJHGQHJWFZMQ@mail.gmail.com/
Link: https://lore.kernel.org/bpf/20200525080400.13195-1-bjorn.topel@gmail.com

xsk: Add missing check on user supplied headroom size

2020-04-15T11:07:18+00:00

Add a check that the headroom cannot be larger than the available
space in the chunk. In the current code, a malicious user can set the
headroom to a value larger than the chunk size minus the fixed XDP
headroom. That way packets with a length larger than the supported
size in the umem could get accepted and result in an out-of-bounds
write.

Fixes: c0c77d8fb787 ("xsk: add user memory registration support sockopt")
Reported-by: Bui Quang Minh 
Signed-off-by: Magnus Karlsson 
Signed-off-by: Daniel Borkmann 
Link: https://bugzilla.kernel.org/show_bug.cgi?id=207225
Link: https://lore.kernel.org/bpf/1586849715-23490-1-git-send-email-magnus.karlsson@intel.com

xsk: Fix out of boundary write in __xsk_rcv_memcpy

2020-04-06T19:48:05+00:00

first_len is the remainder of the first page we're copying.
If this size is larger, then out of page boundary write will
otherwise happen.

Fixes: c05cd3645814 ("xsk: add support to allow unaligned chunk placement")
Signed-off-by: Li RongQing 
Signed-off-by: Daniel Borkmann 
Acked-by: Jonathan Lemon 
Acked-by: Björn Töpel 
Link: https://lore.kernel.org/bpf/1585813930-19712-1-git-send-email-lirongqing@baidu.com

xdp: Replace zero-length array with flexible-array member

2020-02-28T20:08:37+00:00

The current codebase makes use of the zero-length array language
extension to the C90 standard, but the preferred mechanism to declare
variable-length types such as these ones is a flexible array member[1][2],
introduced in C99:

struct foo {
        int stuff;
        struct boo array[];
};

By making use of the mechanism above, we will get a compiler warning
in case the flexible array does not occur last in the structure, which
will help us prevent some kind of undefined behavior bugs from being
inadvertently introduced[3] to the codebase from now on.

Also, notice that, dynamic memory allocations won't be affected by
this change:

"Flexible array members have incomplete type, and so the sizeof operator
may not be applied. As a quirk of the original implementation of
zero-length arrays, sizeof evaluates to zero."[1]

This issue was found with the help of Coccinelle.

[1] https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
[2] https://github.com/KSPP/linux/issues/21
[3] commit 76497732932f ("cxgb3/l2t: Fix undefined behaviour")

Signed-off-by: Gustavo A. R. Silva 
Acked-by: Jonathan Lemon 
Acked-by: Björn Töpel 
Signed-off-by: David S. Miller

xsk: Publish global consumer pointers when NAPI is finished

2020-02-11T14:51:11+00:00

The commit 4b638f13bab4 ("xsk: Eliminate the RX batch size")
introduced a much more lazy way of updating the global consumer
pointers from the kernel side, by only doing so when running out of
entries in the fill or Tx rings (the rings consumed by the
kernel). This can result in a deadlock with the user application if
the kernel requires more than one entry to proceed and the application
cannot put these entries in the fill ring because the kernel has not
updated the global consumer pointer since the ring is not empty.

Fix this by publishing the local kernel side consumer pointer whenever
we have completed Rx or Tx processing in the kernel. This way, user
space will have an up-to-date view of the consumer pointers whenever it
gets to execute in the one core case (application and driver on the
same core), or after a certain number of packets have been processed
in the two core case (application and driver on different cores).

A side effect of this patch is that the one core case gets better
performance, but the two core case gets worse. The reason that the one
core case improves is that updating the global consumer pointer is
relatively cheap since the application by definition is not running
when the kernel is (they are on the same core) and it is beneficial
for the application, once it gets to run, to have pointers that are
as up to date as possible since it then can operate on more packets
and buffers. In the two core case, the most important performance
aspect is to minimize the number of accesses to the global pointers
since they are shared between two cores and bounces between the caches
of those cores. This patch results in more updates to global state,
which means lower performance in the two core case.

Fixes: 4b638f13bab4 ("xsk: Eliminate the RX batch size")
Reported-by: Ryan Goodfellow 
Reported-by: Maxim Mikityanskiy 
Signed-off-by: Magnus Karlsson 
Signed-off-by: Daniel Borkmann 
Acked-by: Jonathan Lemon 
Acked-by: Maxim Mikityanskiy 
Link: https://lore.kernel.org/bpf/1581348432-6747-1-git-send-email-magnus.karlsson@intel.com

mm, tree-wide: rename put_user_page() to unpin_user_page()

2020-01-31T18:30:38+00:00

In order to provide a clearer, more symmetric API for pinning and
unpinning DMA pages.  This way, pin_user_pages*() calls match up with
unpin_user_pages*() calls, and the API is a lot closer to being
self-explanatory.

Link: http://lkml.kernel.org/r/20200107224558.2362728-23-jhubbard@nvidia.com
Signed-off-by: John Hubbard 
Reviewed-by: Jan Kara 
Cc: Alex Williamson 
Cc: Aneesh Kumar K.V 
Cc: Björn Töpel 
Cc: Christoph Hellwig 
Cc: Daniel Vetter 
Cc: Dan Williams 
Cc: Hans Verkuil 
Cc: Ira Weiny 
Cc: Jason Gunthorpe 
Cc: Jason Gunthorpe 
Cc: Jens Axboe 
Cc: Jerome Glisse 
Cc: Jonathan Corbet 
Cc: Kirill A. Shutemov 
Cc: Leon Romanovsky 
Cc: Mauro Carvalho Chehab 
Cc: Mike Rapoport 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

net/xdp: set FOLL_PIN via pin_user_pages()

2020-01-31T18:30:37+00:00

Convert net/xdp to use the new pin_longterm_pages() call, which sets
FOLL_PIN.  Setting FOLL_PIN is now required for code that requires
tracking of pinned pages.

In partial anticipation of this work, the net/xdp code was already calling
put_user_page() instead of put_page().  Therefore, in order to convert
from the get_user_pages()/put_page() model, to the
pin_user_pages()/put_user_page() model, the only change required here is
to change get_user_pages() to pin_user_pages().

Link: http://lkml.kernel.org/r/20200107224558.2362728-18-jhubbard@nvidia.com
Signed-off-by: John Hubbard 
Acked-by: Björn Töpel 
Cc: Alex Williamson 
Cc: Aneesh Kumar K.V 
Cc: Christoph Hellwig 
Cc: Daniel Vetter 
Cc: Dan Williams 
Cc: Hans Verkuil 
Cc: Ira Weiny 
Cc: Jan Kara 
Cc: Jason Gunthorpe 
Cc: Jason Gunthorpe 
Cc: Jens Axboe 
Cc: Jerome Glisse 
Cc: Jonathan Corbet 
Cc: Kirill A. Shutemov 
Cc: Leon Romanovsky 
Cc: Mauro Carvalho Chehab 
Cc: Mike Rapoport 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

xsk, net: Make sock_def_readable() have external linkage

2020-01-21T23:08:52+00:00

XDP sockets use the default implementation of struct sock's
sk_data_ready callback, which is sock_def_readable(). This function
is called in the XDP socket fast-path, and involves a retpoline. By
letting sock_def_readable() have external linkage, and being called
directly, the retpoline can be avoided.

Signed-off-by: Björn Töpel 
Signed-off-by: Daniel Borkmann 
Link: https://lore.kernel.org/bpf/20200120092917.13949-1-bjorn.topel@gmail.com

xsk: Support allocations of large umems

2020-01-15T19:41:52+00:00

When registering a umem area that is sufficiently large (>1G on an
x86), kmalloc cannot be used to allocate one of the internal data
structures, as the size requested gets too large. Use kvmalloc instead
that falls back on vmalloc if the allocation is too large for kmalloc.

Also add accounting for this structure as it is triggered by a user
space action (the XDP_UMEM_REG setsockopt) and it is by far the
largest structure of kernel allocated memory in xsk.

Reported-by: Ryan Goodfellow 
Signed-off-by: Magnus Karlsson 
Signed-off-by: Alexei Starovoitov 
Acked-by: Jonathan Lemon 
Link: https://lore.kernel.org/bpf/1578995365-7050-1-git-send-email-magnus.karlsson@intel.com