linux-stable.git/include/net/udp.h, branch linux-6.13.y

ipv6/udp: Add 4-tuple hash for connected socket

2024-11-18T11:56:21+00:00

Implement ipv6 udp hash4 like that in ipv4. The major difference is that
the hash value should be calculated with udp6_ehashfn(). Besides,
ipv4-mapped ipv6 address is handled before hash() and rehash(). Export
udp_ehashfn because now we use it in udpv6 rehash.

Core procedures of hash/unhash/rehash are same as ipv4, and udpv4 and
udpv6 share the same udptable, so some functions in ipv4 hash4 can also
be shared.

Co-developed-by: Cambda Zhu 
Signed-off-by: Cambda Zhu 
Co-developed-by: Fred Chen 
Signed-off-by: Fred Chen 
Co-developed-by: Yubing Qiu 
Signed-off-by: Yubing Qiu 
Signed-off-by: Philo Lu 
Acked-by: Willem de Bruijn 
Acked-by: Paolo Abeni 
Signed-off-by: David S. Miller

ipv4/udp: Add 4-tuple hash for connected socket

2024-11-18T11:56:21+00:00

Currently, the udp_table has two hash table, the port hash and portaddr
hash. Usually for UDP servers, all sockets have the same local port and
addr, so they are all on the same hash slot within a reuseport group.

In some applications, UDP servers use connect() to manage clients. In
particular, when firstly receiving from an unseen 4 tuple, a new socket
is created and connect()ed to the remote addr:port, and then the fd is
used exclusively by the client.

Once there are connected sks in a reuseport group, udp has to score all
sks in the same hash2 slot to find the best match. This could be
inefficient with a large number of connections, resulting in high
softirq overhead.

To solve the problem, this patch implement 4-tuple hash for connected
udp sockets. During connect(), hash4 slot is updated, as well as a
corresponding counter, hash4_cnt, in hslot2. In __udp4_lib_lookup(),
hslot4 will be searched firstly if the counter is non-zero. Otherwise,
hslot2 is used like before. Note that only connected sockets enter this
hash4 path, while un-connected ones are not affected.

hlist_nulls is used for hash4, because we probably move to another hslot
wrongly when lookup with concurrent rehash. Then we check nulls at the
list end to see if we should restart lookup. Because udp does not use
SLAB_TYPESAFE_BY_RCU, we don't need to touch sk_refcnt when lookup.

Stress test results (with 1 cpu fully used) are shown below, in pps:
(1) _un-connected_ socket as server
    [a] w/o hash4: 1,825176
    [b] w/  hash4: 1,831750 (+0.36%)

(2) 500 _connected_ sockets as server
    [c] w/o hash4:   290860 (only 16% of [a])
    [d] w/  hash4: 1,889658 (+3.1% compared with [b])

With hash4, compute_score is skipped when lookup, so [d] is slightly
better than [b].

Co-developed-by: Cambda Zhu 
Signed-off-by: Cambda Zhu 
Co-developed-by: Fred Chen 
Signed-off-by: Fred Chen 
Co-developed-by: Yubing Qiu 
Signed-off-by: Yubing Qiu 
Signed-off-by: Philo Lu 
Acked-by: Willem de Bruijn 
Acked-by: Paolo Abeni 
Signed-off-by: David S. Miller

net/udp: Add 4-tuple hash list basis

2024-11-18T11:56:21+00:00

Add a new hash list, hash4, in udp table. It will be used to implement
4-tuple hash for connected udp sockets. This patch adds the hlist to
table, and implements helpers and the initialization. 4-tuple hash is
implemented in the following patch.

hash4 uses hlist_nulls to avoid moving wrongly onto another hlist due to
concurrent rehash, because rehash() can happen with lookup().

Co-developed-by: Cambda Zhu 
Signed-off-by: Cambda Zhu 
Co-developed-by: Fred Chen 
Signed-off-by: Fred Chen 
Co-developed-by: Yubing Qiu 
Signed-off-by: Yubing Qiu 
Signed-off-by: Philo Lu 
Acked-by: Willem de Bruijn 
Acked-by: Paolo Abeni 
Signed-off-by: David S. Miller

net/udp: Add a new struct for hash2 slot

2024-11-18T11:56:21+00:00

Preparing for udp 4-tuple hash (uhash4 for short).

To implement uhash4 without cache line missing when lookup, hslot2 is
used to record the number of hashed sockets in hslot4. Thus adding a new
struct udp_hslot_main with field hash4_cnt, which is used by hash2. The
new struct is used to avoid doubling the size of udp_hslot.

Before uhash4 lookup, firstly checking hash4_cnt to see if there are
hashed sks in hslot4. Because hslot2 is always used in lookup, there is
no cache line miss.

Related helpers are updated, and use the helpers as possible.

uhash4 is implemented in following patches.

Signed-off-by: Philo Lu 
Acked-by: Willem de Bruijn 
Acked-by: Paolo Abeni 
Signed-off-by: David S. Miller

net: Correct spelling in headers

2024-08-26T16:37:23+00:00

Correct spelling in Networking headers.
As reported by codespell.

Signed-off-by: Simon Horman 
Link: https://patch.msgid.link/20240822-net-spell-v1-12-3a98971ce2d2@kernel.org
Signed-off-by: Jakub Kicinski

ipv6: udp: constify 'struct net' parameter of socket lookups

2024-08-05T23:27:26+00:00

Following helpers do not touch their 'struct net' argument.

- udp6_lib_lookup()
- __udp6_lib_lookup()

Signed-off-by: Eric Dumazet 
Reviewed-by: Simon Horman 
Link: https://patch.msgid.link/20240802134029.3748005-6-edumazet@google.com
Signed-off-by: Jakub Kicinski

udp: constify 'struct net' parameter of socket lookups

2024-08-05T23:23:59+00:00

Following helpers do not touch their 'struct net' argument.

- udp_sk_bound_dev_eq()
- udp4_lib_lookup()
- __udp4_lib_lookup()

Signed-off-by: Eric Dumazet 
Reviewed-by: Simon Horman 
Link: https://patch.msgid.link/20240802134029.3748005-4-edumazet@google.com
Signed-off-by: Jakub Kicinski

new helper: copy_to_iter_full()

2024-04-07T06:42:36+00:00

... and convert copy_linear_skb() to using that.

Signed-off-by: Al Viro

udp/udplite: Remove unused function declarations udp{,lite}_get_port()

2023-08-07T07:53:55+00:00

Commit 6ba5a3c52da0 ("[UDP]: Make full use of proto.h.udp_hash innovation.")
removed these implementations but leave declarations.

Signed-off-by: Yue Haibing 
Reviewed-by: Willem de Bruijn 
Signed-off-by: David S. Miller

udp: Remove unused function declaration udp_bpf_get_proto()

2023-08-02T19:10:46+00:00

commit 8a59f9d1e3d4 ("sock: Introduce sk->sk_prot->psock_update_sk_prot()")
left behind this.

Signed-off-by: Yue Haibing 
Reviewed-by: Willem de Bruijn 
Link: https://lore.kernel.org/r/20230801133902.3660-1-yuehaibing@huawei.com
Signed-off-by: Jakub Kicinski