Commit:

6a8afb9fff64 ("io_uring/net: allow multishot receive per-invocation cap")

changed how io_mshot_prep_retry() set sr->len, and added the same
initialization in io_mshot_prep_retry(). But it neglected to touch the
send path, which may also uses the mshot retry path. Ensure that
sr->mshot_len always gets initialized correctly.

Fixes: 6a8afb9fff64 ("io_uring/net: allow multishot receive per-invocation cap")
Cc: stable@vger.kernel.org
Reported-by: Sung Keum <kambodi127@gmail.com>
Reviewed-by: Gabriel Krisman Bertazi <krisman@suse.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Pull io_uring updates from Jens Axboe:

 - Rework the task_work infrastructure.

   Both the local (DEFER_TASKRUN) and the normal (tctx) task_work lists
   were llist based, which is LIFO ordered, and hence each run had to do
   an O(n) list reversal pass first to restore queue order.
   Additionally, to cap the amount of task_work run, each method needed
   a retry list as well.

   Add a lockless MPCS FIFO queue (based on Dmitry Vyukov's intrusive
   MPSC algorithm) and switch both task_work lists to it. It performs
   better than llists and we can then also ditch the retry lists as well
   as entries are popped one-at-the-time.

   On top of those changes, run the tctx fallback task_work directly and
   remove the now-unused per-ctx fallback machinery entirely.

 - zcrx user notifications.

   Add a mechanism for zcrx to communicate conditions back to userspace
   via a dedicated CQE, with the initial users being notification on
   running out of buffers and on a frag copy fallback, plus
   shared-memory notification statistics.

   Alongside that, a series of zcrx reliability and cleanup fixes: more
   reliable scrubbing, poisoning pointers on unregistration, dropping an
   extra ifq close, adding a ctx back-pointer, reordering fd allocation
   in the export path, and killing a dead 'sock' member.

 - Allow using io_uring registered buffers for plain SEND and RECV, not
   just for the zero-copy send path.

   This enables targets like ublk's NBD backend to push/pull IO data
   directly to/from a registered buffer over a plain send/recv on a TCP
   socket.

 - Registered buffer improvements: account huge pages correctly, bump
   the io_mapped_ubuf length field to size_t, and raise the previous 1GB
   registered buffer size limit.

 - Restrict the ctx access exposed to io_uring BPF struct_ops programs
   by handing them an opaque type rather than the full io_ring_ctx, and
   add a separate MAINTAINERS entry for the bpf-ops code.

 - Allow opcode filtering on IORING_OP_CONNECT.

 - Validate ring-provided buffer addresses with access_ok(), and align
   the legacy buffer add limit with MAX_BIDS_PER_BGID.

 - Various other cleanups and minor fixes, including avoiding msghdr
   async data on connect/bind, dropping async_size for OP_LISTEN, making
   the POLL_FIRST receive side checks consistent, re-checking
   IO_WQ_BIT_EXIT for each linked work item, and using
   trace_call__##name() at guarded tracepoint call sites.

* tag 'for-7.2/io_uring-20260615' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (31 commits)
  io_uring/bpf-ops: add a separate maintainer entry
  io_uring/net: make POLL_FIRST receive side checks consistent
  io_uring: remove the per-ctx fallback task_work machinery
  io_uring: run the tctx task_work fallback directly
  io_uring: switch normal task_work to a mpscq
  io_uring: switch local task_work to a mpscq
  io_uring/mpscq: add lockless multi-producer, single-consumer FIFO queue
  io_uring: grab RCU read lock marking task run
  io_uring/zcrx: kill dead 'sock' member in struct io_zcrx_args
  io_uring/kbuf: validate ring provided buffer addresses with access_ok()
  io_uring/net: support registered buffer for plain send and recv
  io_uring/nop: Drop a wrong comment in struct io_nop
  io_uring/net: Remove async_size for OP_LISTEN
  io_uring/net: Avoid msghdr on op_connect/op_bind async data
  io_uring/bpf-ops: restrict ctx access to BPF
  io_uring/io-wq: re-check IO_WQ_BIT_EXIT for each linked work item
  io_uring/kbuf: align legacy buffer add limit with MAX_BIDS_PER_BGID
  io_uring/zcrx: add shared-memory notification statistics
  io_uring/zcrx: notify user on frag copy fallback
  io_uring/zcrx: notify user when out of buffers
  ...

Pull crypto updates from Herbert Xu:
 "API:
   - Drop support for off-CPU cryptography in af_alg
   - Document that af_alg is *always* slower
   - Document the deprecation of af_alg
   - Remove zero-copy support from skcipher and aead in af_alg
   - Cap AEAD AD length to 0x80000000 in af_alg
   - Free default RNG on module exit

  Algorithms:
   - Fix vli multiplication carry overflow in ecc
   - Drop unused cipher_null crypto_alg
   - Remove unused variants of drbg
   - Use lib/crypto in drbg
   - Use memcpy_from/to_sglist in authencesn
   - Allow authenc(hmac(sha{256,384}),cts(cbc(aes))) in FIPS mode
   - Disallow RSA PKCS#1 SHA-1 sig algs in FIPS mode
   - Filter out async aead implementations at alloc in krb5
   - Fix non-parallel fallback by rstoring callback in pcrypt
   - Validate poly1305 template argument in chacha20poly1305

  Drivers:
   - Add sysfs PCI reset support to qat
   - Add KPT support for GEN6 devices to qat
   - Remove unused character device and ioctls from qat
   - Add support for hw access via SMCC to mtk
   - Remove prng support from crypto4xx
   - Remove prng support from hisi-trng
   - Remove prng support from sun4i-ss
   - Remove prng support from xilinx-trng
   - Remove loongson-rng
   - Remove exynos-rng

  Others:
   - Remove support for AIO on sockets"

* tag 'v7.2-p1' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (196 commits)
  crypto: tegra - fix refcount leak in tegra_se_host1x_submit()
  crypto: rng - Free default RNG on module exit
  crypto: testmgr - allow authenc(hmac(sha{256,384}),cts(cbc(aes))) in FIPS mode
  hwrng: jh7110 - fix refcount leak in starfive_trng_read()
  crypto: atmel-ecc - drop dead code in atmel_ecdh_max_size
  crypto: cavium/cpt - fix DMA cleanup using wrong loop index
  crypto: marvell/octeontx - fix DMA cleanup using wrong loop index
  MAINTAINERS: make myself the maintainer of the Qualcomm QCE driver
  crypto: amcc - convert irq_of_parse_and_map to platform_get_irq
  crypto: sun4i-ss - Remove insecure and unused rng_alg
  hwrng: xilinx - Move xilinx-rng into drivers/char/hw_random/
  crypto: xilinx-trng - Replace crypto_drbg_ctr_df() with HMAC-SHA512
  crypto: xilinx-trng - Fix return value of xtrng_hwrng_trng_read()
  crypto: xilinx-trng - Remove crypto_rng interface
  crypto: exynos-rng - Remove exynos-rng driver
  hwrng: hisi-trng - Move hisi-trng into drivers/char/hw_random/
  crypto: hisi-trng - Remove crypto_rng interface
  crypto: loongson - Remove broken and unused loongson-rng
  crypto: crypto4xx - Remove insecure and unused rng_alg
  crypto: qat - validate RSA CRT component lengths
  ...

io_recv() and io_recvzc() are the odd ones out, as they checks for
whether POLL_FIRST should be honored before checking if the file is a
socket. It doesn't really matter, but might as well make it consistent
across all receive and send types.

Signed-off-by: Jens Axboe <axboe@kernel.dk>

So far IORING_RECVSEND_FIXED_BUF is only honoured on the SEND_ZC path,
even though the import wiring is already present for plain send and
completely absent for recv. Targets such as ublk's NBD backend want to
push/pull I/O data directly to/from an io_uring registered buffer over a
plain send/recv on a TCP socket.

Wire IORING_RECVSEND_FIXED_BUF into the plain IORING_OP_SEND and
IORING_OP_RECV paths:

 - Accept the flag in SENDMSG_FLAGS / RECVMSG_FLAGS and, at prep time,
   restrict it to the non-vectorized IORING_OP_SEND / IORING_OP_RECV
   opcodes. It is mutually exclusive with buffer select, bundles and
   (for recv) multishot, and records sqe->buf_index.

 - For recv, set REQ_F_IMPORT_BUFFER in setup so the registered buffer
   is imported lazily at issue time, mirroring the send path.

 - In io_send()/io_recv(), import the registered buffer via
   io_import_reg_buf() (ITER_SOURCE for send, ITER_DEST for recv) and
   clear REQ_F_IMPORT_BUFFER. The resulting bvec iter persists in
   async_data, so MSG_WAITALL partial send/recv retries resume at the
   right offset.

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
Link: https://patch.msgid.link/20260608142511.659240-2-ming.lei@redhat.com
[axboe: combine flags checks]
Signed-off-by: Jens Axboe <axboe@kernel.dk>

When a bundle recv retries inside io_recv_finish(), the merge logic OR
the saved cflags from the previous iteration with the cflags returned by
the new iteration:
  cflags = req->cqe.flags | (cflags & CQE_F_MASK);

Bits listed in CQE_F_MASK are inherited from the new iteration, and all
other bits (notably IORING_CQE_F_BUFFER and the buffer ID) come from the
saved cflags. Before this change CQE_F_MASK covered only
IORING_CQE_F_SOCK_NONEMPTY and IORING_CQE_F_MORE.

When using provided buffer rings (IOU_PBUF_RING_INC) with incremental
mode, and bundle recv, io_kbuf_inc_commit() can leave the head ring
entry partially consumed, __io_put_kbufs() then sets
IORING_CQE_F_BUF_MORE on the returned cflags so userspace knows the
buffer ID will be reused for subsequent completions.

Because IORING_CQE_F_BUF_MORE was not in CQE_F_MASK, the merge above
silently dropped it whenever the final retry iteration partially
consumed the buffer, and the subsequent req->cqe.flags = cflags &
~CQE_F_MASK save would have left a stale IORING_CQE_F_BUF_MORE in the
carried-over cflags had one been present. Userspace would then
wrongfully advance it ring head past an entry the kernel still uses.

Add IORING_CQE_F_BUF_MORE to CQE_F_MASK so it is both inherited from the
new iteration into the user-visible CQE and stripped from the saved
cflags between iterations.

Cc: stable@vger.kernel.org
Signed-off-by: Clément Léger <cleger@meta.com>
Assisted-by: Claude:claude-opus-4.6
Fixes: ae98dbf43d75 ("io_uring/kbuf: add support for incremental buffer consumption")
Link: https://patch.msgid.link/20260604160715.2482972-1-cleger@meta.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Both IORING_OP_CONNECT and IORING_OP_BIND reuse the msghdr object just
to store the sockaddr. Beyond allocating a much larger object than
needed, msghdr can also wrap an iovec, which will be recycled
unnecessarily. This uses the sockaddr directly.

Signed-off-by: Gabriel Krisman Bertazi <krisman@suse.de>
Link: https://patch.msgid.link/20260602215327.1885109-2-krisman@suse.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

The only user of msg->msg_iocb was AF_ALG, but that's deprecated.
It can be removed entirely at the cost of only supporting synchronous
operations.  This doesn't break userspace, which will silently block
(for a bounded amount of time) in io_submit instead of operating
asynchronously.

This also makes struct msghdr smaller, helping every other caller of
sendmsg().

Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

For two reasons:

1) An opcode cannot block inside io_uring_enter() doing submissions, as
   it'll stall the submission side pipeline.

2) Ending up in sb_start_write() -> __sb_start_write() ->
   percpu_down_read_freezable() introduces a new lockdep edge, which it
   correctly complains about.

Check if the socket type is AF_UNIX and has a non-empty pathname. If it
does, mark it REQ_F_FORCE_ASYNC to punt the submission to io-wq rather
than attempt to do it inline.

Fixes: 7481fd93fa0a ("io_uring: Introduce IORING_OP_BIND")
Reviewed-by: Gabriel Krisman Bertazi <krisman@suse.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

This adds custom filtering for IORING_OP_CONNECT, where the target
family is always exposed, and (for AF_INET / AF_INET6) port and
address are exposed. port and v4_addr are in network byte order so
filter authors can compare against on-wire constants.

Skip population unless addr_len covers the populated fields, to
avoid leaking stale io_async_msghdr data on short connects.

Signed-off-by: Shouvik Kar <auxcorelabs@gmail.com>
Link: https://patch.msgid.link/20260512110242.26219-1-auxcorelabs@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>