Pull io_uring update from Jens Axboe:
 "On top of the usual cleanups and fixes, the bigger items in here are:

   - zcrx work, most of it centered around adding dynamic area
     provisioning, plus a bunch of prep and cleanups leading up to it:
     scale refilling with large pages, coalesce same-niov RQEs on
     refill, separate the RQ head/tail cache lines and cache the RQ
     tail, and rework the area creation locking.

   - Fix the futex inflight accounting so that only private futex waits
     are marked inflight, and don't mark wake requests as inflight at
     all.

   - Drop the custom iov copy in the buffer select prep and msg header
     copy paths, using the generic helpers instead.

   - Fix a folio size overflow in io_vec_fill_bvec(), and account the
     pages a compound region really uses in the memmap path.

   - Fix an iovec leak in uring_cmd when the async cmd isn't recycled,
     skip the blocking task work for io_uring_cmd_issue_blocking(), and
     don't skip completion for a synchronous multishot cmd

   - Defer eventfd signaling when queued from a wakeup handler

   - Fix io-wq worker accounting when canceling creation callbacks

   - Annotate remote tasks for kcoverage"

* tag 'for-7.3/io_uring-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (30 commits)
  io_uring: Add missing include for ITER_SOURCE and ITER_DEST
  io_uring/uring_cmd: don't skip completion for a synchronous multishot cmd
  io_uring/memmap: account the pages a compound region really uses
  io_uring/zcrx: add dynamic area provisioning
  io_uring/zcrx: lock area creation with pp_lock
  io_uring/zcrx: keep array of areas
  io_uring/zcrx: move freelist lock to struct zcrx
  io_uring/zcrx: unmap under netdev lock
  io_uring/zcrx: split dmabuf unmap and release
  io_uring/zcrx: don't pass ifq_reg to area creation
  io_uring/zcrx: add helper for deriving area token
  io_uring/zcrx: don't reload skb_shinfo
  io_urint/zcrx: narrow var scope in io_zcrx_recv_skb()
  io_uring/zcrx: constify area_reg on import
  io_uring/zcrx: coalesce same-niov RQEs on refill
  io_uring/zcrx: cache RQ tail
  io_uring/zcrx: add RQ iterator
  io_uring/zcrx: move RQ head/tail to separate cache lines
  io_uring/zcrx: scale refilling with large pages
  io_uring/io-wq: fix worker accounting when canceling creation callbacks
  ...

io_mem_alloc_compound() allocates get_order(size) pages, which rounds a
region size that is not a power of two up to the next order. The pages
past the region are part of the same allocation and cannot be used for
anything else, but io_create_region() accounts reg->size >> PAGE_SHIFT,
so they are never charged against RLIMIT_MEMLOCK. For a ring with 4096
SQ entries and the default CQ size the region is 37 pages while the
allocation is 64.

Account the tail pages together with the region, and fall back to the
exact sized bulk allocation when they do not fit the limit, so a user
close to their limit still gets the region rather than an error.
io_free_region() gives the same amount back, the compound case being the
one that set IO_REGION_F_SINGLE_REF.

Counting how many 4096 entry rings an unprivileged user can create under
a given RLIMIT_MEMLOCK, before and after:

	limit (pages)	before	after
	256		2	2
	300		2	2
	350		3	2
	400		3	3
	512		5	4

Five rings under a 512 page limit really pin 640 pages.

Fixes: dfbbfbf19187 ("io_uring: introduce concept of memory regions")
Link: https://lore.kernel.org/all/87ik5ncj8d.fsf@mailhost.krisman.be/
Signed-off-by: Ali Ahmet Memis <ali@iusegentoo.com>
Link: https://patch.msgid.link/20260806180044.275543-1-ali@iusegentoo.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

get_unmapped_area() returns -ENOMEM when io_uring_validate_mmap_request()
fails, but validation errors are -EINVAL. Propagate that errno to
userspace, like io_uring_mmap() already does.

Signed-off-by: Yi Xie <xieyi@kylinos.cn>
Link: https://patch.msgid.link/20260630091206.126206-1-xieyi@kylinos.cn
Signed-off-by: Jens Axboe <axboe@kernel.dk>

io_pin_pages() checks that nr_pages does not exceed INT_MAX, then
allocates a struct page * array of nr_pages entries. kvmalloc() limits
allocations to INT_MAX bytes, but the check counts pages, not bytes.
On 64-bit each entry is 8 bytes, so the array hits the INT_MAX byte
limit at INT_MAX / sizeof(struct page *) pages, well before the page
count check fires.

Since commit b4e41050b212 ("io_uring/rsrc: raise registered buffer 1GB
limit") raised the per-buffer cap to 1TB, a buffer near that cap maps
~2^28 pages, making the array allocation exceed INT_MAX bytes. This
passes the page count check, reaches kvmalloc(), and triggers the
WARN_ON_ONCE() for oversized allocations in __kvmalloc_node_noprof().

Check nr_pages against INT_MAX / sizeof(struct page *) so the buffer is
rejected with -EOVERFLOW before the allocation is attempted.

Reported-by: syzbot+f99b00a963915b6b52c6@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=f99b00a963915b6b52c6
Fixes: b4e41050b212 ("io_uring/rsrc: raise registered buffer 1GB limit")
Tested-by: syzbot+f99b00a963915b6b52c6@syzkaller.appspotmail.com
Signed-off-by: Deepanshu Kartikey <kartikey406@gmail.com>
Reviewed-by: Gabriel Krisman Bertazi <krisman@suse.de>
Link: https://patch.msgid.link/20260621012933.50571-1-kartikey406@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Under !CONFIG_MMU, io_uring_get_unmapped_area() returns the kernel
virtual address of the io_mapped_region's backing pages directly;
the user's VMA aliases the kernel allocation. io_uring_mmap() then
just returns 0 -- it takes no page references.

The CONFIG_MMU path uses vm_insert_pages(), which takes a reference on
each inserted page.  Those references are released when the VMA is torn
down (zap_pte_range -> put_page). io_free_region() -> release_pages()
drops the io_uring-side references, but the pages survive until munmap
drops the VMA-side references.

Under NOMMU there are no VMA-side references. io_unregister_pbuf_ring ->
io_put_bl -> io_free_region -> release_pages drops the only references
and the pages return to the buddy allocator while the user's VMA still
has vm_start pointing into them.  The user can then write into whatever
the allocator hands out next.

Mirror the MMU lifetime: take get_page references in io_uring_mmap() and
release them via vm_ops->close.  NOMMU's delete_vma() calls vma_close()
which runs ->close on munmap.

This also incidentally addresses the duplicate-vm_start case: two mmaps
of SQ_RING and CQ_RING resolve to the same ctx->ring_region pointer.
With page refs taken per mmap, the second mmap takes its own refs and
the pages survive until both mmaps are closed.  The nommu rb-tree BUG_ON
on duplicate vm_start is a separate mm/nommu.c concern (it should share
the existing region rather than BUG), but the page lifetime is now
correct.

Cc: Jens Axboe <axboe@kernel.dk>
Reported-by: Anthropic
Assisted-by: gkh_clanker_t1000
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://patch.msgid.link/2026042115-body-attention-d15b@gregkh
[axboe: get rid of region lookup, just iterate pages in vma]
Signed-off-by: Jens Axboe <axboe@kernel.dk>

This is the result of running the Coccinelle script from
scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to
avoid scalar types (which need careful case-by-case checking), and
instead replace kmalloc-family calls that allocate struct or union
object instances:

Single allocations:	kmalloc(sizeof(TYPE), ...)
are replaced with:	kmalloc_obj(TYPE, ...)

Array allocations:	kmalloc_array(COUNT, sizeof(TYPE), ...)
are replaced with:	kmalloc_objs(TYPE, COUNT, ...)

Flex array allocations:	kmalloc(struct_size(PTR, FAM, COUNT), ...)
are replaced with:	kmalloc_flex(*PTR, FAM, COUNT, ...)

(where TYPE may also be *VAR)

The resulting allocations no longer return "void *", instead returning
"TYPE *".

Signed-off-by: Kees Cook <kees@kernel.org>

Pull io_uring updates from Jens Axboe:

 - Clean up the IORING_SETUP_R_DISABLED and submitter task checking,
   mostly just in preparation for relaxing the locking for SINGLE_ISSUER
   in the future.

 - Improve IOPOLL by using a doubly linked list to manage completions.

   Previously it was singly listed, which meant that to complete request
   N in the chain 0..N-1 had to have completed first. With a doubly
   linked list we can complete whatever request completes in that order,
   rather than need to wait for a consecutive range to be available.
   This reduces latencies.

 - Improve the restriction setup and checking. Mostly in preparation for
   adding further features on top of that. Coming in a separate pull
   request.

 - Split out task_work and wait handling into separate files. These are
   mostly nicely abstracted already, but still remained in the
   io_uring.c file which is on the larger side.

 - Use GFP_KERNEL_ACCOUNT in a few more spots, where appropriate.

 - Ensure even the idle io-wq worker exits if a task no longer has any
   rings open.

 - Add support for a non-circular submission queue.

   By default, the SQ ring keeps moving around, even if only a few
   entries are used for each submission. This can be wasteful in terms
   of cachelines.

   If IORING_SETUP_SQ_REWIND is set for the ring when created, each
   submission will start at offset 0 instead of where we last left off
   doing submissions.

 - Various little cleanups

* tag 'for-7.0/io_uring-20260206' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (30 commits)
  io_uring/kbuf: fix memory leak if io_buffer_add_list fails
  io_uring: Add SPDX id lines to remaining source files
  io_uring: allow io-wq workers to exit when unused
  io_uring/io-wq: add exit-on-idle state
  io_uring/net: don't continue send bundle if poll was required for retry
  io_uring/rsrc: use GFP_KERNEL_ACCOUNT consistently
  io_uring/futex: use GFP_KERNEL_ACCOUNT for futex data allocation
  io_uring/io-wq: handle !sysctl_hung_task_timeout_secs
  io_uring: fix bad indentation for setup flags if statement
  io_uring/rsrc: take unsigned index in io_rsrc_node_lookup()
  io_uring: introduce non-circular SQ
  io_uring: split out CQ waiting code into wait.c
  io_uring: split out task work code into tw.c
  io_uring/io-wq: don't trigger hung task for syzbot craziness
  io_uring: add IO_URING_EXIT_WAIT_MAX definition
  io_uring/sync: validate passed in offset
  io_uring/eventfd: remove unused ctx->evfd_last_cq_tail member
  io_uring/timeout: annotate data race in io_flush_timeouts()
  io_uring/uring_cmd: explicitly disallow cancelations for IOPOLL
  io_uring: fix IOPOLL with passthrough I/O
  ...

For potential long term allocations, ensure that we play nicer with
memcg and use the accounting variant of the GFP_KERNEL allocation.

Signed-off-by: Jens Axboe <axboe@kernel.dk>

Pull io_uring fixes from Jens Axboe:

 - Removed dead argument length for io_uring_validate_mmap_request()

 - Use GFP_NOWAIT for overflow CQEs on legacy ring setups rather than
   GFP_ATOMIC, which makes it play nicer with memcg limits

 - Fix a potential circular locking issue with tctx node removal and
   exec based cancelations

* tag 'io_uring-6.19-20260102' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux:
  io_uring/memmap: drop unused sz param in io_uring_validate_mmap_request()
  io_uring/tctx: add separate lock for list of tctx's in ctx
  io_uring: use GFP_NOWAIT for overflow CQEs on legacy rings

io_uring_validate_mmap_request() doesn't use its size_t sz argument, so
remove it.

Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>