Pull block updates from Jens Axboe:
- NVMe updates via Keith:
- Enable Clang context analysis for the nvme host driver, adding
context annotations across core, fabrics, rdma, tcp and pci
- nvmet reservation state exposed through a new namespace-level
debugfs directory, plus ABI documentation for the host sysfs and
target configfs interfaces
- nvme-tcp host memory disclosure fixes on the read path: reject a
read that transferred too few bytes, don't accept C2HData based
on blk_rq_payload_bytes() alone, and fix the R2T case for a read
command
- Parallelize nvme-rdma I/O queue allocation and startup (Surabhi)
- Apple nvme fixes and quirks: page aligned admin queue buffers,
destroy the admin queue on removal, and various DMA/NVMMU
correctness fixes
- A large pile of nvmet and host fixes for out-of-bounds reads,
refcount/resource leaks, and NULL derefs across auth, zns,
passthru, pci-epf, rdma and configfs
- Various other fixes and cleanups
- MD updates via Yu Kuai:
- llbitmap reshape support, the large series wiring exact bitmap
mapping and reshape lifecycle through raid5 and raid10, growing
the page cache in place, and remapping checkpointed bits as
reshape progresses
- raid5 fixes for lockless max_nr_stripes and recovery_offset
accesses, a reshape deadlock with more failed devices than max
degraded, and bitmap batch counter consistency
- Atomic write handling for raid1/raid10, and removal of the
REQ_NOWAIT support from raid1/10/456
- raid5-ppl use-after-free fix in ppl_do_flush()
- A batch of smaller fixes across md core and the bitmap code
- s390/dasd ESE full-track write support and the surrounding
infrastructure, plus enabling CONTEXT_ANALYSIS for s390/block
- RWF_DONTCACHE support for block devices, built on new task-context
bio completion infrastructure, and wiring it up for the iomap and
buffer dropbehind writeback paths
- Async io_uring zone reset all, plus zone management command cleanups
allowing REQ_NOWAIT and tightening conventional zone rejection
- Block integrity refactoring: lift BIP_CHECK_FLAGS to the shared
header, handle nogenerate/noverify properly in fs-integrity, and drop
the blk-integrity.h include from bdev.c
- Split out a new blk_plug.h header
- ublk improvements: add UBLK_F_IO_DESC_SIZE, split request validation
from io_desc init, reject non-power-of-2 zone sizes in SET_PARAMS,
and a series of hardening fixes around map/unmap and auto buf reg
- null_blk cleanups and configfs serialization fixes
- nbd queue freeze removal on the setup paths, and a new
pre_defined_connections module parameter for pre-created devices
- blk-cgroup fixes for the race between policy activation and blkg
destruction, and accounting per-cpu stats over possible CPUs across
blk-stat, iolatency, iocost and kyber
- Various dio fixes: leak on metadata mapping error, validate user
space vectors during extraction, and set dma_alignment from the
backing file for loop and zloop direct I/O
- bio cleanups
- Various other fixes and cleanups all over
* tag 'for-7.3/block-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (241 commits)
nbd: add pre_defined_connections module parameter for pre-created devices
nbd: remove queue freeze for newly created nbd from netlink path
nbd: factor out a nbd_genl_foreach_sock
nbd: skip queue freeze when setting size at device startup
nbd: remove queue freeze in nbd_add_socket
nbd: clear queue limits on disconnect
nbd: disallow NBD_SET_SOCK on an active device
nbd: simplify find_fallback() by removing redundant logic
blk-mq: add missing call to srcu_barrier() in blk_mq_free_tag_set()
block: mtip32xx: synchronize ioctls with device removal
ublk: avoid teardown retry loop on xarray allocation failure
null_blk: fix UBSAN shift-out-of-bounds when zone_size is 0 or overflows
block: don't include blk-integrity.h in bdev.c
xfs: avoid double deferrals for RWF_DONTCACHE writes
loop: Fix recently introduced lock inversion
block: set QUEUE_FLAG_DYING unconditionally in blk_mark_disk_dead()
swim3: Add missing MODULE_DESCRIPTION
selftests: ublk: add SET_PARAMS validation test
selftests: ublk: add helper for SET_PARAMS
ublk: reject non-power-of-2 zone sizes in SET_PARAMS
...
blkdev.h gets included in various places outside the block layer just for struct blk_plug and related plugging functions. Split blk_plug into a separate helper to reduce the amount of code that needs to get rebuilt when blkdev.h changes and to slightly reduce compile times. In io_uring this requires pulling in a few other headers explicitly that previously were implicitly included through blkdev.h. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Christian Brauner (Amutable) <brauner@kernel.org> Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Reviewed-by: Damien Le Moal <dlemoal@kernel.org> Link: https://patch.msgid.link/20260804125524.740996-1-hch@lst.de Signed-off-by: Jens Axboe <axboe@kernel.dk>
Commit c26339e1df33 ("ext4: Fix data integrity writeout issues in
nojournal mode") made ext4_mark_iloc_dirty() attach the inode to a wb
before marking it for metadata writeback in nojournal mode. This is the
first modular caller of inode_attach_wb() - all users of
__inode_attach_wb() so far were built-in - so with CONFIG_EXT4_FS=m and
CONFIG_CGROUP_WRITEBACK=y the build now fails at the modpost stage:
ERROR: modpost: "__inode_attach_wb" [fs/ext4/ext4.ko] undefined!
Export the symbol. Use EXPORT_SYMBOL_GPL() to match the other cgroup
writeback exports in this file.
Fixes: c26339e1df33 ("ext4: Fix data integrity writeout issues in nojournal mode")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202607281811.F3c6kRvX-lkp@intel.com/
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Currently, inode and in general metadata writeback is handled in a lazy manner. When inode is dirty, __writeback_single_inode() calls .write_inode method which for lots of filesystems just copies inode metadata into the underlying block buffer. Writeback of other metadata associated with the inode (as well as buffers underlying inodes) is usually handled completely separately and implicitely during writeback of block device inode. This is good for efficiency of WB_SYNC_NONE writeback or sync(2). However it becomes problematic for situations where we want to make sure inode and its metadata is really persistent on disk. fsync(2) is the most pronounced example of this and thus we have grown a special file operation and various helper functions to assist with this task. However fsync(2) is not the only case, For example directories with DIRSYNC flag need similar functionality and current use of sync_inode_metadata() for this task in filesystems generally misses writeout of necessary metadata. Furthermore even fsync(2) handling as implemented by simple_fsync() or similar helpers is racy and can fail to properly persist the inode. The problem is that WB_SYNC_NONE writeback can copy inode metadata into underlying buffer and clean inode dirty bits. Following fsync(2) will see inode is clean and will fail to make sure underlying buffer is written out. When multiple fsync(2) calls race, there's also another type of race involving mmb_fsync(). There the problem is buffers already submitted to the disk are no longer tracked in the mmb list and so racing mmb_sync() can return before all of the IO completes. Provide a new inode state bit I_METADATA_WRITEBACK tracking whether writeback of inode related metadata may be needed for successful data integrity sync and if this bit is set __writeback_single_inode() for data integrity writeback will call new superblock operation .sync_inode_metadata whose task is to make sure all metadata associated with the inode (including the inode itself) is properly persisted. This will allow filesystems to address the data integrity issues described above and at the same time somewhat simplify our fsync implementations. Issues with racing fsync(2) calls will be addressed by synchronization on I_SYNC inode state which is set while calling .sync_inode_metadata, issues with missed inode buffer writeback are fixed by filesystems looking up corresponding buffer head and writing it out if needed. Signed-off-by: Jan Kara <jack@suse.cz> Link: https://patch.msgid.link/20260727104923.3828017-25-jack@suse.cz Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
The IOCB_DONTCACHE writeback path in generic_write_sync() calls
filemap_flush_range() on every write, submitting writeback inline in
the writer's context. Perf lock contention profiling shows the
performance problem is not lock contention but the writeback submission
work itself — walking the page tree and submitting I/O blocks the writer
for milliseconds, inflating p99.9 latency from 23ms (buffered) to 93ms
(dontcache).
Replace the inline filemap_flush_range() call with a flusher kick that
drains dirty pages in the background. This moves writeback submission
completely off the writer's hot path.
To avoid flushing unrelated buffered dirty data, add a dedicated
WB_start_dontcache bit and wb_check_start_dontcache() handler that uses
the per-wb WB_DONTCACHE_DIRTY counter to determine how many pages to
write back. The flusher writes back that many pages from the oldest dirty
inodes (not restricted to dontcache-specific inodes). This helps
preserve I/O batching while limiting the scope of expedited writeback.
Like WB_start_all, the WB_start_dontcache bit coalesces multiple
DONTCACHE writes into a single flusher wakeup without per-write
allocations. Use test_and_clear_bit to atomically consume the kick
request before reading the dirty counter and starting writeback, so that
concurrent DONTCACHE writes during writeback can re-set the bit and
schedule a follow-up flusher run.
Read the dirty counter with wb_stat_sum() (aggregating per-CPU batches)
rather than wb_stat() (which reads only the global counter) to ensure
small writes below the percpu batch threshold are visible to the flusher.
In filemap_dontcache_kick_writeback(), set the WB_start_dontcache bit
inside the unlocked_inode_to_wb_begin/end section for correct cgroup
writeback domain targeting, but defer the wb_wakeup() call until after
the section ends, since wb_wakeup() uses spin_unlock_irq() which would
unconditionally re-enable interrupts while the i_pages xa_lock may still
be held under irqsave during a cgroup writeback switch. Pin the wb with
wb_get() inside the RCU critical section before calling wb_wakeup()
outside it, since cgroup bdi_writeback structures are RCU-freed and the
wb pointer could become invalid after unlocked_inode_to_wb_end() drops
the RCU read lock.
Also add WB_REASON_DONTCACHE as a new writeback reason for tracing
visibility.
dontcache-bench results (same host, T6F_SKL_1920GBF, 251 GiB RAM,
xfs on NVMe, fio io_uring):
Buffered and direct I/O paths are unaffected by this patchset. All
improvements are confined to the dontcache path:
Single-stream throughput (MB/s):
Before After Change
seq-write/dontcache 298 897 +201%
rand-write/dontcache 131 236 +80%
Tail latency improvements (seq-write/dontcache):
p99: 135,266 us -> 23,986 us (-82%)
p99.9: 8,925,479 us -> 28,443 us (-99.7%)
Multi-writer (4 jobs, sequential write):
Before After Change
dontcache aggregate (MB/s) 2,529 4,532 +79%
dontcache p99 (us) 8,553 1,002 -88%
dontcache p99.9 (us) 109,314 1,057 -99%
Dontcache multi-writer throughput now matches buffered (4,532 vs
4,616 MB/s).
32-file write (Axboe test):
Before After Change
dontcache aggregate (MB/s) 1,548 3,499 +126%
dontcache p99 (us) 10,170 602 -94%
Peak dirty pages (MB) 1,837 213 -88%
Dontcache now reaches 81% of buffered throughput (was 35%).
Competing writers (dontcache vs buffered, separate files):
Before After
buffered writer 868 433 MB/s
dontcache writer 415 433 MB/s
Aggregate 1,284 866 MB/s
Previously the buffered writer starved the dontcache writer 2:1.
With per-bdi_writeback tracking, both writers now receive equal
bandwidth. The aggregate matches the buffered-vs-buffered baseline
(863 MB/s), indicating fair sharing regardless of I/O mode.
The dontcache writer's p99.9 latency collapsed from 119 ms to
33 ms (-73%), eliminating the severe periodic stalls seen in the
baseline. Both writers now share identical latency profiles,
matching the buffered-vs-buffered pattern.
The per-bdi_writeback dirty tracking dramatically reduces peak dirty
pages in dontcache workloads, with the 32-file test dropping from
1.8 GB to 213 MB. Dontcache sequential write throughput triples and
multi-writer throughput reaches parity with buffered I/O, with tail
latencies collapsing by 1-2 orders of magnitude.
Assisted-by: Claude:claude-opus-4-6
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260511-dontcache-v7-3-2848ddce8090@kernel.org
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Add a per-wb WB_DONTCACHE_DIRTY counter that tracks the number of dirty pages with the dropbehind flag set (i.e., pages dirtied via RWF_DONTCACHE writes). Increment the counter alongside WB_RECLAIMABLE in folio_account_dirtied() when the folio has the dropbehind flag set, and decrement it in folio_clear_dirty_for_io() and folio_account_cleaned(). Also decrement it when a non-DONTCACHE lookup atomically clears the dropbehind flag on a dirty folio in __filemap_get_folio_mpol(), using folio_test_clear_dropbehind() to prevent concurrent lookups from double-decrementing the counter, and guarding the decrement with mapping_can_writeback() to match the increment path. Transfer the counter alongside WB_RECLAIMABLE in inode_do_switch_wbs() so that the stat is properly migrated when an inode switches cgroup writeback domains. The counter will be used by the writeback flusher to determine how many pages to write back when expediting writeback for IOCB_DONTCACHE writes, without flushing the entire BDI's dirty pages. Suggested-by: Jan Kara <jack@suse.cz> Assisted-by: Claude:claude-opus-4-6 Signed-off-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260511-dontcache-v7-2-2848ddce8090@kernel.org Reviewed-by: Jan Kara <jack@suse.cz> Reviewed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Tracking in-flight inode wb switches with a single global counter
(isw_nr_in_flight) plus a synchronize_rcu() based wait in
cgroup_writeback_umount() forces every umount to take a global hit
whenever any other superblock on the system has wb switches in flight,
even if the superblock being unmounted has none of its own.
Replace the global synchronize_rcu()/flush_workqueue() pair with a
per-sb counter, s_isw_nr_in_flight, plus three small helpers:
- cgroup_writeback_pin(sb) - increment counter
- cgroup_writeback_unpin(sb) - decrement and wake drainer if last
- cgroup_writeback_drain(sb) - wait for counter to reach zero
The wiring is:
- inode_prepare_wbs_switch() pins before checking SB_ACTIVE and
grabbing the inode; failure paths unpin before returning. A
lockless SB_ACTIVE check at the top of the function lets us skip
the atomic_inc/smp_mb dance once SB_ACTIVE has been cleared (it
is monotonic and never set back).
- process_inode_switch_wbs() unpins after the matching iput().
- cgroup_writeback_umount() drains the per-sb counter via
wait_var_event().
The smp_mb() pair between inode_prepare_wbs_switch() and
cgroup_writeback_umount() keeps the SB_ACTIVE / counter ordering:
either the umounter sees a non-zero counter and waits, or the
switcher sees SB_ACTIVE cleared and aborts before grabbing the
inode.
The global isw_nr_in_flight is left in place, since it is still used
to throttle in-flight switches via WB_FRN_MAX_IN_FLIGHT.
The rcu_read_lock() extension in inode_switch_wbs() and
cleanup_offline_cgwb() that the race fix added is no longer needed
and is reverted; the synchronize_rcu() that the race fix added to
cgroup_writeback_umount() is dropped as well.
The following numbers were measured on a 16 vCPU QEMU guest with 4
background superblocks each churning "create memcg -> write 1 MiB ->
rmdir memcg" to keep the global isw_nr_in_flight non-zero. Latencies
are wall-clock around umount(8); only the target sb's umount is
measured.
Target sb runs its own cgwb churn:
p50 p95 p99 max
global synchronize_rcu() 67.6 ms 88.3 ms 88.3 ms 96.8 ms
per-sb counter (this) 7.9 ms 10.0 ms 10.0 ms 10.1 ms
Idle target umount latency under cross-sb cgwb-switch pressure:
p50 p95 p99 max
global synchronize_rcu() 62.7 ms 95.4 ms 108.1 ms 108.6 ms
per-sb counter (this) 5.3 ms 6.9 ms 7.4 ms 7.4 ms
no-pressure baseline 4.9 ms 5.9 ms 6.3 ms 6.7 ms
8 concurrent umounts of idle sbs under the same pressure:
p50 p95 max
global synchronize_rcu() 61.3 ms 99.5 ms 113.7 ms
per-sb counter (this) 8.1 ms 9.1 ms 9.5 ms
In-kernel cgroup_writeback_umount() time across the same run
(bpftrace, ~340 calls covering all scenarios):
global synchronize_rcu() 12371 ms total (~36 ms / call)
per-sb counter (this) 1.37 ms total ( ~4 us / call)
Suggested-by: Christian Brauner <brauner@kernel.org>
Link: https://lore.kernel.org/r/177910456953.488929.2169908940676707307.b4-review@b4
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Baokun Li <libaokun@linux.alibaba.com>
Link: https://patch.msgid.link/20260521095016.2791354-4-libaokun@linux.alibaba.com
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Commit e1b849cfa6b6 ("writeback: Avoid contention on wb->list_lock when
switching inodes") replaced the queue_rcu_work() based scheduling of
inode wb switches with a plain queue_work(). Since then no switcher
goes through call_rcu(), so rcu_barrier() in cgroup_writeback_umount()
has no callbacks of its own to wait for. It still drains unrelated
call_rcu() callbacks from other subsystems on busy systems, which
incidentally slows umount down; drop it.
Fixes: e1b849cfa6b6 ("writeback: Avoid contention on wb->list_lock when switching inodes")
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Baokun Li <libaokun@linux.alibaba.com>
Link: https://patch.msgid.link/20260521095016.2791354-3-libaokun@linux.alibaba.com
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
When a container exits, the following BUG_ON() is occasionally triggered:
==================================================================
VFS: Busy inodes after unmount of sdb (ext4)
------------[ cut here ]------------
kernel BUG at fs/super.c:695!
CPU: 3 PID: 6 Comm: containerd-shim Tainted: G OE K 6.6 #1
pstate: 63400009 (nZCv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
pc : generic_shutdown_super+0xf0/0x100
lr : generic_shutdown_super+0xf0/0x100
Call trace:
generic_shutdown_super+0xf0/0x100
kill_block_super+0x20/0x48
ext4_kill_sb+0x28/0x60
deactivate_locked_super+0x54/0x130
deactivate_super+0x84/0xa0
cleanup_mnt+0xa4/0x140
__cleanup_mnt+0x18/0x28
task_work_run+0x78/0xe0
do_notify_resume+0x204/0x240
==================================================================
The root cause is a race between cgroup_writeback_umount() and
inode_switch_wbs()/cleanup_offline_cgwb(). There is a window between
inode_prepare_wbs_switch() returning true and the subsequent
wb_queue_isw() call. Following is the process that triggers the issue:
CPU A (umount) | CPU B (writeback)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
inode_switch_wbs/cleanup_offline_cgwb
atomic_inc(&isw_nr_in_flight)
inode_prepare_wbs_switch
-> passes SB_ACTIVE check
__iget(inode)
generic_shutdown_super
sb->s_flags &= ~SB_ACTIVE
cgroup_writeback_umount(sb)
smp_mb()
atomic_read(&isw_nr_in_flight)
rcu_barrier()
-> no pending RCU callbacks
flush_workqueue(isw_wq)
-> nothing queued, returns
evict_inodes(sb)
-> Inode skipped as isw still holds a ref.
sop->put_super(sb)
/* destroys percpu counters */
-> VFS: Busy inodes after unmount!
wb_queue_isw()
queue_work(isw_wq, ...)
/* later in work function */
inode_switch_wbs_work_fn
process_inode_switch_wbs
iput() -> evict
percpu_counter_dec() // UAF!
Fix this by extending the RCU read-side critical section in
inode_switch_wbs() and cleanup_offline_cgwb() to cover from
inode_prepare_wbs_switch() through wb_queue_isw(). Since there is
no sleep in this window, rcu_read_lock() can be used. Then add a
synchronize_rcu() in cgroup_writeback_umount() before the existing
rcu_barrier(), so that all in-flight switchers that have passed the
SB_ACTIVE check have completed queue_work() before flush_workqueue()
is called.
The existing rcu_barrier() is intentionally retained so this fix can
be backported unchanged to stable kernels (5.10.y, 6.6.y, ...) that
still queue switches via queue_rcu_work(). It is a no-op on current
mainline (since commit e1b849cfa6b6 ("writeback: Avoid contention on
wb->list_lock when switching inodes")) and is removed in a follow-up
patch.
Fixes: a1a0e23e4903 ("writeback: flush inode cgroup wb switches instead of pinning super_block")
Cc: stable@vger.kernel.org
Suggested-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/all/mxnjq2l6guusfchvauxr3v7c4bwjasybxlleqbbh4efloeqspz@iqylk76ohufz
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Baokun Li <libaokun@linux.alibaba.com>
Link: https://patch.msgid.link/20260521095016.2791354-2-libaokun@linux.alibaba.com
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Pull vfs fixes from Christian Brauner:
- eventpoll: fix ep_remove() UAF and follow-up cleanup
- fs: aio: set VMA_DONTCOPY_BIT in mmap to fix NULL-pointer-dereference
error
- writeback: Fix use after free in inode_switch_wbs_work_fn()
- fuse: reject oversized dirents in page cache
- fs: aio: reject partial mremap to avoid Null-pointer-dereference
error
- nstree: fix func. parameter kernel-doc warnings
- fs: Handle multiply claimed blocks more gracefully with mmb
* tag 'vfs-7.1-rc1.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
eventpoll: drop vestigial epi->dying flag
eventpoll: drop dead bool return from ep_remove_epi()
eventpoll: refresh eventpoll_release() fast-path comment
eventpoll: move f_lock acquisition into ep_remove_file()
eventpoll: fix ep_remove struct eventpoll / struct file UAF
eventpoll: move epi_fget() up
eventpoll: rename ep_remove_safe() back to ep_remove()
eventpoll: drop vestigial __ prefix from ep_remove_{file,epi}()
eventpoll: kill __ep_remove()
eventpoll: split __ep_remove()
eventpoll: use hlist_is_singular_node() in __ep_remove()
fs: Handle multiply claimed blocks more gracefully with mmb
nstree: fix func. parameter kernel-doc warnings
fs: aio: reject partial mremap to avoid Null-pointer-dereference error
fuse: reject oversized dirents in page cache
writeback: Fix use after free in inode_switch_wbs_work_fn()
fs: aio: set VMA_DONTCOPY_BIT in mmap to fix NULL-pointer-dereference error