| Age | Commit message (Collapse) | Author |
|---|
|
Add a new BLOCK_URING_CMD_ZONE_RESET_ALL uring cmd to reset all zones
for a given block device. This can be used by storage systems or
file system mkfs tools to initialize multiple devices in parallel.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Link: https://patch.msgid.link/20260804125038.740388-7-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Changhuang Liang <changhuang.liang@starfivetech.com> says:
This serial add support for the StarFive JHB100 SoC SPI Flash
Controller (SFC), which is based on the Synopsys DesignWare SSI
version 2.00a but with some customizations and it also add
enhanced SPI for DesignWare SPI controllers.
I picked up some patches from series [1].
This series depends on the series [2]:
[1] https://lore.kernel.org/all/20221212180732.79167-1-sudip.mukherjee@sifive.com/
[2] https://lore.kernel.org/all/20260521012932.24163-1-changhuang.liang@starfivetech.com/
v1: https://lore.kernel.org/all/20260709055204.138168-1-changhuang.liang@starfivetech.com/
Link: https://patch.msgid.link/20260803124044.156998-1-changhuang.liang@starfivetech.com
|
|
Slabs of some caches never need the objcg part of struct slabobj_ext.
Introduce helpers to query this for a cache or a slab.
Introduce SLAB_MAY_ACCOUNT flag that is currently only internal and all
caches have it set except:
- KMALLOC_NORMAL caches, as long as KMALLOC_RECLAIM caches are separate
- KMALLOC_NO_OBJ_EXT caches, if they exist
For named caches we currently can't derive SLAB_MAY_ACCOUNT from
SLAB_ACCOUNT because some caches might be created without SLAB_ACCOUNT
and then used both with and without __GFP_ACCOUNT concurrently,
allocating obj_ext arrays on demand. So just add the SLAB_MAY_ACCOUNT
to all kmem caches, unless kmem accounting is disabled.
This can be improved later by finding out all caches used with
__GFP_ACCOUNT, creating them with the SLAB_MAY_ACCOUNT flag explicitly,
and then ignoring __GFP_ACCOUNT for all other caches (possibly with a
warning).
To make the evaluation of slab_needs_objcg() faster in the allocation
and free fast paths, add a obj_exts_needs_objcg flag into slab itself.
This optimization is only available on 64bit architectures where free
bits are available for the flag.
Reviewed-by: Hao Li <hao.li@linux.dev>
Link: https://patch.msgid.link/20260727-b4-objext_split-v3-11-c29ef0f1f257@kernel.org
Reviewed-by: Harry Yoo <harry@kernel.org>
Signed-off-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
|
|
Currently, k[v]free_rcu() cannot be called in unknown context since
it could lead to a deadlock when called in the middle of k[v]free_rcu().
Make users' lives easier by introducing kfree_rcu_nolock() variant,
now that kfree_rcu_sheaf() is available on PREEMPT_RT and
__kfree_rcu_sheaf() handles unknown context.
When sheaves path fails, kfree_rcu_nolock() falls back to
defer_kfree_rcu() that uses an irq work to free the object via
kvfree_call_rcu(). In most cases, the sheaves path is expected to
succeed and therefore it's unnecessary to introduce additional
complexity to the existing kvfree_rcu batching by teaching it
how to handle unknown context.
Since defer_kfree_rcu() can be called on caches without sheaves, move
deferred_work_barrier() and rcu_barrier() outside the branch in
kvfree_rcu_barrier_on_cache().
Now that deferred kvfree_rcu objects are submitted to kvfree_call_rcu()
after deferred_work_barrier() and may end up in RCU sheaves,
deferred_work_barrier() must be invoked before
flush_rcu_sheaves_on_cache().
Since the RCU sheaf path has not been used on !KVFREE_RCU_BATCHED
kernels, always fall back when kvfree_rcu() is not batched, for
consistency. kvfree_rcu_barrier{,_on_cache()}() on
!KVFREE_RCU_BATCHED are moved to mm/slab_common.c to invoke
deferred_work_barrier() before rcu_barrier().
Signed-off-by: Harry Yoo (Oracle) <harry@kernel.org>
Link: https://patch.msgid.link/20260729-kfree_rcu_nolock-v5-7-a28cdcda9673@kernel.org
Signed-off-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
|
|
rcu_head is overkill for kvfree_rcu() because the callback
function is always either kfree(), vfree(), or free_large_kmalloc(),
and thus there is no need for a function pointer.
kvfree_rcu batching reuses the field to store the start address
of an object, however, this is not strictly needed because we can
calculate the start address in the slowpath. For the purpose of
kvfree_rcu batching, it is sufficient to implement a linked list using
a single pointer.
Introduce a new struct called kvfree_rcu_head (the name was suggested
by Vlastimil Babka), which is similar to rcu_head but is only a single
pointer to build a linked list, without a function pointer, when
CONFIG_KVFREE_RCU_BATCHED=y.
When kvfree_rcu is not batched, kvfree_rcu_head is the same size
as rcu_head. Note that shrinking struct kvfree_rcu_head on
CONFIG_KVFREE_RCU_BATCHED=n kernels would inevitably require additional
complexity and also some sort of batching (which defeats the purpose of
the config option) because it cannot fall back to call_rcu().
For now there are no user-visible changes to the API. k[v]free_rcu()
simply casts rcu_head to kvfree_rcu_head. While this does not affect
the API, it allows kfree_rcu_nolock() to reuse kvfree_rcu batching
as a fallback when trylock or sheaf allocation fails.
Stop storing the object pointer in rcu_head.func and instead calculate
the object's start address in kvfree_rcu_list(). Factor out the existing
logic to calculate the start address from kvfree_rcu_cb() to
kvmalloc_obj_start_addr(). To avoid losing the KASAN tag, calculate
the offset and subtract it from the address of the kvfree_rcu_head.
Signed-off-by: Harry Yoo (Oracle) <harry@kernel.org>
Link: https://patch.msgid.link/20260729-kfree_rcu_nolock-v5-6-a28cdcda9673@kernel.org
Signed-off-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
|
|
|
|
Similar to memblock_reserved_kern_size(), but calculates only the memory
reserved for hugetlb pages. This is needed in an upcoming commit that
subtracts hugetlb reservation size when computing the size of KHO
scratch areas.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-22-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
Gigantic huge page allocation is somewhat broken currently when KHO is
used.
Firstly, they break KHO scratch size accounting. RSRV_KERN is used to
track how much memory is reserved for use by the kernel. Since
hugetlb::alloc_bootmem() calls the memblock_alloc*() APIs, the hugepages
allocated also get marked as RSRV_KERN.
Allocations marked RSRV_KERN are used by KHO to calculate how much
scratch space it should reserve to make sure the next kernel has enough
memory to boot when it is in scratch-only phase. Counting hugepages in
that blows up scratch size, and can lead to the scratch allocation
failing, making KHO unusable. This will show up when huge pages make up
more than 50% of the system, which is a fairly common use case.
Secondly, while not supported right now, huge pages are user memory and
can be preserved via KHO. The scratch spaces should not have any
preserved memory. Allocating hugepages from scratch (on a KHO boot) can
lead to them being un-preservable.
Introduce memblock_alloc_hugetlb(). This lets memblock tailor to the
needs of hugetb without exposing those details to the general allocation
routines.
First, it does not use mirrored memory for hugetlb. Mirrored memory is a
limited resource that is best saved for kernel data structures, not user
memory.
Second, if the free memory area found by memblock_find_in_range_node()
is a part of a KHO scratch area, the free area is not used. Allocation
is retried starting after the free area to ensure no hugepages come from
KHO scratch.
Third, it simplifies the argument list by baking in some hugetlb
assumptions like alignment and exact_nid. This also simplifies
allocation logic in alloc_bootmem().
Also introduce MEMBLOCK_RSRV_HUGETLB to mark reservations made for
HugeTLB. This will be used by KHO in future patches to correctly
calculate scratch sizes.
Refactor some of the preparation logic like kmemleak tracking and
accepting memory into a separate helper memblock_prep_allocation(), and
use it from both memblock_alloc_hugetlb() and the usual
memblock_alloc_range_nid().
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-21-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
Currently struct page init via memmap_init() or deferred_init_memmap()
only queries the migrate type from KHO for each discrete memory range.
That works currently since KHO scratch memory has a different memory
type so it is always it its own region.
An upcoming patch will add support for discovering blocks of memory with
no preservations and it will mark it as MEMBLOCK_KHO_SCRATCH to allow
allocations from them. This can lead to the bootmem KHO scratch areas to
be merged into larger free ranges. This merging breaks the selection of
migrate type.
Get rid of memblock_is_kho_scratch_memory(). Instead, use
kho_scratch_overlap() to decide the migrate type of the PFN. Since
kho_scratch_migratetype() only uses KHO functions, move it to
kexec_handover.h.
Instead of calling kho_scratch_migratetype() once for each free range,
call it once for each pageblock. Update
pageblock_migratetype_init_range() and memmap_init_range() to do so.
Since the migrate type is now evaluated for each pageblock and not each
free range, drop the migratetype arguments to deferred_free_pages() and
memmap_init_zone_range() and use MIGRATE_MOVABLE directly.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-18-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
In a future patch, mm init will use kho_scratch_overlap() for deciding
the migrate type of pageblocks it initializes. The earliest user
currently is free_area_init(). kho_scratch_overlap()
relies on kho_scratch pointer being initialized. Introduce
kho_memory_init_early() to do this.
kho_populate() would normally be a good place to do this, but
unfortunately, phys_to_virt() does not work at that point on ARM64. So
we need yet another initialization function.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-15-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
Support for discovering memory blocks with no preserved memory will be
added in coming patches. These areas will also be marked as scratch to
allow allocations from them. Memblock will switch to looking through the
scratch array to decide the right migratetype.
Expose kho_scratch_overlap() to KHO users. Since it is now used by
non-debug code, move it out of kexec_handover_debug.c and into
kexec_handover.c. Gate the overlap checks in kho_preserve_folio() and
kho_preserve_pages() by IS_ENABLED(CONFIG_KEXEC_HANDOVER_DEBUG) instead.
Since kexec_handover_debug.c is now empty, delete it.
Add a stub for kho_scratch_overlap() to memblock tests to make sure it
compiles. It will be used in memblock by a coming commit.
No functional changes.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-14-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
Move the initialization logic of the radix tree into
kho_radix_init_tree() instead of having users open-code it. Makes the
boundaries cleaner and reduces code duplication when a new user of the
radix tree will be added in a future commit.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-13-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
Add kho_radix_destroy_tree() which allows destroying the radix tree and
freeing all its pages.
This is will be used by the upcoming scratch extension mechanism. It
creates a radix tree to track free blocks and then frees them after
telling memblock about them.
Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-12-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
Add an opaque data pointer argument to kho_radix_walk_cb_t. This can be
used by callers to pass extra information to the callback.
Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-10-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
The KHO memory preservation radix tree does not mark the table pages
themselves as preserved. This is done to avoid a circular dependency
where preserving a page can lead of allocating other preserved pages.
This means any walker looking for free ranges of memory outside of
scratch areas will ignore the table
Add a table callback that is invoked for each table page. The callback
is given the physical address of the table page.
This is useful for the upcoming mechanism that discovers blocks of
memory with no preserved pages and lets them be used for boot memory.
Another use case is for users of the radix tree other than KHO itself.
The radix tree does not preserve its own pages due to the circular
dependency described above. But external users of the radix tree would
need to preserve and restore their pages for the radix tree to survive
past early boot. They can use this callback to do so.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-9-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
A future commit will add more callbacks for the KHO radix tree. Add a
struct for collecting the callbacks.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-8-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
The extern declaration of taskstats_exit_mutex has never been defined nor
referenced anywhere now. Just remove it.
Link: https://lore.kernel.org/98948e69094b73d6dfa63dcf0770067b57f3becf.1783435695.git.cyyzero16@gmail.com
Signed-off-by: Yiyang Chen <cyyzero16@gmail.com>
Cc: Balbir Singh <balbirs@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
If userspace holds its file descriptor open, it can call PPS_KC_BIND on a
device which has been unplugged, leaving pps_kc_hardpps_dev as a dangling
pointer after close().
After that sequence, PPS_KC_BIND is broken until the system is rebooted,
because the pointer comparison in pps_kc_bind() can never be true.
calling pps_ktimer_init+0x0/0x1000 [pps_ktimer] @ 1081
initcall pps_ktimer_init+0x0/0x1000 [pps_ktimer] returned 0 after 811 usecs
pps pps0: bound kernel consumer: edge=0x1
pps pps0: unbound kernel consumer on device removal
pps pps0: bound kernel consumer: edge=0x1
calling pps_ktimer_init+0x0/0x1000 [pps_ktimer] @ 1085
initcall pps_ktimer_init+0x0/0x1000 [pps_ktimer] returned 0 after 340 usecs
pps pps0: another kernel consumer is already bound
Here is a short reproducer, which uses rmmod of the pps-ktimer testcase
to simulate a device being unplugged:
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <linux/pps.h>
#include <errno.h>
#include <err.h>
int main(void)
{
while (1) {
int fd;
if (system("insmod ./pps-ktimer.ko"))
err(1, "insmod failed");
fd = open("/dev/pps0", O_RDWR);
if (fd == -1)
err(1, "open failed");
struct pps_bind_args args = {
.tsformat = PPS_TSFMT_TSPEC,
.edge = PPS_CAPTUREASSERT,
.consumer = PPS_KC_HARDPPS,
};
if (ioctl(fd, PPS_KC_BIND, &args))
err(1, "first PPS_KC_BIND failed");
if (system("rmmod pps-ktimer"))
err(1, "rmmod failed");
if (ioctl(fd, PPS_KC_BIND, &args)) {
if (errno != ENODEV)
err(1, "second PPS_KC_BIND failed");
else
puts("Got ENODEV, kernel is patched");
}
close(fd);
}
}
Fix this by setting a flag when the device is unplugged, returning -ENODEV
from PPS_KC_BIND if the flag is set.
For userspace to encounter this new behavior, it must do something which
breaks the interface today, so this fix shouldn't cause any observable
behavior change for working programs.
Link: https://lore.kernel.org/672778c177ac9b6fdcb445e35c97ac4ca7d1149f.1780506611.git.calvin@wbinvd.org
Signed-off-by: Calvin Owens <calvin@wbinvd.org>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/cover.1779733602.git.calvin%40wbinvd.org?part=1
Acked-by: Rodolfo Giometti <giometti@enneenne.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Fix misspellings of "softlockup" in the watchdog enabled bit definitions
and related comments. Also fix a nearby "successful" typo.
No functional change.
Link: https://lore.kernel.org/20260615174557.1836562-1-edcr1790@gmail.com
Signed-off-by: Matthew Chen <edcr1790@gmail.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
ublk passes the parameters of incoming I/O in memory shared between the
kernel ublk driver and userspace ublk server in struct ublksrv_io_desc.
The size of this struct is currently fixed to 24 bytes, which has been
an obstacle to extending it with additional fields [1]. Additionally,
with multiple ublk server threads handling I/Os from the same ublk queue
(possible with UBLK_F_PER_IO_DAEMON or UBLK_F_BATCH_IO), false sharing
results from adjacent io_descs sharing the same cache line.
Add a ublk feature UBLK_F_IO_DESC_SIZE to allow a ublk server to
override the size of each io_desc. The size must be at least 24 and a
multiple of 8 to store a properly-aligned struct ublksrv_io_desc. It's
also limited to a maximum of 256, though this bound could be lifted in
the future.
The struct ublksrv_io_desc is located at the beginning of each io_desc
and the remainder is padding. The mmap() performed for each queue must
have a length of queue_depth * io_desc_size rounded up to the page size.
The mmap() offset must be q_id * UBLK_MAX_QUEUE_DEPTH * io_desc_size,
also rounded up to the page size.
[1]: https://lore.kernel.org/linux-block/aV8QfvaNO5P6vOs6@fedora/
Suggested-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Link: https://patch.msgid.link/20260803211441.2538144-6-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
On some ARM64 platforms with 4K PAGE_SIZE, page_pool fragment
allocation in the RX refill path can cause 15-20% throughput
regression under high connection counts (>16 TCP streams).
Add an ethtool private flag "full-page-rx" that allows the user to
force one RX buffer per page, bypassing the page_pool fragment path.
This restores line-rate (180+ Gbps) performance on affected platforms.
Usage:
ethtool --set-priv-flags eth0 full-page-rx on
There is no behavioral change by default. The flag must be explicitly
enabled by the user or udev rule.
The existing single-buffer-per-page logic for XDP and jumbo frames is
consolidated into a new helper mana_use_single_rxbuf_per_page() which
is now the single decision point for both the automatic and
user-controlled paths.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: Dipayaan Roy <dipayanroy@linux.microsoft.com>
Link: https://patch.msgid.link/20260729063347.3388035-3-dipayanroy@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
The f2fs_iostat tracepoint stores the per-order read folio counts in a
fixed-size array and prints a fixed number of buckets, both hardcoded to
11. The sysfs iostat accounting array is instead sized by NR_PAGE_ORDERS
(= MAX_PAGE_ORDER + 1), which is not always 11:
arm64 16K pages -> MAX_PAGE_ORDER 11 -> NR_PAGE_ORDERS 12
arm64 64K pages -> MAX_PAGE_ORDER 13 -> NR_PAGE_ORDERS 14
f2fs enables large folios for immutable, non-compressed files, and the
read folio order is bounded by MAX_PAGECACHE_ORDER, i.e.
min(MAX_XAS_ORDER, PREFERRED_MAX_PAGECACHE_ORDER). With THP enabled this
reaches order 11 on 16K/64K base-page kernels (MAX_XAS_ORDER caps it at
11). So an order-11 read folio is possible there and is accounted into
index 11 of the array.
On those configurations the sysfs file reports the order-11 count
correctly, but the tracepoint silently drops it: the memcpy is capped at
min(NR_PAGE_ORDERS, 11), so index 11 is never copied and the trace
disagrees with sysfs. There is no memory-safety issue, only the order-11
bucket missing from the trace; 4K-page kernels (NR_PAGE_ORDERS == 11,
max order <= 9) are unaffected.
Size the array and the printed buckets by a ceiling that covers the
largest possible NR_PAGE_ORDERS (14) with headroom, and add a
BUILD_BUG_ON() so any future growth of NR_PAGE_ORDERS fails the build
loudly instead of silently truncating again. The human-readable
"order=count" output is preserved.
Fixes: cb8ff3ead9a3 ("f2fs: add page-order information for large folio reads in iostat")
Cc: stable@vger.kernel.org
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
The zero-copy path validates TX metadata while obtaining the descriptor
context, then reads it again later when preparing the hardware request.
User space can change the metadata between those operations and bypass the
original validation.
Validate the metadata in xsk_tx_metadata_request() and use the resulting
flags snapshot for every feature check. Read request fields once so all
zero-copy drivers process only values observed after successful
validation.
Fixes: ca4419f15abd ("xsk: Add launch time hardware offload support to XDP Tx metadata")
Cc: Cen Zhang (Microsoft) <blbllhy@gmail.com>
Signed-off-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20260727161959.885642-7-sdf@fomichev.me
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
xsk_tx_metadata_request() must validate metadata with
xsk_buff_valid_tx_metadata(), which is defined in xdp_sock_drv.h. Move the
helper there before adding that dependency. All callers already include
the destination header, so this has no functional effect.
Fixes: ca4419f15abd ("xsk: Add launch time hardware offload support to XDP Tx metadata")
Cc: Cen Zhang (Microsoft) <blbllhy@gmail.com>
Signed-off-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20260727161959.885642-6-sdf@fomichev.me
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Launch-time metadata extends beyond the first 16 bytes of struct
xsk_tx_metadata. Reject the request when the registered metadata area does
not contain the complete field.
Snapshot the validated flags for the generic transmit path and use that
snapshot for request and completion processing, avoiding inconsistent
decisions if user space changes the flags concurrently.
Note that only xsk_skb_metadata is properly using the flags,
__xsk_buff_get_metadata ignores them. Next commits address that.
Fixes: ca4419f15abd ("xsk: Add launch time hardware offload support to XDP Tx metadata")
Cc: Cen Zhang (Microsoft) <blbllhy@gmail.com>
Signed-off-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20260727161959.885642-5-sdf@fomichev.me
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
User space can change metadata flags after request processing. Rereading
them during completion can therefore make the kernel write a timestamp
that was not requested when the packet was submitted.
Clear the metadata pointer during request processing unless timestamp
completion is requested. Completion handling can then use the pointer
itself instead of rereading the flags.
On the mlx5 multi-packet WQE path metadata is evaluated per batch:
xsk_tx_metadata_request() runs only for the descriptor that starts a
session, just like the checksum offload that is applied once through the
shared WQE. Only that descriptor's pointer is reset, so completion
handling can record a timestamp for the other descriptors of the session
regardless of their own XDP_TXMD_FLAGS_TIMESTAMP bit. The write stays
inside the metadata area; the single-WQE, other zero-copy, and generic
paths reset the pointer per descriptor and are unaffected.
Fixes: ca4419f15abd ("xsk: Add launch time hardware offload support to XDP Tx metadata")
Cc: Cen Zhang (Microsoft) <blbllhy@gmail.com>
Signed-off-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20260727161959.885642-4-sdf@fomichev.me
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Completion handling needs to know whether a timestamp was requested when
the metadata was processed. Let xsk_tx_metadata_request() update the
caller's metadata pointer so that decision can be carried forward without
rereading user-controlled flags.
This only changes the interface; behavior remains unchanged.
Fixes: ca4419f15abd ("xsk: Add launch time hardware offload support to XDP Tx metadata")
Cc: Cen Zhang (Microsoft) <blbllhy@gmail.com>
Signed-off-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20260727161959.885642-3-sdf@fomichev.me
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
An 'F' entry, and every interpreter a 'B' entry binds, holds a file open
from registration until the entry goes away, pinning the file, its inode,
the mount it came from and that mount's superblock. Nothing bounds how
many of those a user namespace can hold. An entry binds at most
BINFMT_MISC_INTERP_MAX interpreters, but nothing caps the entries.
Charge each binding to the user namespace and uid that makes it against a
new UCOUNT_BINFMT_MISC_INTERPRETERS. Going over budget causes -ENOSPC.
A per-instance cap would suck. Instances are keyed on the user
namespace. So any constant is multiplied by the number of namespaces the
caller creates. Creating those is virtually free. A ucount charges the
namespace and every one of its ancestors. And a namespace can raise only
its own limit. So nesting buys nothing.
The knob is /proc/sys/user/max_binfmt_misc_interpreters. Leave it at the
max_threads/2 default fork_init() gives a new type. No existing
configuration comes close to that.
binfmt_misc is tristate, which makes it the first ucount user that can be
built as a module. Export inc_ucount() and dec_ucount(); without them
CONFIG_BINFMT_MISC=m fails to link. Export them to binfmt_misc alone:
charging a ucount type is not something a module has any business doing
in general, and the list is trivial to extend if a second user shows up.
init_user_ns and init_binfmt_misc are already exported for the same
module.
Link: https://patch.msgid.link/20260803-work-binfmt_misc-interplimit-v1-1-4a2435500bd9@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
When rescue demand on a cpu persistently exceeds the configured bandwidth,
tasks age on that cpu's rescue DSQ until the stall watchdog fires. The
watchdog blames the waiting task's owner, but the misbehaving party is
whoever floods the queue, not whoever happens to time out.
Track each sched's recent rescue consumption per cpu as a decaying average.
Once the oldest waiter on a cpu's rescue DSQ has been queued past a
threshold derived from the rescue knobs (4s at the defaults), the rescue
timer ejects the sub with the highest recent consumption on that cpu with
SCX_EXIT_ERROR_RESCUE. With no recent consumer there is no victim and
nothing is ejected - the generic stall watchdog eventually blames the
waiter's owner instead. Ejections on a cpu are spaced one threshold apart so
the freed bandwidth can drain the backlog before another sub is judged.
The overload check only wins the race against the stall watchdog when the
watchdog timeout clears the threshold, and a single in-budget wait must not
cross the trigger on its own. Warn on a scheduler whose timeout doesn't fit
and on knobs whose funding period exceeds half the threshold.
v2: - Track kill_at in jiffies_64 - on 32-bit, the time_before() grace check
wraps 2^31 ticks after the last ejection and suppresses ejections.
(sashiko AI)
- Track rescue_avg_at in jiffies_64 likewise - the unsigned long decay
delta truncates mod 2^32 on 32-bit and can revive a weeks-old usage
average in the victim pick.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A local DSQ insert lacking the needed caps is diverted to the reject DSQ and
bounced back through ops.enqueue() so the scheduler can re-decide. That
recovery assumes the scheduler has somewhere legal to send the task. When it
doesn't, e.g. when the task's affinity is restricted to cids delegated away,
the task starves until the stall watchdog ejects the scheduler. An exiting
task is worse - it skips ops.enqueue() and the rejection becomes a
self-requeuing cycle that burns the CPU until the watchdog fires.
Add SCX_ENQ_RESCUE, a fallback modifier on local DSQ inserts. When the
insert would be rejected for missing caps, the kernel takes over and runs
the task on the target CPU without consulting the owning scheduler. The
kernel sets the flag itself when enqueueing an exiting task.
Rescue is a last-resort forward-progress backstop with a persistent
disadvantage, not a way around cap enforcement. A per-CPU token bucket
accrues rescue_bandwidth_ppt (default 2%) of CPU time and rescues run one at
a time in arrival order. Each is granted a slice of the rescue_quantum_us
(default 5ms) quantum divided across the waiters, waits at the tail of the
local DSQ claiming no priority, and rejoins its scheduler as a fresh arrival
once the slice is served.
The schedulers keep their normal control over an admitted rescuee and may
preempt or reslice it. Service is measured on CPU time actually received, so
neither shortens the rescue. Prolonged denial escalates - the remaining
slice turns into protected execution (SCX_TASK_PROTECTED) and the rescuee
preempts the current task. Escalation is paced by the same bucket, and
delivered service converges on the configured bandwidth no matter how
aggressively the schedulers dispatch.
Both knobs are root-only and SCX_RESCUE_DISABLE turns rescue off, making
SCX_ENQ_RESCUE inserts reject as usual.
v2: - Add SCX_OPS_OPEN() fix-ups for the new ops fields so cpu-form
schedulers setting them still load on older kernels. (Andrea)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A BPF scheduler can displace any of its tasks at will - cut a running one's
slice with an SCX_ENQ_PREEMPT dispatch, an SCX_KICK_PREEMPT kick or a direct
shortening, and jump a queued one with HEAD insertions. Sometimes the kernel
needs a slice and a DSQ position to stick regardless.
Add SCX_TASK_PROTECTED, guarding both:
- The slice becomes immutable. Every scheduler-reachable write is refused
and counted as SCX_EV_SLICE_DENIED. Higher scheduling classes are
unaffected. PREEMPT|IMMED can't preempt a running protected task and gets
reenqueued.
- A protected task that reached the head of its DSQ keeps it - HEAD
insertions land behind the leading run of protected tasks and reenqueue
sweeps skip them. Only rq-owned DSQs can hold protected tasks, so the walk
runs only for them.
The bit lives in p->scx.flags so that both the refusal and the head walk
read it under the rq lock that protects it.
Protection ends when the slice is consumed, when the task leaves the rq
except for a save/restore on the running task, on a yield, when the
scheduler enters bypass, and when the task leaves scx. The flag is
kernel-internal and not used yet.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
p->scx.slice and p->scx.dsq_vtime writes have no synchronization rules. The
dsq insert kfuncs write both fields synchronously from whatever context
they're called in - a direct dispatch from ops.select_cpu() writes with only
pi_lock held - and, as the kfuncs are safe to call spuriously with the
invalid dispatch discarded later, a scheduler can modify any task's slice by
spuriously calling them. The latter stands in the way of an upcoming patch
which adds kernel-granted slices that the schedulers must not be able to
modify.
Give both fields explicit rules. While the task is running, sleeping or
queued on an rq-owned DSQ, the rq lock protects them - these are the states
where the kernel consumes the slice. While queued on a user DSQ or on the
BPF side, the kernel neither consumes nor decides on the fields and every
writer acts for the BPF scheduler - synchronizing the writers is the
scheduler's responsibility and whichever write lands last wins.
To conform, an insert kfunc no longer writes the fields when called. The
values travel with the dispatch and take effect when the task is inserted. A
discarded dispatch has no side effects. The rq lock rule is asserted at the
slice store.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The KHO radix tree constants are somewhat hard to understand. The tree
depth essentially comes from the max key width. The max key width comes
from the need to store a 52-bit PFN plus one more bit for the order.
All this is very obscure with the corrent code. The PFN width is defined
as KHO_ORDER_0_LOG2, which makes very little sense to a new reader not
already familiar with what the value means. Then the fact that an extra
bit is needed is hidden in the KHO_TREE_MAX_DEPTH calculation.
Simplify this by removing KHO_ORDER_0_LOG2 and replace it with
KHO_RADIX_KEY_WIDTH. Update the comment to explain why this value is
used. This moves the +1 from KHO_TREE_MAX_DEPTH to KHO_RADIX_KEY_WIDTH,
making things clearer.
Update kho_{encode,decode}_radix_key() to not use KHO_ORDER_0_LOG2.
Instead, refactor the code and comments to make it clearer how the
encoding and decoding is done.
In kho_encode_radix_key(), add a new variable for the shift for physical
address. Use that in calculating where the order bit goes and in
calculating the shifted PFN. Update comments to make this clearer.
In kho_radix_decode_key(), turn order_bit to 0-indexed to simplify the
eventual calculation for order. Touch up comments to make the
computation clearer.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-3-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
The KHO radix tree is a data structure that can track the presence or
absence of an arbitrary key, with nothing inherently tied to KHO memory
preservation tracking. This was one of the design goals of the radix
tree. This was done to enable it to be re-used by other users of KHO.
Despite that, the radix tree APIs are very closely tied to KHO memory
preservation tracking. Adding a key is done by kho_radix_add_page(),
which encodes it as a page tracking operation and takes in PFN and
order. kho_radix_del_page() does the same. These functions encode the
key internally that goes into the radix tree. kho_radix_walk_tree() does
the same by baking the PFN and order into the callback arguments.
Generalize the APIs by taking the key directly and doing the encoding at
the callers. Rename the functions to kho_radix_add_key() and
kho_radix_del_key(). In practice, this removes a line each from the
functions and moves the encoding function call to the callers.
Similarly, update kho_radix_tree_walk_callback_t to take the key
directly.
Now that key encoding is no longer an inherent part of the radix tree
and can be decided by the user, rename kho_radix_{encode,decode}_key()
to kho_{encode,decode}_radix_key(). This moves them out of the
"kho_radix_" name space into the "kho_" namespace. This emphasizes that
this is KHO's way of encoding the key for its radix tree.
Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-2-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
hidinput_query_battery_capacity() assumes the state-of-charge value is
the first byte following the report ID (buf[1]) and ignores where the
battery field actually sits within the report.
An Apple Magic Trackpad 2 precedes the AbsoluteStateOfCharge byte with a
byte of status flags in its battery reports, so this query returns the
flags byte instead of the charge level.
The device happens to make that easy to observe, because it exposes the
same cell twice: its report descriptor declares AbsoluteStateOfCharge in
two reports (0x90 and 0x9b), so hidinput_setup_battery() registers two
power supplies. Only the first one is refreshed by hid-magicmouse -- it
uses hid_get_battery(), which returns the first battery of the list --
and that refresh goes through the report event path, which parses the
field correctly. Nothing ever reports the second one, so every read of
its capacity takes the query path above. On a USB-C Magic Trackpad over
USB, on an unpatched 7.1.5:
hid-<serial>-battery-144 = 100% (Charging) <- report event path
hid-<serial>-battery-155 = 3% (Discharging) <- query path
Both are the same physical battery. A raw HIDIOCGINPUT of the two
reports at that same moment:
report 0x90 -> [90 03 64]
report 0x9b -> [9b 03 64 64 00 00 10 00 00 00 00 00 00 00]
^flags ^SoC = 0x64 = 100%
The device answers correctly in both cases; only the offset the kernel
reads the capacity from is wrong. 0x03 is the flags byte (present,
charging), reported as "3%".
Bluetooth takes the same query path for its capacity, where the trackpad
reported a bogus near-constant ~4% -- 0b100, the FullyCharged flag --
regardless of the real charge.
Store the battery field's offset within the report at setup time and use
it when querying, so the capacity is read from its real position. The
report event path already parses the field correctly through the HID
core; only the explicit GET_REPORT query was wrong.
Devices whose capacity field is the first field in the report have a
report_offset of 0 and are unaffected (buf[1 + 0] == buf[1]).
Fixes: 581c4484769e ("HID: input: map digitizer battery usage")
Cc: stable@vger.kernel.org
Signed-off-by: Jose Villaseñor Montfort <pepemontfort@gmail.com>
Reviewed-by: Alec Hall <signshop.alec@gmail.com>
Signed-off-by: Jiri Kosina <jkosina@suse.com>
|
|
A 'B' entry's load program selects its interpreter by absolute path,
which open_exec() resolves at exec time in the mount namespace of whoever
runs the binary. The handler names an interpreter but does not get to say
which file that is. Whoever controls the filesystem view of the exec
decides that instead.
Static entries settled this long ago with 'F'. The interpreter is opened
at registration in the registrant's context and every exec runs a clone
of that file. Give a 'B' entry the same, for as many interpreters as it
needs.
An entry registered with 'D' cannot be matched yet, so it still belongs
to whoever is configuring it and can be given interpreters one write at a
time:
echo ':qemu:B::::qemu_user:D' > register
echo '+aarch64 /usr/bin/qemu-aarch64' > qemu
echo '+arm /usr/bin/qemu-arm' > qemu
echo 1 > qemu
Each path is opened by its write, with the credentials the entry file
was opened with, by the same helper that opens an 'F' interpreter. The
load program picks one per exec with bpf_binprm_select_interp() and the
entry hands out a clone of it. Nothing is resolved again, in any
namespace. The path is everything past the first space, so no
interpreter has to fit in a register string. An entry binds at most a
hundred interpreters (BINFMT_MISC_INTERP_MAX). Every binding pins a
struct file that no file descriptor accounts for, so RLIMIT_NOFILE does
not apply and some cap is needed. A hundred is plenty and raising it
later is cheap, lowering it is not.
Selection is by name so the register string and the program need not
agree on an order, and so the handler is not tied to where a distribution
puts its interpreters. A name is a single word of printable ASCII so the
entry file can report 'name path' lines. The interpreter runs under the
path it was registered under.
The entry file reads user memory once. bm_entry_write() copies the write
in and dispatches on the first byte, and parse_command() takes the copied
buffer. The status file has no binding to spell, so it keeps its own
small copy in read_command().
That moves the length cap ahead of the dispatch. A write to an entry file
longer than a binding can be is now refused with -E2BIG, and one from a
bad address reports -EFAULT, where the command parser used to report
-EINVAL for anything past three bytes.
Configurations of one instance are kept apart by the lock removal
already takes. Reading the set out of the entry file takes no lock.
Bindings are rcu-published and the open entry file pins the entry
together with everything it bound, so a reader either sees a whole node
or misses it. The interpreter is opened before the configuration lock
because resolving the path may walk this very filesystem, and only
after the command has been parsed and the name validated from the
copied buffer, so a write that can never bind opens nothing and the
errno reflects the actual failure.
Link: https://patch.msgid.link/20260730-work-binfmt_misc-preopen-v1-7-4a0b0da71f16@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext
Pull sched_ext fixes from Tejun Heo:
- More lifecycle fixes for the new sub-scheduler support: a failed
enable could tear down a never-linked sub-scheduler in a way that
races the root scheduler's disable and leads to a use-after-free,
tasks that were not on the ext class could still get the enable
callback, and a policy-rejection path silently rewrote a running
task's scheduling policy instead of aborting the scheduler.
- Scheduler enable/disable could deadlock with cgroup removal and a
concurrent cgroup weight write through kernfs. Fixed by reordering
lock acquisition.
- Sync wakeups could leave the waker CPU incorrectly marked idle in the
built-in idle-CPU tracking.
- A selftest fix for sleeping tasks whose CPU affinity changes before
wakeup.
* tag 'sched_ext-for-7.2-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext:
selftests/sched_ext: Handle sleeping task affinity changes in numa test
sched_ext: Mark waker CPU busy when selected in WAKE_SYNC case
sched_ext: Don't enable non-ext tasks in the sub-sched task loops
sched_ext: Skip sub-disable teardown for never-linked sub-schedulers
sched_ext: Take cgroup_lock() first in scx_cgroup_lock()
sched_ext: Reject setting disallow from init_task outside the enable path
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
Pull cgroup fixes from Tejun Heo:
- A pressure trigger's poll timer could be re-armed while the last
trigger was being torn down and then fire after the cgroup was freed.
Tie the timer to the cgroup's lifetime and shut it down when the
cgroup is freed.
- Writing to a pressure file forked a worker kthread while holding the
cgroup mutex, creating lock dependencies from the mutex to the whole
fork path. A pressure write racing a sched_ext scheduler enable,
which blocks forks before grabbing the mutex, deadlocked.
Fork the worker with the mutex dropped.
- Documentation fix for io.latency behavior on non-rotational devices.
* tag 'cgroup-for-7.2-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
Docs/admin-guide/cgroup-v2: document io.latency rotational vs non-rotational behavior
sched/psi: Shut down rtpoll_timer in psi_cgroup_free()
sched/psi: Create the psimon kthread outside of cgroup_mutex
|
|
Certain third-party USB game controllers exposing (or spoofing) an Xbox
360-compatible interface (VID:PID 045e:028e) fail to enumerate under Linux.
The device disconnects from the bus without responding to the initial
GET_DESCRIPTOR(CONFIGURATION) request, and the kernel logs 'unable to read
config index 0 descriptor/start: -71'.
The device then falls back to a secondary Android HID mode (with a
different VID:PID), losing XInput functionality including rumble support.
The failure reproduces across multiple machines, host controller types, and
kernel versions including current mainline and LTS. The device enumerates
correctly and remains in XInput mode under Windows. Notably, the device
enumerates correctly in Android mode when the same 9-byte request
is issued for that mode's configuration descriptor, confirming the firmware
bug is specific to the XInput mode.
usbmon traces from Linux and Wireshark/USBPcap traces from Windows are
identical up to the point of failure, with no visible protocol-level
difference explaining the divergence. The root cause was identified when
Michal Pecio discovered via a QEMU bus-level capture that Windows does not
use wLength=9 for the initial config descriptor request; it uses
wLength=255. Alan Stern subsequently confirmed this with a bus
analyzer on a different USB 2.0 device, and Michal verified the behavior
goes back to Windows 95 OSR2.1.
So, add a new quirk flag USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE which causes
usb_get_configuration() to issue a 255 byte sized configuration request
instead of USB_DT_CONFIG_SIZE (9) for the initial
GET_DESCRIPTOR(CONFIGURATION) request, mimicking long-standing Windows
behavior.
This patch intentionally does not add any new VID:PID entries using this
quirk. Some affected Xbox 360-compatible controllers spoof Microsoft's
VID:PID, while genuine Microsoft controllers already enumerate correctly
and do not require this quirk. Other affected clone devices use their own
VID:PID pairs and can be added individually as they are identified.
Suggested-by: Alan Stern <stern@rowland.harvard.edu>
Suggested-by: Michal Pecio <michal.pecio@gmail.com>
Closes: https://lore.kernel.org/linux-usb/CAFgddh+JWdT4LLwMc5qjM8q_pBu-fRo2qADR5ovAKoGHWMQrRw@mail.gmail.com/
Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
Cc: stable <stable@kernel.org>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Nikhil Solanke <nikhilsolanke5@gmail.com>
Link: https://patch.msgid.link/20260728195158.65162-2-nikhilsolanke5@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
|
The rx_trig_bytes sysfs attribute currently relies on 8250-internal
helper functions and assumes a fixed mapping between trigger levels and
FIFO behavior.
Some UARTs provide hardware-specific RX trigger mechanisms that do not
fit this model. Add optional uart_port callbacks for setting and getting
the RX trigger level, and use them when provided, while preserving the
existing 8250 helpers as the default fallback.
Signed-off-by: Crescent Hsieh <crescentcy.hsieh@moxa.com>
Link: https://patch.msgid.link/20260731074820.735619-13-crescentcy.hsieh@moxa.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
|
Some UARTs require driver-specific handling for break signaling, which
cannot be expressed by the generic 8250 break implementation alone.
Add an optional uart_port break_ctl callback and route
serial8250_break_ctl() through it when provided. Rename the existing
8250 implementation to serial8250_do_break_ctl() and export it so
low-level drivers can reuse the default 8250 behavior when appropriate.
Signed-off-by: Crescent Hsieh <crescentcy.hsieh@moxa.com>
Link: https://patch.msgid.link/20260731074820.735619-11-crescentcy.hsieh@moxa.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
|
Add a new 8250 port type for the Moxa MUEx50 UART and describe its basic
FIFO size and trigger characteristics in the 8250 port configuration
table.
The 8250_mxpcie driver sets UPF_FIXED_TYPE and uses PORT_MUEX50 so that
the generic 8250 core applies the correct defaults.
Signed-off-by: Crescent Hsieh <crescentcy.hsieh@moxa.com>
Link: https://patch.msgid.link/20260731074820.735619-3-crescentcy.hsieh@moxa.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
|
Pull 7.2 devel branch for put_device auto-clean fixes.
Signed-off-by: Takashi Iwai <tiwai@suse.de>
|
|
Read chunk position and length validation is currently scattered
across three consumer functions: svc_rdma_read_data_item(),
svc_rdma_read_multiple_chunks(), and svc_rdma_read_call_chunk().
Each independently guards against the same class of unsigned
arithmetic underflow from untrusted wire values. Any new consumer
of the parsed Read chunk list must replicate these checks or risk
re-introducing the defects fixed by earlier patches in this series.
Add pcl_check_read_chunk_positions() to consolidate position and
length validation into a single post-decode pass, called from
svc_rdma_xdr_decode_req() after all three chunk lists have been
parsed and the inline body length is known. The pass verifies
three properties:
- Each Read chunk's inline-body offset (its unreduced-stream
position minus the cumulative length of preceding Read chunks)
falls within the inline body length, or within the Call chunk
length for interleaved reads.
- Adjacent Read chunk positions do not overlap: cumulative read
bytes at each transition do not exceed the next position.
- Each chunk length does not exceed the receive context's page
budget.
Malformed frames are rejected before reaching any consumer. The
existing consumer-side guards remain as defense in depth.
Acked-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260526-rpc-kernel-bugs-v1-6-e251306ccca9@oracle.com
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
|
|
When a parsed chunk list contains a chunk whose ch_segcount is zero,
pcl_for_each_segment computes its inclusive upper bound as
&chunk->ch_segments[ch_segcount - 1]. ch_segcount is u32, so the
subtraction wraps to 0xFFFFFFFF and the bound lands far past the
ch_segments flex array. The loop body then walks unrelated memory at
sizeof(struct svc_rdma_segment) stride until it faults.
A zero-segcount chunk is reachable from the wire:
xdr_check_write_chunk() only rejects segcount values greater than
rc_maxpages, and pcl_alloc_write() links a freshly allocated chunk
onto rc_write_pcl/rc_reply_pcl before its segment-fill loop runs,
so a Write or Reply chunk advertising zero segments leaves
ch_segcount == 0 on the list. When the transport has negotiated
Send-With-Invalidate, svc_rdma_get_inv_rkey() iterates all four
PCLs with pcl_for_each_segment and dereferences segment->rs_handle
on each iteration, turning the underflow into an out-of-bounds read
and a general protection fault.
xdr_check_write_list / xdr_check_reply_chunk
pcl_alloc_write()
chunk = pcl_alloc_chunk(...) /* ch_segcount = 0 */
list_add_tail(&chunk->ch_list, &pcl->cl_chunks)
/* fill loop iterates zero times for wire segcount 0 */
svc_rdma_get_inv_rkey()
pcl_for_each_chunk(rc_write_pcl)
pcl_for_each_segment(segment, chunk)
pos <= &ch_segments[0u - 1u] /* 0xFFFFFFFF */
segment->rs_handle /* OOB read -> GPF */
Fix by switching the macro to a half-open upper bound that uses
ch_segcount directly. For ch_segcount == 0 the loop start equals the
loop end and the body is skipped; for ch_segcount > 0 the iteration
range is unchanged. All six existing call sites in
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c and
net/sunrpc/xprtrdma/svc_rdma_rw.c remain correct under the new bound,
so no caller changes are needed.
Fixes: 78147ca8b4a9 ("svcrdma: Add a "parsed chunk list" data structure")
Cc: stable@vger.kernel.org
Assisted-by: kres (claude-opus-4-7)
Signed-off-by: Chris Mason <clm@meta.com>
Acked-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260526-rpc-kernel-bugs-v1-4-e251306ccca9@oracle.com
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
|
|
Commit 814cba835ef6 ("bpf, x86: Fix trampoline stack size for 128-bit
arguments") changed the x86 trampoline to compute the number of
registers from arg_size for every argument, which removed the last user
of BTF_FMODEL_STRUCT_ARG. No other architecture or verifier code looks
at the flag, so remove the macro and the code in __get_type_fmodel_flags()
which sets it.
Keep BTF_FMODEL_SIGNED_ARG at BIT(1) rather than renumbering it to
BIT(0), so BIT(0) is available for a future flag.
No functional change.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260803052726.2821447-1-yonghong.song@linux.dev
|
|
|
|
uretprobes_srcu currently uses normal SRCU, which issues
two smp_mb() per read lock/unlock pair. This overhead is
paid on every uretprobe hit.
Switch to SRCU-fast-updown, which eliminates the per-reader
memory barriers by moving the ordering cost to the
grace-period side (synchronize_rcu() instead of smp_mb()).
This is acceptable because grace periods (uprobe
unregistration) are infrequent compared to reader-side
uretprobe hits.
The updown flavor is required because the SRCU read lock is
taken in prepare_uretprobe() when a return instance is
created and is held until that return instance is finalized.
The traced thread returns to user space in between, so the
lock is inherently released in a different context from
where it was acquired: on the normal return path via
uprobe_handle_trampoline() -> hprobe_finalize(), or from
ri_timer() (expiry) or dup_utask() (fork) via
hprobe_expire(). srcu_down_read_fast() / srcu_up_read_fast()
are designed for this acquire-here / release-elsewhere
pattern and, unlike the same-context srcu_read_lock_fast()
variant, do not carry the lockdep read-side tracking that
would warn on it.
The short, same-context SRCU sections in ri_timer() and
dup_utask() (which guard the uprobe against reuse across the
hprobe_expire() cmpxchg) instead use guard(srcu_fast_updown)
for proper lockdep coverage.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://patch.msgid.link/20260706172744.3920417-3-puranjay@kernel.org
|
|
Add a guard(srcu_fast_updown) definition for scoped
SRCU-fast-updown read-side critical sections, following the
existing pattern of guard(srcu) and guard(srcu_fast).
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Link: https://patch.msgid.link/20260706172744.3920417-2-puranjay@kernel.org
|
|
Give bpf handlers the per-exec equivalent of the static 'L' flag. A
load program that sets BPF_BINPRM_LOADER has its selected interpreter
substituted for the binary's PT_INTERP instead of run with the binary
as payload. The binary otherwise executes as a fully native exec.
A single handler can now grade its dispatch per binary: native-arch ELF
with PT_INTERP gets loader substitution for full native identity.
Anything else, such as foreign arch, static, non-ELF can use transparent
or classic dispatch. The load program can read the binary's ELF header
from bprm->buf to make that call.
Link: https://patch.msgid.link/20260721-work-bpf-binfmt_misc-ptinterp-v2-19-e57866e4ae0f@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
