diff options
| author | Linus Torvalds <torvalds@linux-foundation.org> | 2026-08-21 08:41:00 -0700 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2026-08-21 08:41:00 -0700 |
| commit | 4e69c1856bfd9ffb7e9d335a25842fa211628929 (patch) | |
| tree | bc47d84baf013634463f55ba26a554cbfcb7c908 /rust/kernel | |
| parent | 818bebeb63dd6bf5f4e07e145f6cdbace520a34c (diff) | |
| parent | c44e278ce02efd0c4be79a8eda1ea6885c1ce5ec (diff) | |
Merge tag 'drm-next-2026-08-20' of https://gitlab.freedesktop.org/drm/kernel
Pull drm updates from Dave Airlie:
"Highlights:
- dmemcg eviction support is good for low VRAM things like Steam
Machine
- AMD adds gfx6-8 modifier support for older GPUs that enables a
bunch of wayland stuff
- i915/xe has some new hw support but also a lot of display
refactoring
Everything:
perf:
- export perf_allow_ APIs for xe
udmabuf:
- remove default size limit of 64MB
rust:
- i/o rework (signed tag from driver-core tree)
- add registration guard and registration data
- fix unbounded lifetimes in ioctl handler args
- fix a drm_dev_register race
- gem_shmem: add DmaResvGuard helper
- gpuvm: require send/sync for driver data
- implement send/sync for GpuVaAlloc and GpuVmBo
- add SmContext lifetime
- rename dma_handle to dma_address
- change pci_sriov_get_totalvfs return to unsigned int
core:
- create drm_of_get_panel_orientation
- send per-connector hotplug events
- add thunderbolt UBHR tunneling support
connector:
- add color format property
dmem:
- introduce a peak file
- accept one region per limit
- add dmemcg support for eviction
gpusvm:
- reorg code to give drivers more flexibility
atomic:
- add create_state callback and helper
- add documentation on atomic commit lifetime
buddy:
- add per-order free
- add used block scoreboard
- fix UAF
- test buffer clearance on resume
- add phys_addr->block helper
gem:
- drop DRIVER_GEM_GPUVA flag
ttm:
- be more aggressive allocating below protection limit
sched:
- add test suite for concurrent job submissions
hdmi:
- hook the color format property in helpers
mipi-dsi:
- add MIPI_DSI_MODE_DSC_ALL_SLICES_IN_PKT
bridge:
- add atomic create callbacks
- drop atomic reset
- display-connector: don't autoenable HPD IRQ
- trigger initial HPD for DP
- ti-sn65dsi83: remove NO_HFP and NO_HBP mode flags
- analogix_dp: switch to DP link training helpers
dp:
- add support for DSC max delta BPP
edid:
- parse panel type from DisplayID 2.x Display Parameters
sysfb:
- improve panel, stride, framebuffer size validation
panel:
- implement ref counting for struct drm_panel
- himax-hx83121a: add backlight regulator support
- novatek-nt36672a: Inline panel init sequences
- visionox-vtdr6130: enable DSC
- novatek-nt37801: Use mipi_dsi_*_multi() functions
- samsung-s6d16d0: Fix prepare error handling
- support Novatek NT36536 plus DT bindings
- sofef00: fix backlight updates
- osd101t2587: use mipi_dsi_*_multi interface
- panel-edp: adjust timing for AUO displays
- panel-lvds: support Opto Logic SCX1001511GGC49
- panel-simple: support Kyocera tcg070wvlq
- panel-edp: quirks
- AUO B116XAT04.3, CMN N116BCP-EA2, CSW MNB601LS1-8
- BOE NV116WH2-M30, BOE NT116WHM-N21, BOE NV116FH1-M31
- BOE NV116FH1-M30, NV140FHM-N5B, TM156VDXP25
- BOE NE160QDM-NY1, MB116AS01
- new:
- Samsung ATNA40HQ08-0, Anbernic TD4310
- Chipone ICNA35XX, Ilitek ILI9488
- Ilitek ILI7807S, Renesas R63419
- MNE001BS6-2, MNF601BS4-1, Sharp LQ120P1JX51
virtio:
- add support for save/restore virtio_gpu_objects
- abort vq wait on device removal
amdgpu:
- add color format DRM property
- initial compute pipe reset support
- add GFX 6-8 modifier support
- initial DCN 6.0.0 support
- dmemcg eviction support
- improved boundary checking for bios parsing
- RAS updates and rework
- VCN secure submission fixes
- 8K panel fix
- Display KUNIT tests
- parse panel type from DisplayID
- Align IP discovery to pci device lifetime
- SOC15 register macro cleanups
- UVD memory placement fixes
- GFX9 mode2 reset fixes
- drop unnecessary BUG/BUG_ON
- GFX8 soft reset rework
- enable soft reset on GFX8
- PSP/SMU 15.0.9 update
- VI ASPM fix
- userq fixes
- amdgpu_vm_get_task_info_pasid lifetime fix
- DC CACP support
- change system_unbound_wq with system_dfl_wq
- Loosen VFCT bios parsing to deal with pci=realloc
- SI/SMU7 AC/DC switch fix
- VM fence handling fix
- GEM close optimisation
- Apple Studio Display fixes
- DC FRL fixes
amdkfd:
- initial compute pipe reset support
- allow applications to opt out of sigbus on fatal errors
- improve CRIU boundary checks
- MQD handling rework
- move TBA/TMA from system to device memory
- avoid topology-lock in kfd_mmap
- SVM eviction fixes
radeon:
- fix unset CONFIG_ACPI build
i915:
- Novalake (NVL display version 35) timing generator enabling
- NVL DC3CO enabling
- enable UBHR link rates on thunderbolt tunnels
- Reduce Xe3+ PM demand peak bandwidth
- enable pipe DMC error interrupts for display 30+
- add kunit tests for DP link config selection
- refactor and document DP link recovery
- i915/xe driver display probe/remove/suspend/resume/shutdown cleanup
and unification
- i915/xe display runtime PM unified
- Break i915 and xe panic dependency on struct intel_framebuffer
- Streamline Pre/Post-CSC LUT loops
- drop TGL DC3DO support
- CDCLK santization
- fix HDMI scrambling enable
- fix phys bo pread/pwrite with offset
- add missing nospec on parallel submit slot
- fix some NULL derefs
xe:
- drop force_execlist module param
- gate observation streams with perf_allow_cpu
- skip FORCE_WC and vm_bound check for external dma-bufs
- dmemcg eviction support
- remove unused NVL-S GuC
- TLB invalidation improvements
- NVL-S updated PCI-IDs and w/a
- madvise: optimise invalidation path
- fix infinite gt-reset loop in timeout recovery
- update TTM device benefical_order
- wait on external BO kernel fences in exec ioctl
- add/use more KLV helpers
- sriov: disable display in admin only PF mode
- add RAS GPU health indicator
- optimise TTM populate for DONTNEED BO
- drop force_probe for NVL-s
- add debugfs for pcode info
amdxdna:
- disable device buffer export
nova:
- build nova-core/nova-drm from drivers/gpu
- export nova-core rust symbols (workaround)
- GSP boot process consolidation
- Boot GSP with vGPU enabled
- TLV firmware image format support
- Hopper/Blackwell fixes and cleanups
- I/O projection adoption
tyr:
- firmware loading and MCU boot
- add generic slot manager + MMU
- GPU VM support ARM64 LPAE page tables
- add kernel buffer object for internal allocations
- add parser for Mali CSF
- add MCU booting
nouveau:
- race fixes
- check instmem iomapping at first use
- add dmemcg support
- expose NVDEC channels
- add scanline position/head state support for GSP
qxl:
- convert simple encoder to regular
ethosu:
- add perf counter support
etnaviv:
- force flush on power register ops
msm:
- support DSC configuration with slice_per_pkt > 1
mxsfb:
- fix disable sequence
panthor:
- support sparse mappings
rockchip:
- switch away from simple helpers
- support YUV background color
- fix layer config timeout
- add edp support for rk3576
- add batch command submission function
rocket:
- error handling and NULL ptr deref fixes
sun4i:
- switch away from simple helpers
imagination:
- mark BXM-4-64 MC1 as support
host1x:
- support tegra264
tegra:
- add DSI for tegra 20/30
v3d:
- reduce PM runtime autosuspend delay
- scheduler fixes and refactoring
- deprecate v3d 3.3 and 4.1
- validate CPU job query boundaries
hibmc:
- improve plane format handling
- switch to gem shmem
mediatek:
- cec: correct compat for mt7623-8167?
exynos:
- remove simple dependency
- add error handling to encoder paths
- take i2c adapter module reference"
* tag 'drm-next-2026-08-20' of https://gitlab.freedesktop.org/drm/kernel: (2074 commits)
drm/xe/mcr: Take vcs1/vecs1 into account for first media slice
drm/xe: Fix a bug in pc_adjust_freq_bounds()
drm/xe: Fix xe_device_probe() failure
drm/xe/drm_ras: Move has_drm_ras check to drm_ras layer
drm/xe/ras: Fix boot-time ras error processing
drm/amd/display: make DC_RUN_WITH_PREEMPTION_ENABLED misuse a build error
drm/amd/pm: silence uninitialized variable warnings
drm/amdgpu: skip BOs being torn down during GTT recovery
drm/amdgpu: Reject UVD message with invalid number of h265 refs
drm/amdgpu: keep PRT mappings off the vm_bo state lists
drm/amdgpu: fix nbif 6.3.1 l1 low power not functional
drm/amd/display: fix BT.2020 YCbCr output CSC matrices for DCE
drm/amd/display: fix BT.2020 YCbCr limited output CSC matrix
drm/amdgpu: Implement insert_end for VCE 3
drm/amdgpu: Fix UVD min buffer sizes
drm/amdgpu: Fix UVD decode image min size calculation
drm/amdgpu: Fix UVD dpb min size calculation for H264
drm/amdgpu: Reject UVD message with dimensions above 4096
drm/amdgpu: check ASPM on the dGPU host link
drm/radeon: fix autosuspend cleanup during teardown
...
Diffstat (limited to 'rust/kernel')
| -rw-r--r-- | rust/kernel/alloc/kvec.rs | 27 | ||||
| -rw-r--r-- | rust/kernel/dma.rs | 70 | ||||
| -rw-r--r-- | rust/kernel/drm/device.rs | 291 | ||||
| -rw-r--r-- | rust/kernel/drm/driver.rs | 113 | ||||
| -rw-r--r-- | rust/kernel/drm/gem/mod.rs | 103 | ||||
| -rw-r--r-- | rust/kernel/drm/gem/shmem.rs | 594 | ||||
| -rw-r--r-- | rust/kernel/drm/gpuvm/mod.rs | 36 | ||||
| -rw-r--r-- | rust/kernel/drm/gpuvm/sm_ops.rs | 4 | ||||
| -rw-r--r-- | rust/kernel/drm/gpuvm/va.rs | 8 | ||||
| -rw-r--r-- | rust/kernel/drm/gpuvm/vm_bo.rs | 9 | ||||
| -rw-r--r-- | rust/kernel/drm/ioctl.rs | 56 | ||||
| -rw-r--r-- | rust/kernel/drm/mod.rs | 4 | ||||
| -rw-r--r-- | rust/kernel/faux.rs | 81 | ||||
| -rw-r--r-- | rust/kernel/firmware.rs | 46 | ||||
| -rw-r--r-- | rust/kernel/iommu/pgtable.rs | 32 | ||||
| -rw-r--r-- | rust/kernel/pci.rs | 13 |
16 files changed, 1162 insertions, 325 deletions
diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs index f7af62835aa8..c7546b9da4fa 100644 --- a/rust/kernel/alloc/kvec.rs +++ b/rust/kernel/alloc/kvec.rs @@ -9,6 +9,7 @@ use super::{ Vmalloc, VmallocPageIter, // }, + flags::__GFP_ZERO, layout::ArrayLayout, AllocError, Allocator, @@ -51,6 +52,8 @@ use core::{ }, // }; +use pin_init::Zeroable; + mod errors; pub use self::errors::{InsertError, PushError, RemoveError}; @@ -532,6 +535,30 @@ where Ok(v) } + /// Creates a new [`Vec`] with `n` zero-initialized elements. + /// + /// # Examples + /// + /// ``` + /// let v = KVec::<u32>::zeroed(20, GFP_KERNEL)?; + /// + /// assert!(v.iter().all(|&x| x == 0)); + /// # Ok::<(), Error>(()) + /// ``` + pub fn zeroed(n: usize, flags: Flags) -> Result<Self, AllocError> + where + T: Zeroable, + { + let mut v = Self::with_capacity(n, flags | __GFP_ZERO)?; + + // SAFETY: + // - `n <= capacity - len`: `with_capacity(n)` guarantees capacity >= n, len is 0. + // - All elements in `[0, n)` are initialized: `__GFP_ZERO` zeroes the allocation, + // and `T: Zeroable` guarantees all-zeroes is a valid bit pattern. + unsafe { v.inc_len(n) }; + Ok(v) + } + /// Creates a `Vec<T, A>` from a pointer, a length and a capacity using the allocator `A`. /// /// # Examples diff --git a/rust/kernel/dma.rs b/rust/kernel/dma.rs index 9a0bf745614c..2ce09f8e90c6 100644 --- a/rust/kernel/dma.rs +++ b/rust/kernel/dma.rs @@ -585,7 +585,7 @@ impl<T: AsBytes + FromBytes + KnownSize + ?Sized> From<CoherentBox<T>> for Coher /// # Invariants /// /// - For the lifetime of an instance of [`Coherent`], the `cpu_addr` is a valid pointer -/// to an allocated region of coherent memory and `dma_handle` is the DMA address base of the +/// to an allocated region of coherent memory and `dma_addr` is the DMA address base of the /// region. /// - The size in bytes of the allocation is equal to size information via pointer. // TODO @@ -602,7 +602,7 @@ impl<T: AsBytes + FromBytes + KnownSize + ?Sized> From<CoherentBox<T>> for Coher // entire `Coherent` including the allocated memory itself. pub struct Coherent<T: KnownSize + ?Sized> { dev: ARef<device::Device>, - dma_handle: DmaAddress, + dma_addr: DmaAddress, cpu_addr: NonNull<T>, dma_attrs: Attrs, } @@ -627,11 +627,10 @@ impl<T: KnownSize + ?Sized> Coherent<T> { self.cpu_addr.as_ptr() } - /// Returns a DMA handle which may be given to the device as the DMA address base of - /// the region. + /// Returns a DMA address which may be given to the device as the base of the region. #[inline] - pub fn dma_handle(&self) -> DmaAddress { - self.dma_handle + pub fn dma_address(&self) -> DmaAddress { + self.dma_addr } /// Returns a reference to the data in the region. @@ -678,13 +677,13 @@ impl<T: AsBytes + FromBytes> Coherent<T> { ); } - let mut dma_handle = 0; + let mut dma_addr = 0; // SAFETY: Device pointer is guaranteed as valid by the type invariant on `Device`. let addr = unsafe { bindings::dma_alloc_attrs( dev.as_raw(), core::mem::size_of::<T>(), - &mut dma_handle, + &mut dma_addr, gfp_flags.as_raw(), dma_attrs.as_raw(), ) @@ -696,7 +695,7 @@ impl<T: AsBytes + FromBytes> Coherent<T> { // - We also hold a refcounted reference to the device. Ok(Self { dev: dev.into(), - dma_handle, + dma_addr, cpu_addr, dma_attrs, }) @@ -795,13 +794,13 @@ impl<T: AsBytes + FromBytes> Coherent<T> { } let size = core::mem::size_of::<T>().checked_mul(len).ok_or(ENOMEM)?; - let mut dma_handle = 0; + let mut dma_addr = 0; // SAFETY: Device pointer is guaranteed as valid by the type invariant on `Device`. let addr = unsafe { bindings::dma_alloc_attrs( dev.as_raw(), size, - &mut dma_handle, + &mut dma_addr, gfp_flags.as_raw(), dma_attrs.as_raw(), ) @@ -813,7 +812,7 @@ impl<T: AsBytes + FromBytes> Coherent<T> { // - We also hold a refcounted reference to the device. Ok(Coherent { dev: dev.into(), - dma_handle, + dma_addr, cpu_addr, dma_attrs, }) @@ -927,14 +926,14 @@ impl<T: KnownSize + ?Sized> Drop for Coherent<T> { fn drop(&mut self) { let size = T::size(self.cpu_addr.as_ptr()); // SAFETY: Device pointer is guaranteed as valid by the type invariant on `Device`. - // The cpu address, and the dma handle are valid due to the type invariants on + // The cpu address, and the dma address are valid due to the type invariants on // `Coherent`. unsafe { bindings::dma_free_attrs( self.dev.as_raw(), size, self.cpu_addr.as_ptr().cast(), - self.dma_handle, + self.dma_addr, self.dma_attrs.as_raw(), ) } @@ -993,13 +992,13 @@ impl<T: KnownSize + AsBytes + ?Sized> debugfs::BinaryWriter for Coherent<T> { /// /// - `cpu_handle` holds the opaque handle returned by `dma_alloc_attrs` with /// `DMA_ATTR_NO_KERNEL_MAPPING` set, and is only valid for passing back to `dma_free_attrs`. -/// - `dma_handle` is the corresponding bus address for device DMA. +/// - `dma_addr` is the corresponding bus address for device DMA. /// - `size` is the allocation size in bytes as passed to `dma_alloc_attrs`. /// - `dma_attrs` contains the attributes used for the allocation, always including /// `DMA_ATTR_NO_KERNEL_MAPPING`. pub struct CoherentHandle { dev: ARef<device::Device>, - dma_handle: DmaAddress, + dma_addr: DmaAddress, cpu_handle: NonNull<c_void>, size: usize, dma_attrs: Attrs, @@ -1023,13 +1022,13 @@ impl CoherentHandle { } let dma_attrs = dma_attrs | Attrs(bindings::DMA_ATTR_NO_KERNEL_MAPPING); - let mut dma_handle = 0; + let mut dma_addr = 0; // SAFETY: `dev.as_raw()` is valid by the type invariant on `device::Device`. let cpu_handle = unsafe { bindings::dma_alloc_attrs( dev.as_raw(), size, - &mut dma_handle, + &mut dma_addr, gfp_flags.as_raw(), dma_attrs.as_raw(), ) @@ -1038,11 +1037,11 @@ impl CoherentHandle { let cpu_handle = NonNull::new(cpu_handle).ok_or(ENOMEM)?; // INVARIANT: `cpu_handle` is the opaque handle from a successful `dma_alloc_attrs` call - // with `DMA_ATTR_NO_KERNEL_MAPPING`, `dma_handle` is the corresponding DMA address, + // with `DMA_ATTR_NO_KERNEL_MAPPING`, `dma_addr` is the corresponding DMA address, // and we hold a refcounted reference to the device. Ok(Self { dev: dev.into(), - dma_handle, + dma_addr, cpu_handle, size, dma_attrs, @@ -1059,12 +1058,12 @@ impl CoherentHandle { Self::alloc_with_attrs(dev, size, gfp_flags, Attrs(0)) } - /// Returns the DMA handle for this allocation. + /// Returns the DMA address for this allocation. /// /// This address can be programmed into device hardware for DMA access. #[inline] - pub fn dma_handle(&self) -> DmaAddress { - self.dma_handle + pub fn dma_address(&self) -> DmaAddress { + self.dma_addr } /// Returns the size in bytes of this allocation. @@ -1083,28 +1082,29 @@ impl Drop for CoherentHandle { self.dev.as_raw(), self.size, self.cpu_handle.as_ptr(), - self.dma_handle, + self.dma_addr, self.dma_attrs.as_raw(), ) } } } -// SAFETY: `CoherentHandle` only holds a device reference, a DMA handle, an opaque CPU handle, +// SAFETY: `CoherentHandle` only holds a device reference, a DMA address, an opaque CPU handle, // and a size. None of these are tied to a specific thread. unsafe impl Send for CoherentHandle {} // SAFETY: `CoherentHandle` provides no CPU access to the underlying allocation. The only -// operations on `&CoherentHandle` are reading the DMA handle and size, both of which are +// operations on `&CoherentHandle` are reading the DMA address and size, both of which are // plain `Copy` values. unsafe impl Sync for CoherentHandle {} /// View type for `Coherent`. /// -/// This is same as [`SysMem`] but with additional information that allows handing out a DMA handle. +/// This is same as [`SysMem`] but with additional information that allows handing out a DMA +/// address. pub struct CoherentView<'a, T: ?Sized> { cpu_addr: SysMem<'a, T>, - dma_handle: DmaAddress, + dma_addr: DmaAddress, } impl<T: ?Sized> Copy for CoherentView<'_, T> {} @@ -1116,16 +1116,16 @@ impl<T: ?Sized> Clone for CoherentView<'_, T> { } impl<'a, T: ?Sized> CoherentView<'a, T> { - /// Erase the DMA handle information and obtain a [`SysMem`] view of the same memory region. + /// Erase the DMA address information and obtain a [`SysMem`] view of the same memory region. #[inline] pub fn as_sys_mem(self) -> SysMem<'a, T> { self.cpu_addr } - /// Returns a DMA handle which may be given to the device as the DMA address base of the region. + /// Returns the DMA address which may be given to the device as base of the region. #[inline] - pub fn dma_handle(self) -> DmaAddress { - self.dma_handle + pub fn dma_address(self) -> DmaAddress { + self.dma_addr } /// Returns a reference to the data in the region. @@ -1178,9 +1178,9 @@ impl IoBackend for CoherentIoBackend { ) -> Self::View<'a, U> { let offset = ptr.addr() - view.cpu_addr.as_ptr().addr(); // CAST: The offset DMA address can never overflow. - let dma_handle = view.dma_handle + offset as DmaAddress; + let dma_addr = view.dma_addr + offset as DmaAddress; CoherentView { - dma_handle, + dma_addr, // SAFETY: Per safety requirement. cpu_addr: unsafe { SysMemBackend::project_view(view.cpu_addr, ptr) }, } @@ -1245,7 +1245,7 @@ impl<'a, T: ?Sized + KnownSize> IoBase<'a> for &'a Coherent<T> { CoherentView { // SAFETY: `cpu_addr` is valid and aligned kernel accessible memory. cpu_addr: unsafe { SysMem::new(self.cpu_addr.as_ptr()) }, - dma_handle: self.dma_handle, + dma_addr: self.dma_addr, } } } diff --git a/rust/kernel/drm/device.rs b/rust/kernel/drm/device.rs index 81f9f7e59817..6b88ade28e24 100644 --- a/rust/kernel/drm/device.rs +++ b/rust/kernel/drm/device.rs @@ -32,6 +32,7 @@ use crate::{ }; use core::{ alloc::Layout, + cell::UnsafeCell, marker::PhantomData, mem, ops::Deref, @@ -74,66 +75,59 @@ macro_rules! drm_legacy_fields { /// A trait implemented by all possible contexts a [`Device`] can be used in. /// -/// Setting up a new [`Device`] is a multi-stage process. Each step of the process that a user -/// interacts with in Rust has a respective [`DeviceContext`] typestate. For example, -/// `Device<T, Registered>` would be a [`Device`] that reached the [`Registered`] [`DeviceContext`]. +/// A [`Device`] can be in one of the following contexts: /// -/// Each stage of this process is described below: +/// - [`Normal`]: The general-purpose, reference-counted context. A [`Device`] in this context may +/// or may not be registered with userspace. +/// - [`Ioctl`]: The device has been registered with userspace at some point; used in ioctl +/// dispatch context. +/// - [`Registered`]: The device is currently registered with userspace and the parent bus device +/// is bound. /// -/// ```text -/// 1 2 3 -/// +--------------+ +------------------+ +-----------------------+ -/// |Device created| → |Device initialized| → |Registered w/ userspace| -/// +--------------+ +------------------+ +-----------------------+ -/// (Uninit) (Registered) -/// ``` -/// -/// 1. The [`Device`] is in the [`Uninit`] context and is not guaranteed to be initialized or -/// registered with userspace. Only a limited subset of DRM core functionality is available. -/// 2. The [`Device`] is guaranteed to be fully initialized, but is not guaranteed to be registered -/// with userspace. All DRM core functionality which doesn't interact with userspace is -/// available. We currently don't have a context for representing this. -/// 3. The [`Device`] is guaranteed to be fully initialized, and is guaranteed to have been -/// registered with userspace at some point - thus putting it in the [`Registered`] context. +/// Both `Device<T, Ioctl>` and `Device<T, Registered>` dereference to `Device<T>` ([`Normal`]), +/// so any method available on a [`Normal`] device is also available in the other contexts. +pub trait DeviceContext: Sealed + Send + Sync + 'static {} + +/// The general-purpose, reference-counted [`DeviceContext`]. /// -/// An important caveat of [`DeviceContext`] which must be kept in mind: when used as a typestate -/// for a reference type, it can only guarantee that a [`Device`] reached a particular stage in the -/// initialization process _at the time the reference was taken_. No guarantee is made in regards to -/// what stage of the process the [`Device`] is currently in. This means for instance that a -/// `&Device<T, Uninit>` may actually be registered with userspace, it just wasn't known to be -/// registered at the time the reference was taken. -pub trait DeviceContext: Sealed + Send + Sync {} - -/// The [`DeviceContext`] of a [`Device`] that was registered with userspace at some point. +/// A [`Device`] in this context may or may not be registered with userspace. This context is used +/// for reference-counted device handles and during device setup via [`UnregisteredDevice`]. /// -/// This represents a [`Device`] which is guaranteed to have been registered with userspace at -/// some point in time. Such a DRM device is guaranteed to have been fully-initialized. +/// [`AlwaysRefCounted`] is only implemented for `Device<T, Normal>`, making this the required +/// context for [`ARef`]-based device handles. +pub struct Normal; + +impl Sealed for Normal {} +impl DeviceContext for Normal {} + +/// The [`DeviceContext`] of a [`Device`] that is currently registered with userspace. /// -/// Note: A device in this context is not guaranteed to remain registered with userspace for its -/// entire lifetime, as this is impossible to guarantee at compile-time. +/// A [`Device`] in this context is guaranteed to be registered and its parent bus device is +/// guaranteed to be bound. This is enforced at runtime by [`RegistrationGuard`], which holds a +/// `drm_dev_enter()` / `drm_dev_exit()` SRCU critical section. /// /// # Invariants /// -/// A [`Device`] in this [`DeviceContext`] is guaranteed to have been registered with userspace -/// at some point in time. +/// The parent bus device is bound for the duration of any reference to a `Device<T, Registered>`. pub struct Registered; impl Sealed for Registered {} impl DeviceContext for Registered {} -/// The [`DeviceContext`] of a [`Device`] that may be unregistered and partly uninitialized. +/// The [`DeviceContext`] of a [`Device`] that has been registered with userspace previously. /// -/// A [`Device`] in this context is only guaranteed to be partly initialized, and may or may not -/// be registered with userspace. Thus operations which depend on the [`Device`] being fully -/// initialized, or which depend on the [`Device`] being registered with userspace are not -/// available through this [`DeviceContext`]. +/// A [`Device`] in this context has been registered at some point, but may be concurrently +/// unregistering or already unregistered. `drm_dev_enter()` can guard against this, ensuring the +/// device remains registered for the duration of the critical section. /// -/// A [`Device`] in this context can be used to create a -/// [`Registration`](drm::driver::Registration). -pub struct Uninit; +/// # Invariants +/// +/// A [`Device`] in this context has been registered with userspace via `drm_dev_register()` at +/// some point. +pub struct Ioctl; -impl Sealed for Uninit {} -impl DeviceContext for Uninit {} +impl Sealed for Ioctl {} +impl DeviceContext for Ioctl {} /// A [`Device`] which is known at compile-time to be unregistered with userspace. /// @@ -147,10 +141,10 @@ impl DeviceContext for Uninit {} /// /// The device in `self.0` is guaranteed to be a newly created [`Device`] that has not yet been /// registered with userspace until this type is dropped. -pub struct UnregisteredDevice<T: drm::Driver>(ARef<Device<T, Uninit>>, NotThreadSafe); +pub struct UnregisteredDevice<T: drm::Driver>(ARef<Device<T, Normal>>, NotThreadSafe); impl<T: drm::Driver> Deref for UnregisteredDevice<T> { - type Target = Device<T, Uninit>; + type Target = Device<T, Normal>; fn deref(&self) -> &Self::Target { &self.0 @@ -178,15 +172,13 @@ impl<T: drm::Driver> UnregisteredDevice<T> { master_drop: None, debugfs_init: None, - // Ignore the Uninit DeviceContext below. It is only provided because it is required by the - // compiler, and it is not actually used by these functions. - gem_create_object: T::Object::<Uninit>::ALLOC_OPS.gem_create_object, - prime_handle_to_fd: T::Object::<Uninit>::ALLOC_OPS.prime_handle_to_fd, - prime_fd_to_handle: T::Object::<Uninit>::ALLOC_OPS.prime_fd_to_handle, - gem_prime_import: T::Object::<Uninit>::ALLOC_OPS.gem_prime_import, - gem_prime_import_sg_table: T::Object::<Uninit>::ALLOC_OPS.gem_prime_import_sg_table, - dumb_create: T::Object::<Uninit>::ALLOC_OPS.dumb_create, - dumb_map_offset: T::Object::<Uninit>::ALLOC_OPS.dumb_map_offset, + gem_create_object: T::Object::ALLOC_OPS.gem_create_object, + prime_handle_to_fd: T::Object::ALLOC_OPS.prime_handle_to_fd, + prime_fd_to_handle: T::Object::ALLOC_OPS.prime_fd_to_handle, + gem_prime_import: T::Object::ALLOC_OPS.gem_prime_import, + gem_prime_import_sg_table: T::Object::ALLOC_OPS.gem_prime_import_sg_table, + dumb_create: T::Object::ALLOC_OPS.dumb_create, + dumb_map_offset: T::Object::ALLOC_OPS.dumb_map_offset, show_fdinfo: None, fbdev_probe: None, @@ -209,10 +201,13 @@ impl<T: drm::Driver> UnregisteredDevice<T> { /// Create a new `UnregisteredDevice` for a `drm::Driver`. /// /// This can be used to create a [`Registration`](kernel::drm::Registration). - pub fn new(dev: &device::Device, data: impl PinInit<T::Data, Error>) -> Result<Self> { + pub fn new( + dev: &T::ParentDevice<device::Bound>, + data: impl PinInit<T::Data, Error>, + ) -> Result<Self> { // `__drm_dev_alloc` uses `kmalloc()` to allocate memory, hence ensure a `kmalloc()` // compatible `Layout`. - let layout = Kmalloc::aligned_layout(Layout::new::<Device<T, Uninit>>()); + let layout = Kmalloc::aligned_layout(Layout::new::<Device<T, Normal>>()); // Use a temporary vtable without a `release` callback until `data` is initialized, so // init failure can release the DRM device without dropping uninitialized fields. @@ -224,12 +219,12 @@ impl<T: drm::Driver> UnregisteredDevice<T> { // SAFETY: // - `alloc_vtable` reference remains valid until no longer used, // - `dev` is valid by its type invarants, - let raw_drm: *mut Device<T, Uninit> = unsafe { + let raw_drm: *mut Device<T, Normal> = unsafe { bindings::__drm_dev_alloc( - dev.as_raw(), + dev.as_ref().as_raw(), &alloc_vtable, layout.size(), - mem::offset_of!(Device<T, Uninit>, dev), + mem::offset_of!(Device<T, Normal>, dev), ) } .cast(); @@ -254,6 +249,9 @@ impl<T: drm::Driver> UnregisteredDevice<T> { // SAFETY: `drm_dev` is still private to this function. unsafe { (*drm_dev).driver = const { &Self::VTABLE } }; + // SAFETY: `raw_drm` is valid; no concurrent access before registration. + unsafe { (*raw_drm.as_ptr()).registration_data = UnsafeCell::new(NonNull::dangling()) }; + // SAFETY: The reference count is one, and now we take ownership of that reference as a // `drm::Device`. // INVARIANT: We just created the device above, but have yet to call `drm_dev_register`. @@ -265,16 +263,8 @@ impl<T: drm::Driver> UnregisteredDevice<T> { /// A typed DRM device with a specific [`drm::Driver`] implementation and [`DeviceContext`]. /// -/// Since DRM devices can be used before being fully initialized and registered with userspace, `C` -/// represents the furthest [`DeviceContext`] we can guarantee that this [`Device`] has reached. -/// -/// Keep in mind: this means that an unregistered device can still have the registration state -/// [`Registered`] as long as it was registered with userspace once in the past, and that the -/// behavior of such a device is still well-defined. Additionally, a device with the registration -/// state [`Uninit`] simply does not have a guaranteed registration state at compile time, and could -/// be either registered or unregistered. Since there is no way to guarantee a long-lived reference -/// to an unregistered device would remain unregistered, we do not provide a [`DeviceContext`] for -/// this. +/// A device in the [`Registered`] context is currently registered with userspace and its parent +/// bus device is bound. The [`Normal`] context is the general-purpose, reference-counted context. /// /// # Invariants /// @@ -282,9 +272,10 @@ impl<T: drm::Driver> UnregisteredDevice<T> { /// * The data layout of `Self` remains the same across all implementations of `C`. /// * Any invariants for `C` also apply. #[repr(C)] -pub struct Device<T: drm::Driver, C: DeviceContext = Registered> { +pub struct Device<T: drm::Driver, C: DeviceContext = Normal> { dev: Opaque<bindings::drm_device>, data: T::Data, + pub(super) registration_data: UnsafeCell<NonNull<T::RegistrationData<'static>>>, _ctx: PhantomData<C>, } @@ -353,7 +344,111 @@ impl<T: drm::Driver, C: DeviceContext> Device<T, C> { } } -impl<T: drm::Driver, C: DeviceContext> Deref for Device<T, C> { +impl<T: drm::Driver> Device<T, Ioctl> { + /// Guard against the parent bus device being unbound. + /// + /// Returns a [`RegistrationGuard`] if the device has not been unplugged, [`None`] otherwise. + /// + /// While [`RegistrationGuard`] is held the parent device is guaranteed to be bound. + #[must_use] + pub fn registration_guard(&self) -> Option<RegistrationGuard<'_, T>> { + let mut idx: i32 = 0; + // SAFETY: `self.as_raw()` is a valid pointer to a `struct drm_device`. + if unsafe { bindings::drm_dev_enter(self.as_raw(), &mut idx) } { + // INVARIANT: + // - `idx` is the SRCU index from the successful `drm_dev_enter()` above. + // - The parent bus device is bound: `drm_dev_enter()` succeeded, meaning + // `drm_dev_unplug()` has not completed; since it is only called from + // `Registration::drop()` during parent unbind, the parent is still bound. + Some(RegistrationGuard { + // SAFETY: See INVARIANT above; the `Registered` context invariant holds. + dev: unsafe { self.assume_ctx() }, + idx, + _not_send: NotThreadSafe, + }) + } else { + None + } + } +} + +/// A guard proving the DRM device is registered and the parent bus device is bound. +/// +/// The guard dereferences to [`Device<T, Registered>`], providing access to the DRM device with +/// the guarantee that the parent bus device is bound for the entire duration of the critical +/// section. +/// +/// Internally this is backed by a `drm_dev_enter()` / `drm_dev_exit()` SRCU critical section. +/// +/// # Invariants +/// +/// - `idx` is the SRCU read lock index returned by a successful `drm_dev_enter()` call. +/// - The parent bus device of `dev` is bound for the lifetime of this guard. +#[must_use] +pub struct RegistrationGuard<'a, T: drm::Driver> { + dev: &'a Device<T, Registered>, + idx: i32, + _not_send: NotThreadSafe, +} + +impl<T: drm::Driver> Device<T, Registered> { + /// Returns a reference to the registration data with lifetime shortened from `'static`. + /// + /// # Safety + /// + /// The returned reference must not be exposed to code that can choose a concrete lifetime for + /// it, as that would be unsound for types that are invariant over their lifetime parameter + /// (e.g. it must be passed through an HRTB-bounded closure). + #[inline] + unsafe fn registration_data_unchecked(&self) -> &T::RegistrationData<'_> { + // SAFETY: + // - `Registered` guarantees the parent bus device is bound, hence the pointer is valid. + // - The pointer cast from `Of<'static>` to `Of<'_>` is layout-compatible since lifetimes + // are erased at runtime. + // - Caller guarantees the reference is only used behind an HRTB, making the lifetime + // shortening sound regardless of variance. + unsafe { (*self.registration_data.get()).cast::<_>().as_ref() } + } + + /// Access the registration data through a closure, with the lifetime tied to the closure + /// scope. + /// + /// The data is owned by [`Registration`](drm::Registration) and is guaranteed to remain valid + /// as long as the device is registered, since [`Registration`](drm::Registration)'s `drop` + /// calls `drm_dev_unplug()` which waits for all `drm_dev_enter()` critical sections to + /// complete. + #[inline] + pub fn registration_data_with<R, F>(&self, f: F) -> R + where + F: for<'a> FnOnce(&'a T::RegistrationData<'a>) -> R, + { + // SAFETY: `Registered` guarantees the device is registered and the parent bus device is + // bound. The closure's HRTB `for<'a>` prevents the caller from smuggling in references + // with a concrete short lifetime, satisfying the lifetime requirement of + // `registration_data_unchecked`. + f(unsafe { self.registration_data_unchecked() }) + } +} + +impl<T: drm::Driver> Deref for RegistrationGuard<'_, T> { + type Target = Device<T, Registered>; + + #[inline] + fn deref(&self) -> &Self::Target { + self.dev + } +} + +impl<T: drm::Driver> Drop for RegistrationGuard<'_, T> { + #[inline] + fn drop(&mut self) { + // SAFETY: `self.idx` was returned by a successful `drm_dev_enter()` call, as guaranteed + // by the type invariants of `RegistrationGuard`. + unsafe { bindings::drm_dev_exit(self.idx) }; + } +} + +impl<T: drm::Driver> Deref for Device<T> { type Target = T::Data; fn deref(&self) -> &Self::Target { @@ -361,9 +456,31 @@ impl<T: drm::Driver, C: DeviceContext> Deref for Device<T, C> { } } +impl<T: drm::Driver> Deref for Device<T, Registered> { + type Target = Device<T>; + + #[inline] + fn deref(&self) -> &Self::Target { + // SAFETY: The caller holds a `Device<T, Registered>`, which guarantees all invariants + // of the weaker `Normal` context. + unsafe { self.assume_ctx() } + } +} + +impl<T: drm::Driver> Deref for Device<T, Ioctl> { + type Target = Device<T>; + + #[inline] + fn deref(&self) -> &Self::Target { + // SAFETY: The caller holds a `Device<T, Ioctl>`, which guarantees all invariants + // of the weaker `Normal` context. + unsafe { self.assume_ctx() } + } +} + // SAFETY: DRM device objects are always reference counted and the get/put functions // satisfy the requirements. -unsafe impl<T: drm::Driver, C: DeviceContext> AlwaysRefCounted for Device<T, C> { +unsafe impl<T: drm::Driver> AlwaysRefCounted for Device<T> { fn inc_ref(&self) { // SAFETY: The existence of a shared reference guarantees that the refcount is non-zero. unsafe { bindings::drm_dev_get(self.as_raw()) }; @@ -378,11 +495,29 @@ unsafe impl<T: drm::Driver, C: DeviceContext> AlwaysRefCounted for Device<T, C> } } -impl<T: drm::Driver, C: DeviceContext> AsRef<device::Device> for Device<T, C> { - fn as_ref(&self) -> &device::Device { +impl<T: drm::Driver> AsRef<T::ParentDevice<device::Normal>> for Device<T> { + fn as_ref(&self) -> &T::ParentDevice<device::Normal> { // SAFETY: `bindings::drm_device::dev` is valid as long as the DRM device itself is valid, // which is guaranteed by the type invariant. - unsafe { device::Device::from_raw((*self.as_raw()).dev) } + let dev = unsafe { device::Device::from_raw((*self.as_raw()).dev) }; + + // SAFETY: The DRM device was constructed in `UnregisteredDevice::new()` with a parent + // device of type `T::ParentDevice`, hence `dev` is contained in a `T::ParentDevice`. + unsafe { device::AsBusDevice::from_device(dev) } + } +} + +impl<T: drm::Driver> AsRef<T::ParentDevice<device::Bound>> for Device<T, Registered> { + #[inline] + fn as_ref(&self) -> &T::ParentDevice<device::Bound> { + let dev = (**self).as_ref().as_ref(); + + // SAFETY: A `Device<T, Registered>` guarantees that the parent device is bound. + let dev = unsafe { dev.as_bound() }; + + // SAFETY: The DRM device was constructed in `UnregisteredDevice::new()` with a parent + // device of type `T::ParentDevice`, hence `dev` is contained in a `T::ParentDevice`. + unsafe { device::AsBusDevice::from_device(dev) } } } @@ -393,12 +528,10 @@ unsafe impl<T: drm::Driver, C: DeviceContext> Send for Device<T, C> {} // by the synchronization in `struct drm_device`. unsafe impl<T: drm::Driver, C: DeviceContext> Sync for Device<T, C> {} -impl<T, C, const ID: u64> WorkItem<ID> for Device<T, C> +impl<T: drm::Driver, const ID: u64> WorkItem<ID> for Device<T> where - T: drm::Driver, T::Data: WorkItem<ID, Pointer = ARef<Self>>, T::Data: HasWork<Self, ID>, - C: DeviceContext, { type Pointer = ARef<Self>; diff --git a/rust/kernel/drm/driver.rs b/rust/kernel/drm/driver.rs index 25f7e233884d..74f6ed690d8b 100644 --- a/rust/kernel/drm/driver.rs +++ b/rust/kernel/drm/driver.rs @@ -7,16 +7,12 @@ use crate::{ bindings, device, - devres, drm, error::to_result, prelude::*, sync::aref::ARef, // }; -use core::{ - mem, - ptr::NonNull, // -}; +use core::ptr::NonNull; /// Driver use the GEM memory manager. This should be set for all modern drivers. pub(crate) const FEAT_GEM: u32 = bindings::drm_driver_feature_DRIVER_GEM; @@ -110,12 +106,23 @@ pub trait Driver { /// Context data associated with the DRM driver type Data: Sync + Send; + /// Data owned by the [`Registration`] and accessible within a + /// [`RegistrationGuard`](drm::RegistrationGuard) critical section via + /// [`Device::registration_data_with()`](drm::Device::registration_data_with). + /// + /// The lifetime parameter is tied to the [`Registration`] scope, which is enclosed in the + /// parent bus device binding scope but may be shorter. + type RegistrationData<'a>: Send + Sync + 'a; + /// The type used to manage memory for this driver. - type Object<Ctx: drm::DeviceContext>: AllocImpl; + type Object: AllocImpl; /// The type used to represent a DRM File (client) type File: drm::file::DriverFile; + /// The bus device type of the parent device that the DRM device is associated with. + type ParentDevice<Ctx: device::DeviceContext>: device::AsBusDevice<Ctx>; + /// Driver metadata const INFO: DriverInfo; @@ -125,7 +132,7 @@ pub trait Driver { /// Sets the `DRIVER_RENDER` feature for this driver. /// /// When enabled, the driver exposes `/dev/dri/renderDXX` render nodes to - /// userspace. The render node is an alternate low-priviledge way to access + /// userspace. The render node is an alternate low-privilege way to access /// the driver, which is enforced on a per-ioctl level. Userspace processes /// that open the render node can only invoke ioctls explicitly listed as /// usable from the render node (i.e. marked DRM_RENDER_ALLOW), whereas @@ -136,68 +143,84 @@ pub trait Driver { /// The registration type of a `drm::Device`. /// /// Once the `Registration` structure is dropped, the device is unregistered. -pub struct Registration<T: Driver>(ARef<drm::Device<T>>); - -impl<T: Driver> Registration<T> { - fn new(drm: drm::UnregisteredDevice<T>, flags: usize) -> Result<Self> { - // SAFETY: `drm.as_raw()` is valid by the invariants of `drm::Device`. - to_result(unsafe { bindings::drm_dev_register(drm.as_raw(), flags) })?; - - // SAFETY: We just called `drm_dev_register` above - let new = NonNull::from(unsafe { drm.assume_ctx() }); - - // Leak the ARef from UnregisteredDevice in preparation for transferring its ownership. - mem::forget(drm); - - // SAFETY: `drm`'s `Drop` constructor was never called, ensuring that there remains at least - // one reference to the device - which we take ownership over here. - let new = unsafe { ARef::from_raw(new) }; - - Ok(Self(new)) - } +pub struct Registration<'a, T: Driver> { + drm: ARef<drm::Device<T>>, + _reg_data: Pin<KBox<T::RegistrationData<'a>>>, +} - /// Registers a new [`UnregisteredDevice`](drm::UnregisteredDevice) with userspace. +impl<'a, T: Driver> Registration<'a, T> { + /// Register a new [`UnregisteredDevice`](drm::UnregisteredDevice) with userspace. /// - /// Ownership of the [`Registration`] object is passed to [`devres::register`]. - pub fn new_foreign_owned<'a>( - drm: drm::UnregisteredDevice<T>, + /// # Safety + /// + /// The caller must not `mem::forget()` the returned [`Registration`] or otherwise prevent its + /// [`Drop`] implementation from running, since the registration data may contain borrowed + /// references that become invalid after `'a` ends. + pub unsafe fn new<E>( dev: &'a device::Device<device::Bound>, + drm: drm::UnregisteredDevice<T>, + reg_data: impl PinInit<T::RegistrationData<'a>, E>, flags: usize, - ) -> Result<&'a drm::Device<T>> + ) -> Result<Self> where - T: 'static, + Error: From<E>, { - if drm.as_ref().as_raw() != dev.as_raw() { + let parent = drm.as_ref(); + if parent.as_ref().as_raw() != dev.as_raw() { return Err(EINVAL); } - let reg = Registration::<T>::new(drm, flags)?; - let drm = NonNull::from(reg.device()); + let reg_data: Pin<KBox<T::RegistrationData<'a>>> = KBox::pin_init(reg_data, GFP_KERNEL)?; + + // Store the registration data pointer in the device before registration, so that it is + // visible once ioctls can be called. + let ptr: NonNull<T::RegistrationData<'static>> = + NonNull::from(Pin::get_ref(reg_data.as_ref())).cast(); - devres::register(dev, reg, GFP_KERNEL)?; + // SAFETY: No concurrent access; the device is not yet registered. + unsafe { *drm.registration_data.get() = ptr }; + + // SAFETY: `drm` is a valid, initialized but not yet registered DRM device. + let ret = unsafe { bindings::drm_dev_register(drm.as_raw(), flags) }; + if let Err(e) = to_result(ret) { + // SAFETY: `drm_dev_register()` synchronizes SRCU on failure, so no concurrent + // access to `registration_data` is possible at this point. + unsafe { *drm.registration_data.get() = NonNull::dangling() }; + return Err(e); + } - // SAFETY: Since `reg` was passed to devres::register(), the device now owns the lifetime - // of the DRM registration - ensuring that this references lives for at least as long as 'a. - Ok(unsafe { drm.as_ref() }) + Ok(Self { + drm: (&*drm).into(), + _reg_data: reg_data, + }) } /// Returns a reference to the `Device` instance for this registration. pub fn device(&self) -> &drm::Device<T> { - &self.0 + &self.drm } } // SAFETY: `Registration` doesn't offer any methods or access to fields when shared between // threads, hence it's safe to share it. -unsafe impl<T: Driver> Sync for Registration<T> {} +unsafe impl<T: Driver> Sync for Registration<'_, T> {} // SAFETY: Registration with and unregistration from the DRM subsystem can happen from any thread. -unsafe impl<T: Driver> Send for Registration<T> {} +unsafe impl<T: Driver> Send for Registration<'_, T> {} -impl<T: Driver> Drop for Registration<T> { +impl<T: Driver> Drop for Registration<'_, T> { fn drop(&mut self) { + // Use `drm_dev_unplug` rather than `drm_dev_unregister` to ensure that existing + // `drm_dev_enter()` critical sections complete before unregistration proceeds. This + // is required for the safety of `RegistrationGuard`, which relies on the SRCU barrier in + // `drm_dev_unplug()` to guarantee that the parent device is still bound within the + // critical section. + // // SAFETY: Safe by the invariant of `ARef<drm::Device<T>>`. The existence of this - // `Registration` also guarantees the this `drm::Device` is actually registered. - unsafe { bindings::drm_dev_unregister(self.0.as_raw()) }; + // `Registration` also guarantees that this `drm::Device` is actually registered. + unsafe { bindings::drm_dev_unplug(self.drm.as_raw()) }; + // After drm_dev_unplug(), the SRCU barrier guarantees that all RegistrationGuard critical + // sections have completed, so no one holds a reference to reg_data anymore. + // reg_data is dropped here automatically. } } diff --git a/rust/kernel/drm/gem/mod.rs b/rust/kernel/drm/gem/mod.rs index a7ba1453d40b..e1ebad77ebe2 100644 --- a/rust/kernel/drm/gem/mod.rs +++ b/rust/kernel/drm/gem/mod.rs @@ -10,7 +10,7 @@ use crate::{ self, device::{ DeviceContext, - Registered, // + Normal, // }, driver::{ AllocImpl, @@ -81,11 +81,10 @@ pub type DriverFile<T> = drm::File<<<T as DriverObject>::Driver as drm::Driver>: /// A type alias for retrieving the current [`AllocImpl`] for a given [`DriverObject`]. /// /// [`Driver`]: drm::Driver -pub type DriverAllocImpl<T, Ctx = Registered> = - <<T as DriverObject>::Driver as drm::Driver>::Object<Ctx>; +pub type DriverAllocImpl<T> = <<T as DriverObject>::Driver as drm::Driver>::Object; /// GEM object functions, which must be implemented by drivers. -pub trait DriverObject: Sync + Send + Sized { +pub trait DriverObject: Sync + Send + Sized + 'static { /// Parent `Driver` for this object. type Driver: drm::Driver; @@ -93,8 +92,8 @@ pub trait DriverObject: Sync + Send + Sized { type Args; /// Create a new driver data object for a GEM object of a given size. - fn new<Ctx: DeviceContext>( - dev: &drm::Device<Self::Driver, Ctx>, + fn new( + dev: &drm::Device<Self::Driver>, size: usize, args: Self::Args, ) -> impl PinInit<Self, Error>; @@ -109,7 +108,7 @@ pub trait DriverObject: Sync + Send + Sized { } /// Trait that represents a GEM object subtype -pub trait IntoGEMObject: Sized + super::private::Sealed + AlwaysRefCounted { +pub trait IntoGEMObject: Sized + super::private::Sealed { /// Returns a reference to the raw `drm_gem_object` structure, which must be valid as long as /// this owning object is valid. fn as_raw(&self) -> *mut bindings::drm_gem_object; @@ -118,7 +117,8 @@ pub trait IntoGEMObject: Sized + super::private::Sealed + AlwaysRefCounted { /// /// # Safety /// - /// - `self_ptr` must be a valid pointer to `Self`. + /// - `self_ptr` must be a valid pointer to the `struct drm_gem_object` embedded in a + /// valid instance of `Self`. /// - The caller promises that holding the immutable reference returned by this function does /// not violate rust's data aliasing rules and remains valid throughout the lifetime of `'a`. unsafe fn from_raw<'a>(self_ptr: *mut bindings::drm_gem_object) -> &'a Self; @@ -183,7 +183,7 @@ pub trait BaseObject: IntoGEMObject { fn create_handle<D, F>(&self, file: &drm::File<F>) -> Result<u32> where Self: AllocImpl<Driver = D>, - D: drm::Driver<Object<Registered> = Self, File = F>, + D: drm::Driver<Object = Self, File = F>, F: drm::file::DriverFile<Driver = D>, { let mut handle: u32 = 0; @@ -197,8 +197,8 @@ pub trait BaseObject: IntoGEMObject { /// Looks up an object by its handle for a given `File`. fn lookup_handle<D, F>(file: &drm::File<F>, handle: u32) -> Result<ARef<Self>> where - Self: AllocImpl<Driver = D>, - D: drm::Driver<Object<Registered> = Self, File = F>, + Self: AllocImpl<Driver = D> + AlwaysRefCounted, + D: drm::Driver<Object = Self, File = F>, F: drm::file::DriverFile<Driver = D>, { // SAFETY: The arguments are all valid per the type invariants. @@ -254,7 +254,7 @@ impl<T: IntoGEMObject> BaseObjectPrivate for T {} /// * Any type invariants of `Ctx` apply to the parent DRM device for this GEM object. #[repr(C)] #[pin_data] -pub struct Object<T: DriverObject + Send + Sync, Ctx: DeviceContext = Registered> { +pub struct Object<T: DriverObject + Send + Sync, Ctx: DeviceContext = Normal> { obj: Opaque<bindings::drm_gem_object>, #[pin] data: T, @@ -280,12 +280,43 @@ impl<T: DriverObject, Ctx: DeviceContext> Object<T, Ctx> { rss: None, }; + /// Returns the `Device` that owns this GEM object. + pub fn dev(&self) -> &drm::Device<T::Driver, Ctx> { + // SAFETY: + // - `struct drm_gem_object.dev` is initialized and valid for as long as the GEM + // object lives. + // - The device we used for creating the gem object is passed as &drm::Device<T::Driver> to + // Object::<T>::new(), so we know that `T::Driver` is the right generic parameter to use + // here. + // - Any type invariants of `Ctx` are upheld by using the same `Ctx` for the `Device` we + // return. + unsafe { drm::Device::from_raw((*self.as_raw()).dev) } + } + + fn as_raw(&self) -> *mut bindings::drm_gem_object { + self.obj.get() + } + + extern "C" fn free_callback(obj: *mut bindings::drm_gem_object) { + let ptr: *mut Opaque<bindings::drm_gem_object> = obj.cast(); + + // SAFETY: All of our objects are of type `Object<T>`. + let this = unsafe { crate::container_of!(ptr, Self, obj) }; + + // SAFETY: The C code only ever calls this callback with a valid pointer to a `struct + // drm_gem_object`. + unsafe { bindings::drm_gem_object_release(obj) }; + + // SAFETY: All of our objects are allocated via `KBox`, and we're in the + // free callback which guarantees this object has zero remaining references, + // so we can drop it. + let _ = unsafe { KBox::from_raw(this) }; + } +} + +impl<T: DriverObject> Object<T> { /// Create a new GEM object. - pub fn new( - dev: &drm::Device<T::Driver, Ctx>, - size: usize, - args: T::Args, - ) -> Result<ARef<Self>> { + pub fn new(dev: &drm::Device<T::Driver>, size: usize, args: T::Args) -> Result<ARef<Self>> { let obj: Pin<KBox<Self>> = KBox::pin_init( try_pin_init!(Self { obj: Opaque::new(bindings::drm_gem_object::default()), @@ -321,46 +352,12 @@ impl<T: DriverObject, Ctx: DeviceContext> Object<T, Ctx> { // SAFETY: We take over the initial reference count from `drm_gem_object_init()`. Ok(unsafe { ARef::from_raw(ptr) }) } - - /// Returns the `Device` that owns this GEM object. - pub fn dev(&self) -> &drm::Device<T::Driver, Ctx> { - // SAFETY: - // - `struct drm_gem_object.dev` is initialized and valid for as long as the GEM - // object lives. - // - The device we used for creating the gem object is passed as &drm::Device<T::Driver> to - // Object::<T>::new(), so we know that `T::Driver` is the right generic parameter to use - // here. - // - Any type invariants of `Ctx` are upheld by using the same `Ctx` for the `Device` we - // return. - unsafe { drm::Device::from_raw((*self.as_raw()).dev) } - } - - fn as_raw(&self) -> *mut bindings::drm_gem_object { - self.obj.get() - } - - extern "C" fn free_callback(obj: *mut bindings::drm_gem_object) { - let ptr: *mut Opaque<bindings::drm_gem_object> = obj.cast(); - - // SAFETY: All of our objects are of type `Object<T>`. - let this = unsafe { crate::container_of!(ptr, Self, obj) }; - - // SAFETY: The C code only ever calls this callback with a valid pointer to a `struct - // drm_gem_object`. - unsafe { bindings::drm_gem_object_release(obj) }; - - // SAFETY: All of our objects are allocated via `KBox`, and we're in the - // free callback which guarantees this object has zero remaining references, - // so we can drop it. - let _ = unsafe { KBox::from_raw(this) }; - } } impl_aref_for_gem_obj! { - impl<T, C> for Object<T, C> + impl<T> for Object<T> where - T: DriverObject, - C: DeviceContext + T: DriverObject } impl<T: DriverObject, Ctx: DeviceContext> super::private::Sealed for Object<T, Ctx> {} diff --git a/rust/kernel/drm/gem/shmem.rs b/rust/kernel/drm/gem/shmem.rs index 34af402899a0..a687d46d170d 100644 --- a/rust/kernel/drm/gem/shmem.rs +++ b/rust/kernel/drm/gem/shmem.rs @@ -11,28 +11,57 @@ use crate::{ container_of, + device::{ + self, + Bound, // + }, + devres::*, drm::{ driver, gem, private::Sealed, - Device, - DeviceContext, - Registered, // + Device, // + }, + error::{ + from_err_ptr, + to_result, // + }, + io::{ + IoBase, + Region, + SysMem, + SysMemBackend, // }, - error::to_result, prelude::*, - sync::aref::ARef, - types::Opaque, // + scatterlist, + sync::{ + aref::ARef, + new_mutex, + Mutex, + SetOnce, // + }, + types::{ + NotThreadSafe, + Opaque, // + }, }; use core::{ - marker::PhantomData, + ffi::c_void, + mem::{ + ManuallyDrop, + MaybeUninit, // + }, ops::{ Deref, DerefMut, // }, - ptr::NonNull, // + ptr::{ + self, + NonNull, // + }, }; use gem::{ + BaseObject, BaseObjectPrivate, DriverObject, IntoGEMObject, // @@ -42,15 +71,24 @@ use gem::{ /// /// This is used with [`Object::new()`] to control various properties that can only be set when /// initially creating a shmem-backed GEM object. -#[derive(Default)] -pub struct ObjectConfig<'a, T: DriverObject, C: DeviceContext = Registered> { +pub struct ObjectConfig<'a, T: DriverObject> { /// Whether to set the write-combine map flag. pub map_wc: bool, /// Reuse the DMA reservation from another GEM object. /// /// The newly created [`Object`] will hold an owned refcount to `parent_resv_obj` if specified. - pub parent_resv_obj: Option<&'a Object<T, C>>, + pub parent_resv_obj: Option<&'a Object<T>>, +} + +impl<'a, T: DriverObject> Default for ObjectConfig<'a, T> { + #[inline(always)] + fn default() -> Self { + Self { + map_wc: false, + parent_resv_obj: None, + } + } } /// A shmem-backed GEM object. @@ -59,33 +97,35 @@ pub struct ObjectConfig<'a, T: DriverObject, C: DeviceContext = Registered> { /// /// - `obj` contains a valid initialized `struct drm_gem_shmem_object` for the lifetime of this /// object. -/// - Any type invariants of `C` apply to the parent DRM device for this GEM object. #[repr(C)] #[pin_data] -pub struct Object<T: DriverObject, C: DeviceContext = Registered> { +pub struct Object<T: DriverObject> { #[pin] obj: Opaque<bindings::drm_gem_shmem_object>, /// Parent object that owns this object's DMA reservation object. - parent_resv_obj: Option<ARef<Object<T, C>>>, + parent_resv_obj: Option<ARef<Object<T>>>, + /// Devres object for unmapping any SGTable on driver-unbind. + sgt_res: ManuallyDrop<SetOnce<Devres<SGTableMap<T>>>>, + #[pin] + /// Lock for protecting initialization of `sgt_res`. + sgt_lock: Mutex<()>, #[pin] inner: T, - _ctx: PhantomData<C>, } super::impl_aref_for_gem_obj! { - impl<T, C> for Object<T, C> + impl<T> for Object<T> where - T: DriverObject, - C: DeviceContext + T: DriverObject } // SAFETY: All GEM objects are thread-safe. -unsafe impl<T: DriverObject, C: DeviceContext> Send for Object<T, C> {} +unsafe impl<T: DriverObject> Send for Object<T> {} // SAFETY: All GEM objects are thread-safe. -unsafe impl<T: DriverObject, C: DeviceContext> Sync for Object<T, C> {} +unsafe impl<T: DriverObject> Sync for Object<T> {} -impl<T: DriverObject, C: DeviceContext> Object<T, C> { +impl<T: DriverObject> Object<T> { /// `drm_gem_object_funcs` vtable suitable for GEM shmem objects. const VTABLE: bindings::drm_gem_object_funcs = bindings::drm_gem_object_funcs { free: Some(Self::free_callback), @@ -112,21 +152,166 @@ impl<T: DriverObject, C: DeviceContext> Object<T, C> { self.obj.get() } + /// Returns the `Device` that owns this GEM object. + pub fn dev(&self) -> &Device<T::Driver> { + // SAFETY: `dev` will have been initialized in `Self::new()` by `drm_gem_shmem_init()`. + unsafe { Device::from_raw((*self.as_raw()).dev) } + } + + extern "C" fn free_callback(obj: *mut bindings::drm_gem_object) { + // SAFETY: + // - DRM always passes a valid gem object here + // - We used drm_gem_shmem_create() in our create_gem_object callback, so we know that + // `obj` is contained within a drm_gem_shmem_object + let base = unsafe { container_of!(obj, bindings::drm_gem_shmem_object, base) }; + + // SAFETY: + // - We verified above that `obj` is valid, which makes `this` valid + // - This function is set in AllocOps, so we know that `this` is contained within an + // `Object<T>` + let this = unsafe { container_of!(Opaque::cast_from(base), Self, obj) }.cast_mut(); + + // We need to drop `sgt_res` first, since doing so requires that the GEM object is still + // alive. + // SAFETY: + // - We verified above that `this` is valid. + // - We are in free_callback, guaranteeing we have exclusive access to `this` and that + // `sgt_res` will not be used after dropping it here. + unsafe { ManuallyDrop::drop(&mut (*this).sgt_res) }; + + // SAFETY: + // - We're in free_callback - so this function is safe to call. + // - We won't be using the gem resources on `this` after this call. + unsafe { bindings::drm_gem_shmem_release(base) }; + + // SAFETY: We're recovering the Kbox<> we created in gem_create_object() + let _ = unsafe { KBox::from_raw(this) }; + } + + /// Attempt to create a vmap from the gem object, and confirm the size of said vmap. + fn make_vmap<'a, R, const SIZE: usize>(&'a self) -> Result<VMap<T, R, SIZE>> + where + R: Deref<Target = Self> + From<&'a Self>, + { + // INVARIANT: We check here that the gem object is at least as large as `SIZE`. + if self.size() < SIZE { + return Err(ENOSPC); + } + + let mut map: MaybeUninit<bindings::iosys_map> = MaybeUninit::uninit(); + let guard = DmaResvGuard::new(self); + + // SAFETY: `drm_gem_shmem_vmap()` can be called with the DMA reservation lock held. + to_result(unsafe { + bindings::drm_gem_shmem_vmap_locked(self.as_raw_shmem(), map.as_mut_ptr()) + })?; + + // Drop the guard explicitly here, since we may need to call `raw_vunmap()` (which + // re-acquires the lock). + drop(guard); + + // SAFETY: The call to `drm_gem_shmem_vmap_locked()` succeeded above, so we are guaranteed + // that map is properly initialized. + let map = unsafe { map.assume_init() }; + + // XXX: We don't currently support iomem allocations + if map.is_iomem { + // SAFETY: The vmap operation above succeeded, guaranteeing that `map` points to a valid + // memory mapping. + unsafe { self.raw_vunmap(map) }; + + Err(ENOTSUPP) + } else { + Ok(VMap { + // INVARIANT: `addr` remains valid for as long as `owner` does, which extends to the + // lifetime of `VMap` itself. + // SAFETY: We checked that this is not an iomem allocation, making it safe to read + // vaddr. + addr: unsafe { map.__bindgen_anon_1.vaddr }, + owner: self.into(), + }) + } + } + + /// Unmap a vmap from the gem object. + /// + /// # Safety + /// + /// - The caller promises that `map` is a valid vmap on this gem object. + /// - The caller promises that the memory pointed to by map will no longer be accesed through + /// this instance. + unsafe fn raw_vunmap(&self, mut map: bindings::iosys_map) { + let _guard = DmaResvGuard::new(self); + + // SAFETY: + // - This function is safe to call with the DMA reservation lock held. + // - The caller promises that `map` is a valid vmap on this gem object. + unsafe { bindings::drm_gem_shmem_vunmap_locked(self.as_raw_shmem(), &mut map) }; + } + + /// Creates and returns a virtual kernel memory mapping for this object. + #[inline] + pub fn vmap<const SIZE: usize>(&self) -> Result<VMapRef<'_, T, SIZE>> { + self.make_vmap() + } + + /// Creates (if necessary) and returns an immutable reference to a scatter-gather table of DMA + /// pages for this object. + /// + /// This will pin the object in memory. It is expected that `dev` should be a pointer to the + /// same [`device::Device`] which `self` belongs to, otherwise this function will return + /// `Err(EINVAL)`. + pub fn sg_table<'a>( + &'a self, + dev: &'a device::Device<Bound>, + ) -> Result<&'a scatterlist::SGTable> { + let parent = self.dev().as_ref(); + if dev.as_raw() != parent.as_ref().as_raw() { + return Err(EINVAL); + } + + let sgt_res = 'out: { + // Fast path: sgt_res is already initialized + if let Some(sgt_res) = self.sgt_res.as_ref() { + break 'out sgt_res; + } + + // Slow path: Grab the lock and see if we need to initialize sgt_res. + let _guard = self.sgt_lock.lock(); + + // If someone initialized it while we were waiting, we can exit early. + if let Some(sgt_res) = self.sgt_res.as_ref() { + break 'out sgt_res; + } + + // If not, finish initializing and return. `populate()` cannot return false, as + // `sgt_res` must be unpopulated, and we must hold `sgt_lock` to reach this point. + self.sgt_res + .populate(Devres::new(dev, SGTableMap::new(self))?); + + // SAFETY: We just populated sgt_res above. + unsafe { self.sgt_res.as_ref().unwrap_unchecked() } + }; + + Ok(sgt_res.access(dev)?) + } + /// Create a new shmem-backed DRM object of the given size. /// /// Additional config options can be specified using `config`. pub fn new( - dev: &Device<T::Driver, C>, + dev: &Device<T::Driver>, size: usize, - config: ObjectConfig<'_, T, C>, + config: ObjectConfig<'_, T>, args: T::Args, ) -> Result<ARef<Self>> { let new: Pin<KBox<Self>> = KBox::try_pin_init( try_pin_init!(Self { obj <- Opaque::init_zeroed(), parent_resv_obj: config.parent_resv_obj.map(|p| p.into()), + sgt_res: ManuallyDrop::new(SetOnce::new()), + sgt_lock <- new_mutex!(()), inner <- T::new(dev, size, args), - _ctx: PhantomData::<C>, }), GFP_KERNEL, )?; @@ -158,36 +343,14 @@ impl<T: DriverObject, C: DeviceContext> Object<T, C> { Ok(obj) } - /// Returns the `Device` that owns this GEM object. - pub fn dev(&self) -> &Device<T::Driver, C> { - // SAFETY: `dev` will have been initialized in `Self::new()` by `drm_gem_shmem_init()`. - unsafe { Device::from_raw((*self.as_raw()).dev) } - } - - extern "C" fn free_callback(obj: *mut bindings::drm_gem_object) { - // SAFETY: - // - DRM always passes a valid gem object here - // - We used drm_gem_shmem_create() in our create_gem_object callback, so we know that - // `obj` is contained within a drm_gem_shmem_object - let this = unsafe { container_of!(obj, bindings::drm_gem_shmem_object, base) }; - - // SAFETY: - // - We're in free_callback - so this function is safe to call. - // - We won't be using the gem resources on `this` after this call. - unsafe { bindings::drm_gem_shmem_release(this) }; - - // SAFETY: - // - We verified above that `obj` is valid, which makes `this` valid - // - This function is set in AllocOps, so we know that `this` is contained within a - // `Object<T, C>` - let this = unsafe { container_of!(Opaque::cast_from(this), Self, obj) }.cast_mut(); - - // SAFETY: We're recovering the Kbox<> we created in gem_create_object() - let _ = unsafe { KBox::from_raw(this) }; + /// Creates and returns an owned reference to a virtual kernel memory mapping for this object. + #[inline] + pub fn owned_vmap<const SIZE: usize>(&self) -> Result<VMapOwned<T, SIZE>> { + self.make_vmap() } } -impl<T: DriverObject, C: DeviceContext> Deref for Object<T, C> { +impl<T: DriverObject> Deref for Object<T> { type Target = T; fn deref(&self) -> &Self::Target { @@ -195,15 +358,15 @@ impl<T: DriverObject, C: DeviceContext> Deref for Object<T, C> { } } -impl<T: DriverObject, C: DeviceContext> DerefMut for Object<T, C> { +impl<T: DriverObject> DerefMut for Object<T> { fn deref_mut(&mut self) -> &mut Self::Target { &mut self.inner } } -impl<T: DriverObject, C: DeviceContext> Sealed for Object<T, C> {} +impl<T: DriverObject> Sealed for Object<T> {} -impl<T: DriverObject, C: DeviceContext> gem::IntoGEMObject for Object<T, C> { +impl<T: DriverObject> gem::IntoGEMObject for Object<T> { fn as_raw(&self) -> *mut bindings::drm_gem_object { // SAFETY: // - Our immutable reference is proof that this is safe to dereference. @@ -222,7 +385,7 @@ impl<T: DriverObject, C: DeviceContext> gem::IntoGEMObject for Object<T, C> { } } -impl<T: DriverObject, C: DeviceContext> driver::AllocImpl for Object<T, C> { +impl<T: DriverObject> driver::AllocImpl for Object<T> { type Driver = T::Driver; const ALLOC_OPS: driver::AllocOps = driver::AllocOps { @@ -235,3 +398,324 @@ impl<T: DriverObject, C: DeviceContext> driver::AllocImpl for Object<T, C> { dumb_map_offset: None, }; } + +/// Private helper-type for holding the `dma_resv` object for a GEM shmem object. +/// +/// When this is dropped, the `dma_resv` lock is dropped as well. +/// +// TODO: This should be replace with a WwMutex equivalent once we have such bindings in the kernel. +struct DmaResvGuard<'a, T: DriverObject>(&'a Object<T>, NotThreadSafe); + +impl<'a, T: DriverObject> DmaResvGuard<'a, T> { + #[inline] + fn new(obj: &'a Object<T>) -> Self { + // SAFETY: This lock is initialized throughout the lifetime of `object`. + unsafe { bindings::dma_resv_lock(obj.raw_dma_resv(), ptr::null_mut()) }; + + Self(obj, NotThreadSafe) + } +} + +impl<'a, T: DriverObject> Drop for DmaResvGuard<'a, T> { + #[inline] + fn drop(&mut self) { + // SAFETY: We are releasing the lock grabbed during the creation of this object. + unsafe { bindings::dma_resv_unlock(self.0.raw_dma_resv()) }; + } +} + +/// A reference to a virtual mapping for an shmem-based GEM object in kernel address space. +/// +/// # Invariants +/// +/// - The size of `owner` is >= SIZE. +/// - The memory pointed to by `addr` remains valid at least until this object is dropped. +pub struct VMap<D, R, const SIZE: usize = 0> +where + D: DriverObject, + R: Deref<Target = Object<D>>, +{ + addr: *mut c_void, + owner: R, +} + +/// An alias type for a reference to a shmem-based GEM object's VMap. +pub type VMapRef<'a, D, const SIZE: usize = 0> = VMap<D, &'a Object<D>, SIZE>; + +/// An alias type for an owned reference to a shmem-based GEM object's VMap. +pub type VMapOwned<D, const SIZE: usize = 0> = VMap<D, ARef<Object<D>>, SIZE>; + +impl<D, R, const SIZE: usize> VMap<D, R, SIZE> +where + D: DriverObject, + R: Deref<Target = Object<D>>, +{ + /// Borrows a reference to the object that owns this virtual mapping. + #[inline] + pub fn owner(&self) -> &Object<D> { + &self.owner + } +} + +impl<'a, D, R, const SIZE: usize> IoBase<'a> for &'a VMap<D, R, SIZE> +where + D: DriverObject, + R: Deref<Target = Object<D>>, +{ + type Backend = SysMemBackend; + type Target = Region<SIZE>; + + #[inline] + fn as_view(self) -> SysMem<'a, Region<SIZE>> { + let ptr = Region::ptr_from_raw_parts_mut(self.addr.cast(), self.owner.size()); + + // SAFETY: Per type invariants of `VMap`: + // - `addr .. addr + owner.size()` is a valid kernel accessible memory region. + // - `addr` is page-aligned, which satisfies `Region`'s 4-byte alignment requirement. + // - The memory remains valid until this `VMap` is dropped; since `self` is `&'a VMap`, + // the borrow prevents the `VMap` from being dropped for the lifetime `'a`. + unsafe { SysMem::new(ptr) } + } +} + +impl<D, R, const SIZE: usize> Drop for VMap<D, R, SIZE> +where + D: DriverObject, + R: Deref<Target = Object<D>>, +{ + #[inline] + fn drop(&mut self) { + // SAFETY: + // - Our existence is proof that this map was previously created using self.owner. + // - Since we are in Drop, we are guaranteed that no one will access the memory + // through this mapping after calling this. + unsafe { + self.owner.raw_vunmap(bindings::iosys_map { + is_iomem: false, + __bindgen_anon_1: bindings::iosys_map__bindgen_ty_1 { vaddr: self.addr }, + }) + }; + } +} + +// SAFETY: `addr` points to a valid memory address for as long as `owner` exists, meaning that so +// long as `owner` is `Send` so is `VMap`. +unsafe impl<D, R, const SIZE: usize> Send for VMap<D, R, SIZE> +where + D: DriverObject, + R: Deref<Target = Object<D>> + Send, +{ +} + +// SAFETY: `addr` points to a valid memory address for as long as `owner` exists, meaning that so +// long as `owner` is `Sync` so is `VMap`. +unsafe impl<D, R, const SIZE: usize> Sync for VMap<D, R, SIZE> +where + D: DriverObject, + R: Deref<Target = Object<D>> + Sync, +{ +} + +/// A reference to a GEM object that is known to have a mapped [`SGTable`]. +/// +/// This is used by the Rust bindings with [`Devres`] in order to ensure that mappings for SGTables +/// on GEM shmem objects are revoked on driver-unbind. +/// +/// # Invariants +/// +/// - `self.obj` always points to a valid GEM object. +/// - This object is proof that `self.obj.owner.sgt_res` has an initialized and valid pointer to an +/// [`SGTable`]. +/// +/// [`SGTable`]: scatterlist::SGTable +pub struct SGTableMap<T: DriverObject> { + obj: NonNull<Object<T>>, +} + +impl<T: DriverObject> Deref for SGTableMap<T> { + type Target = scatterlist::SGTable; + + fn deref(&self) -> &Self::Target { + // SAFETY: + // - The NonNull is guaranteed to be valid via our type invariants. + // - The sgt field is guaranteed to be initialized and valid via our type invariants. + unsafe { scatterlist::SGTable::from_raw((*self.obj.as_ref().as_raw_shmem()).sgt) } + } +} + +impl<T: DriverObject> Drop for SGTableMap<T> { + fn drop(&mut self) { + // SAFETY: `obj` is always valid via our type invariants + let obj = unsafe { self.obj.as_ref() }; + let _lock = DmaResvGuard::new(obj); + + // SAFETY: We acquired the lock needed for calling this function above + unsafe { bindings::__drm_gem_shmem_free_sgt_locked(obj.as_raw_shmem()) }; + } +} + +impl<T: DriverObject> SGTableMap<T> { + fn new(obj: &Object<T>) -> impl Init<Self, Error> { + // INVARIANT: + // - We call drm_gem_shmem_get_pages_sgt below and check whether or not it succeeds, + // fulfilling the invariant of SGTableMap that the object's `sgt` field is initialized. + // SAFETY: + // - `obj` is fully initialized, making this function safe to call. + from_err_ptr(unsafe { bindings::drm_gem_shmem_get_pages_sgt(obj.as_raw_shmem()) })?; + + Ok(Self { obj: obj.into() }) + } +} + +// SAFETY: The NonNull in SGTableMap is guaranteed valid by our type invariants, and the GEM object +// it points to is guaranteed to be thread-safe. +unsafe impl<T: DriverObject> Send for SGTableMap<T> {} +// SAFETY: The NonNull in SGTableMap is guaranteed valid by our type invariants, and the GEM object +// it points to is guaranteed to be thread-safe. +unsafe impl<T: DriverObject> Sync for SGTableMap<T> {} + +#[kunit_tests(rust_drm_gem_shmem)] +mod tests { + use super::*; + use crate::{ + drm::{ + self, + UnregisteredDevice, // + }, + faux, + io::Io, + page::PAGE_SIZE, // + }; + + // The bare minimum needed to create a fake drm driver for kunit + + #[pin_data] + struct KunitData {} + struct KunitDriver; + struct KunitFile; + #[pin_data] + struct KunitObject {} + + const INFO: drm::DriverInfo = drm::DriverInfo { + major: 0, + minor: 0, + patchlevel: 0, + name: c"kunit", + desc: c"Kunit", + }; + + impl drm::file::DriverFile for KunitFile { + type Driver = KunitDriver; + + fn open(_dev: &drm::Device<KunitDriver>) -> Result<Pin<KBox<Self>>> { + Ok(KBox::new(Self, GFP_KERNEL)?.into()) + } + } + + impl gem::DriverObject for KunitObject { + type Driver = KunitDriver; + type Args = (); + + fn new( + _dev: &drm::Device<KunitDriver>, + _size: usize, + _args: Self::Args, + ) -> impl PinInit<Self, Error> { + try_pin_init!(KunitObject {}) + } + } + + #[vtable] + impl drm::Driver for KunitDriver { + type Data = KunitData; + type RegistrationData<'a> = (); + type File = KunitFile; + type Object = Object<KunitObject>; + type ParentDevice<Ctx: device::DeviceContext> = faux::Device<Ctx>; + + const INFO: drm::DriverInfo = INFO; + const IOCTLS: &'static [drm::ioctl::DrmIoctlDescriptor] = &[]; + } + + fn create_drm_dev() -> Result<(faux::Registration, UnregisteredDevice<KunitDriver>)> { + // Create a faux DRM device so we can test gem object creation. + let data = try_pin_init!(KunitData {}); + let reg = faux::Registration::new(c"Kunit", None)?; + let fdev = reg.as_ref(); + let drm = UnregisteredDevice::new(fdev, data)?; + + Ok((reg, drm)) + } + + #[test] + fn compile_time_vmap_sizes() -> Result { + let (_dev, drm) = create_drm_dev()?; + + let obj = Object::<KunitObject>::new(&drm, PAGE_SIZE, ObjectConfig::default(), ())?; + + // Try creating a normal vmap + obj.vmap::<PAGE_SIZE>()?; + + // Try creating a vmap that's smaller then the size we specified + let vmap = obj.vmap::<{ PAGE_SIZE - 100 }>()?; + + // Verify the owner matches + assert!(ptr::eq(vmap.owner(), obj.deref())); + + // Verify the size matches the actual object size + assert_eq!(vmap.size(), PAGE_SIZE); + + // Make sure creating a vmap that's too large fails + assert!(obj.vmap::<{ PAGE_SIZE + 200 }>().is_err()); + + Ok(()) + } + + #[test] + fn vmap_io() -> Result { + let (_dev, drm) = create_drm_dev()?; + + let obj = Object::<KunitObject>::new(&drm, PAGE_SIZE, ObjectConfig::default(), ())?; + + let vmap = obj.vmap::<PAGE_SIZE>()?; + + vmap.write8(0xDE, 0x0); + assert_eq!(vmap.read8(0x0), 0xDE); + vmap.write32(0xFEDCBA98, 0x20); + + assert_eq!(vmap.read32(0x20), 0xFEDCBA98); + + // Ensure the ordering in memory is correct + let expected = 0xFEDCBA98_u32.to_ne_bytes().into_iter(); + for (offset, expected) in (0x20..=0x23).zip(expected) { + assert_eq!(vmap.try_read8(offset).unwrap(), expected); + } + + Ok(()) + } + + // TODO: I would love to actually test the success paths of sg_table(), but that would require + // also implementing dummy dma_ops so that trying to create a mapping doesn't explode. So, leave + // that for someone else. + + // Ensures that passing the wrong device to sg_table() fails as we expect, and also ensure it + // skips initializing `sgt_res` since we could otherwise create `sgt_res` with the wrong device + // bound to it. + #[test] + fn fail_sg_table_on_wrong_dev() -> Result { + let (_dev, drm) = create_drm_dev()?; + let reg = faux::Registration::new(c"EvilKunit", None)?; + let wrong_dev = reg.as_ref(); + + let obj = Object::<KunitObject>::new(&drm, PAGE_SIZE, ObjectConfig::default(), ())?; + + assert_eq!(obj.sg_table(wrong_dev.as_ref()).err().unwrap(), EINVAL); + + // If sgt_res was not initialized mistakenly with the wrong device, this should still fail. + assert_eq!(obj.sg_table(wrong_dev.as_ref()).err().unwrap(), EINVAL); + + // TODO: Someday, we should test that creating an sg_table here still succeeds. + + Ok(()) + } +} diff --git a/rust/kernel/drm/gpuvm/mod.rs b/rust/kernel/drm/gpuvm/mod.rs index ae58f6f667c1..d9d43d719761 100644 --- a/rust/kernel/drm/gpuvm/mod.rs +++ b/rust/kernel/drm/gpuvm/mod.rs @@ -72,10 +72,12 @@ pub struct GpuVm<T: DriverGpuVm> { data: UnsafeCell<T>, } -// SAFETY: The GPUVM api does not assume that it is tied to a specific thread. The destructor will -// drop the `data` field, which is okay because it is guaranteed `Send` by the `DriverGpuVm` trait. +// SAFETY: It is safe to send a `GpuVm<T>` to another thread: all data reachable through it +// (`T`, `T::VmBoData`, and the GEM `T::Object`) is `Send` by the `DriverGpuVm` bounds. unsafe impl<T: DriverGpuVm> Send for GpuVm<T> {} -// SAFETY: The GPUVM api is designed to allow &self methods to be called in parallel. +// SAFETY: It is safe to share a `&GpuVm<T>` between threads: `&self` methods only alias data +// that is `Sync` by the `DriverGpuVm` bounds, and any thread may drop that data, or upgrade the +// reference and ultimately drop `T`, which the same bounds make `Send`. unsafe impl<T: DriverGpuVm> Sync for GpuVm<T> {} // SAFETY: By type invariants, the allocation is managed by the refcount in `self.vm`. @@ -116,9 +118,9 @@ impl<T: DriverGpuVm> GpuVm<T> { /// Creates a GPUVM instance. #[expect(clippy::new_ret_no_self)] - pub fn new<E>( + pub fn new<E, Ctx: drm::DeviceContext>( name: &'static CStr, - dev: &drm::Device<T::Driver>, + dev: &drm::Device<T::Driver, Ctx>, r_obj: &T::Object, range: Range<u64>, reserve_range: Range<u64>, @@ -250,21 +252,27 @@ impl<T: DriverGpuVm> GpuVm<T> { } /// The manager for a GPUVM. -pub trait DriverGpuVm: Sized + Send { +pub trait DriverGpuVm: Sized + Send + Sync { /// Parent `Driver` for this object. - type Driver: drm::Driver<Object = Self::Object>; + type Driver: drm::Driver; /// The kind of GEM object stored in this GPUVM. - type Object: IntoGEMObject; + type Object: drm::driver::AllocImpl<Driver = Self::Driver> + Send + Sync; /// Data stored with each [`struct drm_gpuva`](struct@GpuVa). - type VaData; + /// + /// Only `Send` is required: the data has a single owner at all times, moving + /// between threads by value (handed back as a [`GpuVaRemoved`]) but never + /// accessed by two threads concurrently. + type VaData: Send; /// Data stored with each [`struct drm_gpuvm_bo`](struct@GpuVmBo). - type VmBoData; + type VmBoData: Send + Sync; /// The private data passed to callbacks. - type SmContext<'ctx>; + type SmContext<'ctx> + where + Self: 'ctx; /// Indicates that a new mapping should be created. fn sm_step_map<'op, 'ctx>( @@ -296,12 +304,10 @@ pub trait DriverGpuVm: Sized + Send { /// # Invariants /// /// Each `GpuVm` instance has at most one `UniqueRefGpuVm` reference. +// `Send`/`Sync` derive from `ARef<GpuVm<T>>`; the trait bounds make them correct for the unique +// handle's `&mut T` access. pub struct UniqueRefGpuVm<T: DriverGpuVm>(ARef<GpuVm<T>>); -// SAFETY: The GPUVM api is designed to allow &self methods to be called in parallel, and -// concurrent access to `data` is safe due to the `T: Sync` requirement. -unsafe impl<T: DriverGpuVm + Sync> Sync for UniqueRefGpuVm<T> {} - impl<T: DriverGpuVm> UniqueRefGpuVm<T> { /// Access the data owned by this `UniqueRefGpuVm` immutably. #[inline] diff --git a/rust/kernel/drm/gpuvm/sm_ops.rs b/rust/kernel/drm/gpuvm/sm_ops.rs index 69a8e5ab2821..742c151b2540 100644 --- a/rust/kernel/drm/gpuvm/sm_ops.rs +++ b/rust/kernel/drm/gpuvm/sm_ops.rs @@ -3,7 +3,7 @@ use super::*; /// The actual data that gets threaded through the callbacks. -struct SmData<'a, 'ctx, T: DriverGpuVm> { +struct SmData<'a, 'ctx, T: DriverGpuVm + 'ctx> { gpuvm: &'a mut UniqueRefGpuVm<T>, user_context: &'a mut T::SmContext<'ctx>, } @@ -20,7 +20,7 @@ struct SmMapData<'a, 'ctx, T: DriverGpuVm> { } /// The argument for [`UniqueRefGpuVm::sm_map`]. -pub struct OpMapRequest<'a, 'ctx, T: DriverGpuVm> { +pub struct OpMapRequest<'a, 'ctx, T: DriverGpuVm + 'ctx> { /// Address in GPU virtual address space. pub addr: u64, /// Length of mapping to create. diff --git a/rust/kernel/drm/gpuvm/va.rs b/rust/kernel/drm/gpuvm/va.rs index bf927b8e6fbb..46493f0ec5a6 100644 --- a/rust/kernel/drm/gpuvm/va.rs +++ b/rust/kernel/drm/gpuvm/va.rs @@ -104,6 +104,14 @@ impl<T: DriverGpuVm> GpuVa<T> { /// The memory is zeroed. pub struct GpuVaAlloc<T: DriverGpuVm>(KBox<MaybeUninit<GpuVa<T>>>); +// SAFETY: A `GpuVaAlloc` is an owned, uninitialised allocation with no live `T::VaData` and no +// thread-bound state. +unsafe impl<T: DriverGpuVm> Send for GpuVaAlloc<T> {} + +// SAFETY: A `GpuVaAlloc` has no `&self` method that reaches its contents, so a shared +// `&GpuVaAlloc` cannot access the allocation. +unsafe impl<T: DriverGpuVm> Sync for GpuVaAlloc<T> {} + impl<T: DriverGpuVm> GpuVaAlloc<T> { /// Pre-allocate a [`GpuVa`] object. pub fn new(flags: AllocFlags) -> Result<GpuVaAlloc<T>, AllocError> { diff --git a/rust/kernel/drm/gpuvm/vm_bo.rs b/rust/kernel/drm/gpuvm/vm_bo.rs index ab12b710267e..5989972da829 100644 --- a/rust/kernel/drm/gpuvm/vm_bo.rs +++ b/rust/kernel/drm/gpuvm/vm_bo.rs @@ -19,6 +19,15 @@ pub struct GpuVmBo<T: DriverGpuVm> { data: T::VmBoData, } +// SAFETY: It is safe to send a `GpuVmBo<T>` to another thread: dropping it there drops +// `T::VmBoData` and the GEM `T::Object`, both `Send` by the `DriverGpuVm` bounds. +unsafe impl<T: DriverGpuVm> Send for GpuVmBo<T> {} + +// SAFETY: It is safe to share a `&GpuVmBo<T>` between threads: it effectively shares +// `&T::VmBoData` and the GEM `&T::Object` (both `Sync`), and any thread may upgrade to an +// `ARef` and ultimately drop them (both `Send`), per the `DriverGpuVm` bounds. +unsafe impl<T: DriverGpuVm> Sync for GpuVmBo<T> {} + // SAFETY: By type invariants, the allocation is managed by the refcount in `self.inner`. unsafe impl<T: DriverGpuVm> AlwaysRefCounted for GpuVmBo<T> { fn inc_ref(&self) { diff --git a/rust/kernel/drm/ioctl.rs b/rust/kernel/drm/ioctl.rs index cf328101dde4..64af9eacc306 100644 --- a/rust/kernel/drm/ioctl.rs +++ b/rust/kernel/drm/ioctl.rs @@ -70,6 +70,18 @@ pub mod internal { pub use bindings::drm_device; pub use bindings::drm_file; pub use bindings::drm_ioctl_desc; + + /// Cast an [`Ioctl`] DRM device pointer to [`Registered`], preserving the driver type + /// parameter `T`. + /// + /// Used by [`declare_drm_ioctls!`] to anchor type inference. + #[doc(hidden)] + #[inline] + pub const fn __dev_ctx_cast<T: crate::drm::Driver>( + ptr: *const crate::drm::Device<T, crate::drm::Ioctl>, + ) -> *const crate::drm::Device<T, crate::drm::Registered> { + ptr.cast() + } } /// Declare the DRM ioctls for a driver. @@ -82,7 +94,8 @@ pub mod internal { /// `user_callback` should have the following prototype: /// /// ```ignore -/// fn foo(device: &kernel::drm::Device<Self>, +/// fn foo(device: &kernel::drm::Device<Self, kernel::drm::Registered>, +/// reg_data: &Self::RegistrationData<'_>, /// data: &mut uapi::argument_type, /// file: &kernel::drm::File<Self::File>, /// ) -> Result<u32> @@ -131,10 +144,45 @@ macro_rules! declare_drm_ioctls { // - The DRM device must have been registered when we're called through // an IOCTL. // + // INVARIANT: The `Ioctl` context requires that the device has been + // registered via `drm_dev_register()` at some point; the DRM core + // guarantees this for ioctl dispatch callbacks. + // // FIXME: Currently there is nothing enforcing that the types of the // dev/file match the current driver these ioctls are being declared // for, and it's not clear how to enforce this within the type system. - let dev = $crate::drm::device::Device::from_raw(raw_dev); + let dev: &$crate::drm::device::Device<_, $crate::drm::Ioctl> = + $crate::drm::device::Device::from_raw(raw_dev); + + // Type-inference anchor: the closure is never called but ties `dev`'s + // type to `$func`'s first parameter, which the compiler cannot infer + // through method resolution and associated-type projections alone. + #[allow(unreachable_code)] + let _ = || { + let __ptr = $crate::drm::ioctl::internal::__dev_ctx_cast( + ::core::ptr::from_ref(dev), + ); + + $func( + // SAFETY: This closure is never executed; the dereference + // exists purely to unify the type parameter with `$func`. + // The pointer is valid regardless. + unsafe { &*__ptr }, + unreachable!(), + unreachable!(), + unreachable!(), + ) + }; + + // Enforce that the handler accepts higher-ranked + // lifetimes, preventing it from requiring 'static + // references that could escape this scope. + let _: for<'a> fn(&'a _, &'a _, &'a mut _, &'a _) -> _ = $func; + + let Some(guard) = dev.registration_guard() else { + return $crate::error::code::ENODEV.to_errno(); + }; + // SAFETY: The ioctl argument has size `_IOC_SIZE(cmd)`, which we // asserted above matches the size of this type, and all bit patterns of // UAPI structs must be valid. @@ -147,7 +195,9 @@ macro_rules! declare_drm_ioctls { // SAFETY: This is just the DRM file structure let file = unsafe { $crate::drm::File::from_raw(raw_file) }; - match $func(dev, data, file) { + match guard.registration_data_with(|reg_data| { + $func(&*guard, reg_data, data, file) + }) { Err(e) => e.to_errno(), Ok(i) => i.try_into() .unwrap_or($crate::error::code::ERANGE.to_errno()), diff --git a/rust/kernel/drm/mod.rs b/rust/kernel/drm/mod.rs index a66e7166f66b..fd6ed35bc35a 100644 --- a/rust/kernel/drm/mod.rs +++ b/rust/kernel/drm/mod.rs @@ -11,8 +11,10 @@ pub mod ioctl; pub use self::device::Device; pub use self::device::DeviceContext; +pub use self::device::Ioctl; +pub use self::device::Normal; pub use self::device::Registered; -pub use self::device::Uninit; +pub use self::device::RegistrationGuard; pub use self::device::UnregisteredDevice; pub use self::driver::Driver; pub use self::driver::DriverInfo; diff --git a/rust/kernel/faux.rs b/rust/kernel/faux.rs index 43b4974f48cd..cd4198fbb232 100644 --- a/rust/kernel/faux.rs +++ b/rust/kernel/faux.rs @@ -9,15 +9,63 @@ use crate::{ bindings, device, - prelude::*, // + prelude::*, + types::Opaque, // }; -use core::ptr::{ - addr_of_mut, - null, - null_mut, - NonNull, // +use core::{ + marker::PhantomData, + ptr::{ + null, + null_mut, + NonNull, // + }, }; +/// A faux device. +/// +/// A faux device is a virtual device backed by the faux bus, primarily used for scenarios where a +/// real hardware device is not available or for testing. +/// +/// # Invariants +/// +/// The underlying `struct faux_device` is valid. +#[repr(transparent)] +pub struct Device<Ctx: device::DeviceContext = device::Normal>( + Opaque<bindings::faux_device>, + PhantomData<Ctx>, +); + +impl<Ctx: device::DeviceContext> Device<Ctx> { + #[inline] + fn as_raw(&self) -> *mut bindings::faux_device { + self.0.get() + } + + /// # Safety + /// + /// `ptr` must be a valid pointer to a `struct faux_device`. + #[inline] + unsafe fn from_raw<'a>(ptr: *mut bindings::faux_device) -> &'a Self { + // SAFETY: `Device` is a transparent wrapper of `Opaque<bindings::faux_device>`. + unsafe { &*ptr.cast() } + } +} + +impl<Ctx: device::DeviceContext> AsRef<device::Device<Ctx>> for Device<Ctx> { + #[inline] + fn as_ref(&self) -> &device::Device<Ctx> { + // SAFETY: By the type invariant of `Self`, `self.as_raw()` is a pointer to a valid + // `struct faux_device`. `dev` points to a valid `struct device`. + unsafe { device::Device::from_raw(&raw mut (*self.as_raw()).dev) } + } +} + +// SAFETY: `faux::Device` is a transparent wrapper of `struct faux_device`. +// The offset is guaranteed to point to a valid device field inside `faux::Device`. +unsafe impl<Ctx: device::DeviceContext> device::AsBusDevice<Ctx> for Device<Ctx> { + const OFFSET: usize = core::mem::offset_of!(bindings::faux_device, dev); +} + /// The registration of a faux device. /// /// This type represents the registration of a [`struct faux_device`]. When an instance of this type @@ -25,7 +73,8 @@ use core::ptr::{ /// /// # Invariants /// -/// `self.0` always holds a valid pointer to an initialized and registered [`struct faux_device`]. +/// - `self.0` always holds a valid pointer to an initialized and registered [`struct faux_device`]. +/// - This object is proof that the object described by this `Registration` is bound to a device. /// /// [`struct faux_device`]: srctree/include/linux/device/faux.h pub struct Registration(NonNull<bindings::faux_device>); @@ -59,11 +108,19 @@ impl Registration { } } -impl AsRef<device::Device> for Registration { - fn as_ref(&self) -> &device::Device { - // SAFETY: The underlying `device` in `faux_device` is guaranteed by the C API to be - // a valid initialized `device`. - unsafe { device::Device::from_raw(addr_of_mut!((*self.as_raw()).dev)) } +impl AsRef<Device<device::Bound>> for Registration { + #[inline] + fn as_ref(&self) -> &Device<device::Bound> { + // SAFETY: + // - The underlying `struct faux_device` is guaranteed by the C API to be a valid + // initialized `device`. + // - `faux_match()` always returns 1, and probe runs synchronously + // (PROBE_FORCE_SYNCHRONOUS). + // - `suppress_bind_attrs = true` on faux_driver prevents userspace-triggered unbind via + // sysfs. + // - `mem::forget(Registration)` is not a problem; if the `Registration` is leaked, the faux + // device stays bound forever. + unsafe { Device::from_raw(self.as_raw()) } } } diff --git a/rust/kernel/firmware.rs b/rust/kernel/firmware.rs index 1e3dd2e3422d..a18f8b84f3e3 100644 --- a/rust/kernel/firmware.rs +++ b/rust/kernel/firmware.rs @@ -7,9 +7,9 @@ use crate::{ bindings, device::Device, - error::Error, - error::Result, + error::to_result, ffi, + prelude::*, str::{CStr, CStrExt as _}, }; use core::ptr::NonNull; @@ -116,6 +116,48 @@ impl Drop for Firmware { } } +/// Load firmware directly into the caller-provided `buf`. +/// +/// On success the firmware image has been copied into `buf`; the caller accesses the data +/// through `buf` itself. +/// +/// This is intentionally a stand-alone function rather than a `Firmware` constructor. For +/// the `into_buf` path, the firmware data lives in the caller's `buf`, not in a +/// kernel-owned buffer, so returning a `Firmware` would expose `Firmware::data()` as a +/// second handle aliasing `buf` (and `release_firmware()` does not free `buf` anyway). +pub fn request_into_buf(name: &CStr, dev: &Device, buf: &mut [u8]) -> Result { + // `as_mut_ptr()` on an empty slice returns a non-NULL pointer to + // memory which the loader does not own. Passing that pointer with `size == 0` + // makes the loader believe that it is buffer it allocated itself, so when + // `release_firmware()` is called, it will vfree the pointer and trigger a + // bug. Reject empty slices to avoid this situation. + if buf.is_empty() { + return Err(EINVAL); + } + + let mut fw: *const bindings::firmware = core::ptr::null(); + + // SAFETY: `&raw mut fw` is a valid pointer to a NULL initialized `bindings::firmware` pointer. + // `name` and `dev` are valid as by their type invariants. `buf` is a valid writable + // buffer of `buf.len()` bytes. + to_result(unsafe { + bindings::request_firmware_into_buf( + &raw mut fw, + name.as_char_ptr(), + dev.as_raw(), + buf.as_mut_ptr().cast(), + buf.len(), + ) + })?; + + // The firmware bytes are now in `buf`, which the caller owns, so we don't need + // the kernel to hang on to it any more. + // SAFETY: `fw` is a valid pointer returned by `request_firmware_into_buf`. + unsafe { bindings::release_firmware(fw) }; + + Ok(()) +} + // SAFETY: `Firmware` only holds a pointer to a C `struct firmware`, which is safe to be used from // any thread. unsafe impl Send for Firmware {} diff --git a/rust/kernel/iommu/pgtable.rs b/rust/kernel/iommu/pgtable.rs index eddda8d9d62a..5f9b42ca92c2 100644 --- a/rust/kernel/iommu/pgtable.rs +++ b/rust/kernel/iommu/pgtable.rs @@ -16,7 +16,6 @@ use crate::{ Bound, Device, // }, - devres::Devres, error::to_result, io::PhysAddr, prelude::*, // @@ -59,15 +58,16 @@ pub struct Config { /// # Invariants /// /// The pointer references a valid io page table. -pub struct IoPageTable<F: IoPageTableFmt> { +pub struct IoPageTable<'a, F: IoPageTableFmt> { ptr: NonNull<bindings::io_pgtable_ops>, + _dev: PhantomData<&'a Device<Bound>>, _marker: PhantomData<F>, } // SAFETY: `struct io_pgtable_ops` is not restricted to a single thread. -unsafe impl<F: IoPageTableFmt> Send for IoPageTable<F> {} +unsafe impl<F: IoPageTableFmt> Send for IoPageTable<'_, F> {} // SAFETY: `struct io_pgtable_ops` may be accessed concurrently. -unsafe impl<F: IoPageTableFmt> Sync for IoPageTable<F> {} +unsafe impl<F: IoPageTableFmt> Sync for IoPageTable<'_, F> {} /// The format used by this page table. pub trait IoPageTableFmt: 'static { @@ -75,25 +75,10 @@ pub trait IoPageTableFmt: 'static { const FORMAT: io_pgtable_fmt; } -impl<F: IoPageTableFmt> IoPageTable<F> { - /// Create a new `IoPageTable` as a device resource. - #[inline] - pub fn new( - dev: &Device<Bound>, - config: Config, - ) -> impl PinInit<Devres<IoPageTable<F>>, Error> + '_ { - // SAFETY: Devres ensures that the value is dropped during device unbind. - Devres::new(dev, unsafe { Self::new_raw(dev, config) }) - } - +impl<'a, F: IoPageTableFmt> IoPageTable<'a, F> { /// Create a new `IoPageTable`. - /// - /// # Safety - /// - /// If successful, then the returned `IoPageTable` must be dropped before the device is - /// unbound. #[inline] - pub unsafe fn new_raw(dev: &Device<Bound>, config: Config) -> Result<IoPageTable<F>> { + pub fn new(dev: &'a Device<Bound>, config: Config) -> Result<IoPageTable<'a, F>> { let mut raw_cfg = bindings::io_pgtable_cfg { quirks: config.quirks, pgsize_bitmap: config.pgsize_bitmap, @@ -117,6 +102,7 @@ impl<F: IoPageTableFmt> IoPageTable<F> { // INVARIANT: We successfully created a valid page table. Ok(IoPageTable { ptr: NonNull::new(ops).ok_or(ENOMEM)?, + _dev: PhantomData, _marker: PhantomData, }) } @@ -239,7 +225,7 @@ extern "C" fn rust_tlb_flush_walk_noop( ) { } -impl<F: IoPageTableFmt> Drop for IoPageTable<F> { +impl<F: IoPageTableFmt> Drop for IoPageTable<'_, F> { fn drop(&mut self) { // SAFETY: The caller of `Self::ttbr()` promised that the page table is not live when this // destructor runs. @@ -254,7 +240,7 @@ impl IoPageTableFmt for ARM64LPAES1 { const FORMAT: io_pgtable_fmt = bindings::io_pgtable_fmt_ARM_64_LPAE_S1 as io_pgtable_fmt; } -impl IoPageTable<ARM64LPAES1> { +impl IoPageTable<'_, ARM64LPAES1> { /// Access the `ttbr` field of the configuration. /// /// This is the physical address of the page table, which may be passed to the device that diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs index aee6485dc58e..3ec897709e89 100644 --- a/rust/kernel/pci.rs +++ b/rust/kernel/pci.rs @@ -25,6 +25,7 @@ use crate::{ use core::{ marker::PhantomData, mem::offset_of, + num::NonZero, ptr::{ addr_of_mut, NonNull, // @@ -447,6 +448,18 @@ impl Device { } impl<'a> Device<device::Core<'a>> { + /// Returns the total number of VFs, or [`None`] if SR-IOV is not available. + #[inline] + pub fn sriov_get_totalvfs(&self) -> Option<NonZero<u16>> { + // SAFETY: `self.as_raw()` is a valid pointer to a `struct pci_dev`. + let total_vfs = unsafe { bindings::pci_sriov_get_totalvfs(self.as_raw()) }; + + // CAST: The C function returns `unsigned int`, but the value originates + // from TotalVFs/driver_max_VFs (which are defined as `u16`), so this cast + // cannot truncate. + NonZero::new(total_vfs as u16) + } + /// Enable memory resources for this device. pub fn enable_device_mem(&self) -> Result { // SAFETY: `self.as_raw` is guaranteed to be a pointer to a valid `struct pci_dev`. |
