linux-stable.git/include/linux/userfaultfd_k.h, branch v7.3-rc2

mm: fold userfaultfd_rwp() to false without CONFIG_ARCH_HAS_PTE_PROTNONE

2026-08-07T01:57:25+00:00

RWP tracks accesses by installing PAGE_NONE (protnone) PTEs, so its code
paths are gated on userfaultfd_rwp().  Without
CONFIG_ARCH_HAS_PTE_PROTNONE there is no PAGE_NONE -- 
defines it to a BUILD_BUG() stub, relying on callers folding such paths to
dead code via IS_ENABLED(CONFIG_ARCH_HAS_PTE_PROTNONE).

userfaultfd_rwp() was not a compile-time constant, so the compiler could
not fold those paths.  With an older compiler (gcc 8.5.0, sparc64) the
PAGE_NONE reference in move_pages_huge_pmd() survived to codegen:

  mm/huge_memory.c:2874: _dst_pmd = pmd_modify(_dst_pmd, PAGE_NONE);
  compiler_types.h:702: error: call to '__compiletime_assert_501'
    declared with attribute error: BUILD_BUG failed

RWP cannot exist without protnone, so return a compile-time false when
CONFIG_ARCH_HAS_PTE_PROTNONE is unset; every RWP path then folds away.

Link: https://lore.kernel.org/amcitKvUvFYr8W38@thinkstation
Signed-off-by: Kiryl Shutsemau (Meta) 
Reported-by: kernel test robot 
Closes: https://lore.kernel.org/oe-kbuild-all/202607250853.VaJWGLeA-lkp@intel.com/
Cc: Andrea Arcangeli 
Cc: David Hildenbrand 
Cc: James Houghton 
Cc: Jonathan Corbet 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes 
Cc: Mike Rapoport (Microsoft) 
Cc: Paolo Bonzini 
Cc: Peter Xu 
Cc: Sean Christopherson 
Cc: SeongJae Park 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

userfaultfd: add UFFD_FEATURE_RWP_ASYNC for async fault resolution

2026-08-05T02:18:39+00:00

Sync RWP delivers a message and blocks the faulting thread until the
handler resolves the fault.  For working-set tracking the VMM does not
need the message: it just needs to know, at scan time, which pages were
touched.  Async RWP serves that use case — the kernel restores access
in-place and the faulting thread continues without blocking.

The VMM reconstructs the access pattern after the fact via PAGEMAP_SCAN:
pages whose uffd bit is still set (inverted PAGE_IS_ACCESSED) were not
re-accessed since the last RWP cycle.

Worth calling out: async resolution upgrades writable private anon PTEs
via pte_mkwrite() when can_change_pte_writable() allows, mirroring
do_numa_page().  Without it, every re-access of an RWP'd writable page
would COW-fault a second time.

UFFD_FEATURE_RWP_ASYNC requires UFFD_FEATURE_RWP.

Link: https://lore.kernel.org/20260708111417.173443-13-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau 
Assisted-by: Claude:claude-opus-4-6
Acked-by: Mike Rapoport (Microsoft) 
Cc: Andrea Arcangeli 
Cc: David Hildenbrand 
Cc: James Houghton 
Cc: Jonathan Corbet 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes 
Cc: Paolo Bonzini 
Cc: Peter Xu 
Cc: Sean Christopherson 
Cc: SeongJae Park 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm/userfaultfd: add RWP fault delivery and expose UFFDIO_REGISTER_MODE_RWP

2026-08-05T02:18:38+00:00

Wire the fault side of read-write protection tracking and turn the
userspace interface on.

An RWP-protected PTE is PAGE_NONE with the uffd bit set.  The PROT_NONE
triggers a fault on any access; the uffd bit distinguishes it from plain
mprotect(PROT_NONE) or NUMA hinting.

Fault dispatch, per level:

  PTE     handle_pte_fault()    -> do_uffd_rwp()
  PMD     __handle_mm_fault()   -> do_huge_pmd_uffd_rwp()
  hugetlb hugetlb_fault()       -> hugetlb_handle_userfault()

The RWP branches gate on userfaultfd_pte_rwp() /
userfaultfd_huge_pmd_rwp() (VM_UFFD_RWP plus the uffd bit) and fall
through to do_numa_page() / do_huge_pmd_numa_page() otherwise.  Each
delivers a UFFD_PAGEFAULT_FLAG_RWP message through handle_userfault(); the
handler resolves it with UFFDIO_RWPROTECT clearing MODE_RWP.

userfaultfd_must_wait() and userfaultfd_huge_must_wait() add matching
protnone+uffd waiters so sync-mode fault handlers block correctly.

Expose the UAPI:

  UFFDIO_REGISTER_MODE_RWP   -> UFFD_API_REGISTER_MODES
  UFFD_FEATURE_RWP           -> UFFD_API_FEATURES
  _UFFDIO_RWPROTECT          -> UFFD_API_RANGE_IOCTLS
                                UFFD_API_RANGE_IOCTLS_BASIC

UFFD_FEATURE_RWP is masked out at UFFDIO_API time when PROT_NONE is not
available or VM_UFFD_RWP aliases VM_NONE (32-bit), so userspace never sees
an advertised-but-broken feature.

Works on anonymous, shmem, and hugetlb memory.

Link: https://lore.kernel.org/20260708111417.173443-11-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau 
Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Mike Rapoport (Microsoft) 
Cc: Andrea Arcangeli 
Cc: David Hildenbrand 
Cc: James Houghton 
Cc: Jonathan Corbet 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes 
Cc: Paolo Bonzini 
Cc: Peter Xu 
Cc: Sean Christopherson 
Cc: SeongJae Park 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

userfaultfd: add UFFDIO_REGISTER_MODE_RWP and UFFDIO_RWPROTECT plumbing

2026-08-05T02:18:38+00:00

Add the userspace interface for read-write protection tracking:

  - UFFDIO_REGISTER_MODE_RWP      register a range for RWP tracking
  - UFFD_FEATURE_RWP              capability bit
  - UFFDIO_RWPROTECT              install / remove RWP on a range

Introduce CONFIG_USERFAULTFD_RWP, auto-selected on 64-bit kernels with
ARCH_HAS_PTE_PROTNONE and HAVE_ARCH_USERFAULTFD_WP.  The symbol gates
VM_UFFD_RWP (previously aliased to VM_NONE) and the smaps/trace-flag hooks
added in the preparatory patches; without it the UAPI bits added here have
nothing to drive and would be unreachable.

Registration sets VM_UFFD_RWP on the VMA.  Combining MODE_WP with MODE_RWP
is rejected because both modes claim the uffd PTE bit.

UFFDIO_RWPROTECT is the bidirectional counterpart of
UFFDIO_WRITEPROTECT:

  - MODE_RWP              change_protection() with MM_CP_UFFD_RWP
                          installs PAGE_NONE and sets the uffd bit on
                          present PTEs
  - !MODE_RWP             change_protection() with MM_CP_UFFD_RWP_RESOLVE
                          restores vma->vm_page_prot and clears the bit

userfaultfd_clear_vma() runs the same resolve pass on unregister so RWP
state cannot outlive the uffd.

Re-registering a range must not drop a mode that installs per-PTE markers
(WP or RWP); doing so returns -EBUSY.  This also closes a pre-existing
window where re-registering without MODE_WP would strand uffd-wp markers:
before, those caused extra write-faults but were otherwise benign; with
RWP preservation in place, a subsequent mprotect() on a VM_UFFD_RWP VMA
would silently promote the stale markers to RWP.

The feature is not yet advertised.  UFFDIO_REGISTER_MODE_RWP,
UFFD_FEATURE_RWP, and _UFFDIO_RWPROTECT are intentionally absent from
UFFD_API_REGISTER_MODES, UFFD_API_FEATURES, and UFFD_API_RANGE_IOCTLS, so
UFFDIO_API masks them out and the register-mode validator rejects the bit.
The follow-up patch adds fault dispatch and exposes the UAPI.

Link: https://lore.kernel.org/20260708111417.173443-10-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau 
Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Mike Rapoport (Microsoft) 
Cc: Andrea Arcangeli 
Cc: David Hildenbrand 
Cc: James Houghton 
Cc: Jonathan Corbet 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes 
Cc: Paolo Bonzini 
Cc: Peter Xu 
Cc: Sean Christopherson 
Cc: SeongJae Park 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm: add MM_CP_UFFD_RWP change_protection() flag

2026-08-05T02:18:37+00:00

Preparatory patch.  Add the change_protection() primitive that userfaultfd
RWP will use.

An RWP-protected PTE is PAGE_NONE with the uffd PTE bit set.  The
PROT_NONE half makes the CPU fault on any access; the uffd bit
distinguishes an RWP fault from a plain mprotect(PROT_NONE) or NUMA
hinting fault.  MM_CP_UFFD_WP and MM_CP_UFFD_RWP share the same PTE bit,
so the two cannot be used together on the same range.

Two new change_protection() flags:

  MM_CP_UFFD_RWP            install PAGE_NONE and set the uffd bit
  MM_CP_UFFD_RWP_RESOLVE    restore vma->vm_page_prot, clear the uffd bit

Both are wired through change_pte_range(), change_huge_pmd(), and
hugetlb_change_protection() so anon, shmem, THP, and hugetlb all share the
same semantics.

Link: https://lore.kernel.org/20260708111417.173443-7-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau 
Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Mike Rapoport (Microsoft) 
Reviewed-by: SeongJae Park 
Cc: Andrea Arcangeli 
Cc: David Hildenbrand 
Cc: James Houghton 
Cc: Jonathan Corbet 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes 
Cc: Paolo Bonzini 
Cc: Peter Xu 
Cc: Sean Christopherson 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm: add VM_UFFD_RWP VMA flag

2026-08-05T02:18:37+00:00

Preparatory patch for userfaultfd read-write protection (RWP).  RWP
extends userfaultfd protection from plain write-protection (WP) to full
read-write protection: accesses to an RWP-protected range -- reads as well
as writes -- trap through userfaultfd.

Reserve VM_UFFD_RWP, add the userfaultfd_rwp() and userfaultfd_protected()
helpers, and wire up the smaps "ur" entry and the trace-flag table the
rest of the series will use.  The flag is gated on CONFIG_USERFAULTFD_RWP,
which is introduced together with the UAPI in a later patch; until then
VM_UFFD_RWP aliases VM_NONE and every downstream check folds to dead code.

Nothing sets or queries the flag yet.

Link: https://lore.kernel.org/20260708111417.173443-6-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau 
Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Mike Rapoport (Microsoft) 
Reviewed-by: SeongJae Park 
Reviewed-by: Lorenzo Stoakes 
Cc: Andrea Arcangeli 
Cc: David Hildenbrand 
Cc: James Houghton 
Cc: Jonathan Corbet 
Cc: Liam Howlett 
Cc: Paolo Bonzini 
Cc: Peter Xu 
Cc: Sean Christopherson 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

userfaultfd: test uffd VMA flags through the vma_flags_t API

2026-08-05T02:18:36+00:00

The uffd VMA-flag helpers read vma->vm_flags directly.  Now that
config-gated per-mode masks exist, switch them to the vma_flags_t accessor
vma_test_any_mask(), which is the going-forward API and keeps a single
place (the VMA_UFFD_* masks) that knows which modes are available on the
current build.

No functional change: vma_flags_t is in union with vm_flags, so the same
bits are read, and the masks fold to the same code the open-coded vm_flags
tests produced -- verified identical on gcc and clang, 32- and 64-bit.

Link: https://lore.kernel.org/20260708111417.173443-5-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau 
Suggested-by: Lorenzo Stoakes 
Assisted-by: Claude:claude-opus-4-8
Acked-by: Mike Rapoport (Microsoft) 
Reviewed-by: Lorenzo Stoakes 
Cc: Andrea Arcangeli 
Cc: David Hildenbrand 
Cc: James Houghton 
Cc: Jonathan Corbet 
Cc: Liam Howlett 
Cc: Paolo Bonzini 
Cc: Peter Xu 
Cc: Sean Christopherson 
Cc: SeongJae Park 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

mm: rename uffd-wp PTE accessors to uffd

2026-08-05T02:18:36+00:00

Userfaultfd RWP will reuse the uffd-wp PTE bit to mark access-tracking
PTEs, alongside the write-protected ones it already marks.  The bit's
meaning now depends on the VMA flag (WP or RWP), not on its name.

Rename the kernel-internal names that describe the bit:

  - pte/pmd/huge_pte accessors (and swap variants)
  - pgtable_supports_uffd() capability query
  - SCAN_PTE_UFFD khugepaged enum

The ftrace string emitted by mm_khugepaged_scan_pmd for this enum is
kept as "pte_uffd_wp" so existing trace-based tooling keeps matching.

Pure mechanical rename -- no behavior change.

Link: https://lore.kernel.org/20260708111417.173443-4-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau 
Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Mike Rapoport (Microsoft) 
Reviewed-by: SeongJae Park 
Cc: Andrea Arcangeli 
Cc: David Hildenbrand 
Cc: James Houghton 
Cc: Jonathan Corbet 
Cc: Liam Howlett 
Cc: Lorenzo Stoakes 
Cc: Paolo Bonzini 
Cc: Peter Xu 
Cc: Sean Christopherson 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Zi Yan 
Signed-off-by: Andrew Morton

userfaultfd: build __VMA_UFFD_FLAGS from config-gated masks

2026-06-09T01:21:29+00:00

The VMA flags bitmap is a single word today: NUM_VMA_FLAG_BITS is
BITS_PER_LONG, so on 32-bit vma_flags_t holds only 32 bits.  (The bitmap
type exists so this can grow past BITS_PER_LONG later; until it does,
anything declared above the first word is out of range on 32-bit.) The bit
enum nevertheless declares some bits unconditionally above BITS_PER_LONG
-- VMA_UFFD_MINOR_BIT is 41, with VM_UFFD_MINOR == VM_NONE on 32-bit so no
VMA actually carries the bit.

__VMA_UFFD_FLAGS feeds VMA_UFFD_MINOR_BIT to mk_vma_flags()
unconditionally.  On 32-bit that becomes __set_bit(41, &one_long), a write
one word past the end of the single-word bitmap.  The compiler folds the
out-of-bounds store with wraparound (1UL << (41 % 32) == bit 9) into the
first word; bit 9 is already in __VMA_UFFD_FLAGS so the mask happens to
come out right today, but it is an out-of-bounds write all the same, and
any high-numbered bit whose mod-BITS_PER_LONG position is otherwise unused
would silently OR an extra bit into the mask.

Rather than feed bit numbers that may not exist on the current build to
mk_vma_flags(), build the mask from whole per-mode masks that collapse to
EMPTY_VMA_FLAGS when their feature is unavailable.  Add
mk_vma_flags_from_masks() for that, and define VMA_UFFD_MISSING / _WP /
_MINOR alongside the VM_UFFD_* flags, gating VMA_UFFD_MINOR on the same
config as VM_UFFD_MINOR (which implies 64BIT, where bit 41 fits).  An
out-of-range bit is then never materialised, on any arch, and the in-range
fast path stays a compile-time constant.

Link: https://lore.kernel.org/20260529172331.356655-7-kas@kernel.org
Fixes: 9ea35a25d51b ("mm: introduce VMA flags bitmap type")
Signed-off-by: Kiryl Shutsemau 
Reported-by: Sashiko AI review 
Suggested-by: Lorenzo Stoakes 
Reviewed-by: Lorenzo Stoakes 
Assisted-by: Claude:claude-opus-4-8
Cc: David Hildenbrand 
Cc: Michal Hocko 
Cc: Mike Rapoport 
Cc: Peter Xu 
Cc: Suren Baghdasaryan 
Cc: Vlastimil Babka 
Cc: Balbir Singh 
Cc: 
Signed-off-by: Andrew Morton

userfaultfd: make functions that are not used outside uffd static

2026-06-04T21:45:03+00:00

After merging fs/userfaultfd.c into mm/userfaultfd.c, several functions
that were previously shared between the two files are now only used within
mm/userfaultfd.c.

Make them static and remove their declarations from
include/linux/userfaultfd_k.h.

Link: https://lore.kernel.org/20260523173759.3964908-3-rppt@kernel.org
Assisted-by: Copilot:claude-opus-4-6
Signed-off-by: Mike Rapoport (Microsoft) 
Cc: Al Viro 
Cc: Christian Brauner 
Cc: David Hildenbrand 
Cc: Jan Kara 
Cc: "Kirill A. Shutemov" 
Cc: Peter Xu 
Signed-off-by: Andrew Morton