Pull workqueue fixes from Tejun Heo:

 - An unbound worker pool could be freed while still reachable through
   the pending-activation list, leading to a use-after-free. Unlink
   before dropping the reference

 - On PREEMPT_RT, the BH workqueue kick raised softirqs from preemptible
   context, tripping a lockdep assertion and possibly losing
   concurrently raised softirq bits

 - Draining BH work off a dead CPU nests two pools' callback locks,
   which lockdep misreported as recursive locking. The nesting cannot
   deadlock. Annotate it

 - Reject watchdog thresholds that overflow the conversion to jiffies

 - Make the drgn workqueue dump script work again on kernels and vmcores
   from before the workqueue attrs field rename

* tag 'wq-for-7.3-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq:
  tools/workqueue/wq_dump.py: Support backward compatibility for wq->attrs rename
  workqueue: reject watchdog thresholds that overflow jiffies
  workqueue: Fix unbound pool lifetime for pending pwqs
  workqueue: Use raise_softirq() to trigger softirq in irq_work handler
  workqueue: Annotate cb_lock nesting when draining a dead BH pool

The watchdog threshold is supplied in seconds but is multiplied by HZ
before being used as a jiffies interval. Reject values that exceed
MAX_JIFFY_OFFSET / HZ so the multiplication cannot wrap and the
time_after() comparisons remain within their supported range.

The check is performed before changing the threshold or watchdog timer.
Zero remains the value used to disable the watchdog.

Fixes: 82607adcf9cdf ("workqueue: implement lockup detector")
Signed-off-by: Jiacheng Xu <stitch@zju.edu.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>

Pull NFS client updates from Trond Myklebust:
 "Highlights include:

  Stable fixes:
   - Use-after-free fixes for the sunrpc client code
   - Delegation hash table leak
   - NULL dereference on lockowner allocation failure
   - Fix a handshake completion race in the TLS code
   - Fix an error sign checking issue when deciding whether the pNFS
     layout is still in use, or can be returned
   - Fix a layout segment leak in pnfs_layout_process()

  Other bugfixes:
   - Fix a missing NULL check in the rpcbind client
   - annotate shared socket callbacks with READ_ONCE/WRITE_ONCE
   - nfs_inode_set_delegation() error paths should return the delegation
   - Use clear_and_wake_up_bit() in nfs_clear_invalid_mapping() and the
     pNFS code.
   - Fix the nfs4_alloc_client() error paths to free the IDR allocation
   - fix folio dereference before NULL check in
     nfs_inode_remove_request()
   - Fix delayed delegation return
   - Fix another state manager race with umount
   - Fix device leaks on parse failure
   - Avoid cancelling in-flight I/O during a layout recall if the server
     doesn't require it
   - flexfiles: report cancelled I/O as a layout error
   - flexfiles: fix NULL dereference for NFSv4.0 data servers
   - Fix incorrect argument passed to nfs4_delete_lease()
   - Fix several symlink issues resulting from nfs_atomic_open_v23()
   - Fix an uninitialised variable issue in the NFSv4.1 callback code
   - fix LAYOUTSTATS send buffer exhaustion

  Features and cleanups:
   - NFSv4.2: Allow the server to specify that file data may not be cached
   - localio: optimise I/O submission when when not doing memory reclaim
   - localio: Remove duplicate wait code in nfs_local_commit
   - flexfiles: support loosely coupled NFSv4.x data servers
   - pNFS: key the data server cache on the NFS version"

* tag 'nfs-for-7.3-1' of git://git.linux-nfs.org/projects/trondmy/linux-nfs: (33 commits)
  NFSv4.1: fix layout segment leak on the pnfs_layout_process() forget path
  NFSv4/pnfs: key the data server cache on the NFS version
  NFSv4.2: fix LAYOUTSTATS send buffer exhaustion
  pNFS: Fix EBUSY check in pnfs_layout_need_return
  NFSv4.1: zero referring call lists before decoding
  nfs: fix ENXIO on O_CREAT open of existing symlink over NFSv3
  SUNRPC: wait for in-flight client TLS handshake callback
  NFSv4: Fix incorrect argument passed to nfs4_delete_lease() in nfs4_add_lease()
  lockd: fix NULL dereference on lockowner allocation failure
  NFS: fix delegation_hash_table leak when nfs4_server_common_setup() fails
  NFSv4/flexfiles: support loosely coupled data servers
  NFSv4/flexfiles: fix NULL dereference for NFSv4.0 data servers
  NFSv4: pin the superblock for active state owners
  sunrpc: fix use-after-free in __rpc_clnt_handle_event and __rpc_clnt_remove_pipedir
  NFS/localio: issue commit inline when not in a memory-reclaim context
  NFS/localio: remove dead FLUSH_SYNC handling from nfs_local_commit
  NFS/localio: issue IO inline when not in a memory-reclaim context
  NFS: Fix delayed delegation return list handling
  NFS: Verify symlink inode before caching target
  NFS: fix folio dereference before NULL check in nfs_inode_remove_request()
  ...

KASAN reports a use-after-free of an unbound worker_pool in
node_activate_pending_pwq():

  BUG: KASAN: slab-use-after-free in _raw_spin_trylock+0x6d/0x120
  Read of size 4 at addr ffff8880089ce000 by task kworker/u22:0/318

  CPU: 1 UID: 0 PID: 318 Comm: kworker/u22:0 Not tainted 7.2.0 #1 PREEMPT(lazy)
  Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
  Workqueue:  0x0 (flush-8:0)
  Call Trace:
   <TASK>
   dump_stack_lvl+0x53/0x70
   print_report+0xce/0x610
   kasan_report+0xce/0x100
   _raw_spin_trylock+0x6d/0x120
   pwq_dec_nr_in_flight+0x4b4/0xcb0
   process_one_work+0x921/0x11a0
   worker_thread+0x4d0/0xd20
   kthread+0x2de/0x3c0
   ret_from_fork+0x3aa/0x620
   ret_from_fork_asm+0x1a/0x30
   </TASK>

  Allocated by task 311:
   alloc_pwq+0x439/0xca0
   apply_wqattrs_prepare+0x75e/0xd10
   apply_workqueue_attrs_locked+0x44/0xa0
   wq_nice_store+0x350/0x450

  Freed by task 0:
   kfree+0x127/0x3b0
   rcu_core+0x523/0x1780
   handle_softirqs+0x1b3/0x610

  Last potentially related work creation:
   put_unbound_pool+0x3f3/0x7d0
   pwq_release_workfn+0x494/0x8e0
   kthread_worker_fn+0x1ff/0x790

Canceling the last inactive work skips pwq_dec_nr_active(), so an empty
pwq can remain on pending_pwqs when its refcnt reaches zero.
pwq_release_workfn() currently puts the pool before removing that pwq.
If this drops the last pool reference, the pool can be RCU-freed while
the pwq remains reachable, and node_activate_pending_pwq() may trylock
the freed pool->lock.

Remove the pwq from pending_pwqs before putting the pool.

Fixes: 5797b1c18919 ("workqueue: Implement system-wide nr_active enforcement for unbound workqueues")
Cc: stable@vger.kernel.org
Signed-off-by: Yao Kai <yaokai34@huawei.com>
Signed-off-by: Tejun Heo <tj@kernel.org>

bh_pool_kick_normal() and bh_pool_kick_highpri() are registered via
init_irq_work() without the IRQ_WORK_HARD_IRQ flag. On PREEMPT_RT, such
irq_work items are processed by the per-CPU irq_workd kthread in preemptible
task context with IRQs enabled. However, raise_softirq_irqoff() requires
IRQs to be disabled. Calling it from irq_workd trips the lockdep assertion
in __raise_softirq_irqoff() and the non-atomic update of the softirq pending
mask can lose bits raised by an interrupt on the same CPU. Replace
raise_softirq_irqoff() with raise_softirq() in the irq_work handlers.

Fixes: 2f34d7337d98 ("workqueue: Fix queue_work_on() with BH workqueues")
Cc: stable@vger.kernel.org # v6.9+
Signed-off-by: Zqiang <qiang.zhang@linux.dev>
Signed-off-by: Tejun Heo <tj@kernel.org>

On PREEMPT_RT, bh_worker() wraps work item execution in pool->cb_lock to
provide a handshake for canceling BH work items. When a CPU goes down,
drain_dead_softirq_workfn() runs the dead pool's bh_worker() nested inside
the local pool's bh_worker(), acquiring the cb_locks of two different pools
without a nesting annotation. lockdep reports possible recursive locking:

  ============================================
  WARNING: possible recursive locking detected
  --------------------------------------------
  ktimers/0/16 is trying to acquire lock:
  ffff8880b873a990 (&pool->cb_lock){+...}-{3:3}, at: bh_worker+0x7d/0x880

  but task is already holding lock:
  ffff8880b863a990 (&pool->cb_lock){+...}-{3:3}, at: bh_worker+0x7d/0x880

  Call Trace:
   bh_worker+0x7d/0x880 kernel/workqueue.c:3688
   drain_dead_softirq_workfn+0x95/0x220 kernel/workqueue.c:3763
   process_scheduled_works+0xa8e/0x14e0 kernel/workqueue.c:3405
   bh_worker+0x46a/0x880 kernel/workqueue.c:3708
   tasklet_action+0xc/0x70 kernel/softirq.c:965

The nesting can't deadlock. A pool's bh_worker() runs nested only while the
pool's CPU is dead, entered from a live pool's bh_worker() on the draining
CPU, so the ordering is always live to dead. CPU hotplug operations are
serialized and the drain is synchronous, so the nesting depth never exceeds
two. Annotate the inner acquisition with SINGLE_DEPTH_NESTING.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: syzbot+1bd20115328f8254ed62@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=1bd20115328f8254ed62
Fixes: ad7c7f4b9c6c ("workqueue: Provide a handshake for canceling BH workers")
Cc: stable@vger.kernel.org # v6.18+

Every LOCALIO read and write is currently bounced through the dedicated
!WQ_MEM_RECLAIM nfslocaliod_workqueue.  That bounce is only actually
required when the submitting context is a memory-reclaim context: LOCALIO
issues IO directly into a stacked local filesystem (e.g. XFS) which may in
turn flush its own !WQ_MEM_RECLAIM workqueue.  Doing that from a
WQ_MEM_RECLAIM worker (most importantly writeback's wb_workfn on bdi_wq) or
an explicit PF_MEMALLOC reclaim task trips check_flush_dependency() and
risks a forward-progress deadlock, which is why commit b9f5dd57f4a5
("nfs/localio: use dedicated workqueues for filesystem read and write")
introduced the intermediate workqueue.

Outside of reclaim context -- ordinary application/task submission such as
O_DIRECT or fsync-driven writeback -- the workqueue hop buys nothing and
merely adds a context switch and scheduling latency per IO while discarding
the NFS client's inherent application-context parallelism.

Add current_is_workqueue_mem_reclaim(), which reports whether %current is a
WQ_MEM_RECLAIM worker using the same predicate check_flush_dependency()
warns on.  Use it, together with the PF_MEMALLOC check, in the new
nfs_local_defer_io() helper to decide per-IO whether nfs_local_do_read()
and nfs_local_do_write() must defer to nfslocaliod_workqueue or may issue
the IO inline.  Buffered writeback continues to bounce (wb_workfn is a
WQ_MEM_RECLAIM worker); O_DIRECT and app-context submission now run inline.

Running nfs_local_call_write() inline is safe: it already saves and
restores current->flags around the PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO it
sets and scopes the file opener's creds.  The async O_DIRECT completion
path is likewise unaffected: when the underlying filesystem returns
-EIOCBQUEUED, the kiocb ki_complete callback (nfs_local_read_aio_complete /
nfs_local_write_aio_complete) can run in bottom-half context and so must
still defer the pgio completion (nfs_local_pgio_release -> rpc_call_done) to
nfsiod_workqueue via nfs_local_pgio_aio_complete().  That completion hop is
independent of how the IO was submitted, and this change leaves it as-is;
only the submission side stops unconditionally hopping through
nfslocaliod_workqueue.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Mike Snitzer <snitzer@kernel.org>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>

kick_pool_pick() reads and writes p->wake_cpu while the scheduler can
update it concurrently. KCSAN reports:

  BUG: KCSAN: data-race in kick_pool_pick+0xf8/0x2d8

  race at unknown origin, with read to 0xffff000663229da4 of 4 bytes by
  task 1817002 on cpu 40:
   kick_pool_pick+0xf8/0x2d8
   process_scheduled_works+0x2bc/0x888
   worker_thread+0x394/0x548
   kthread+0x1b8/0x1f0
   ret_from_fork+0x10/0x20

  value changed: 0x0000002b -> 0x0000002f

The race is harmless. wake_cpu is a best-effort placement hint: every
writer stores a valid CPU id and the wakeup path validates it through
select_task_rq(), so a stale value only affects which CPU the worker
wakes up on.

Mark both accesses with READ_ONCE() and WRITE_ONCE() to document that
they are intentionally racy and to stop the compiler from reloading or
tearing them.

Signed-off-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Signed-off-by: Tejun Heo <tj@kernel.org>

wq_node_nr_active() warns and returns NULL when @wq is not unbound, but
every caller dereferences the result right away, so the WARN_ON_ONCE()
only moves the oops one frame up, as raised by Tejun.

Fix it by BUGing_ON() instead of this silly WARN_ON_ONCE();

Fixes: b72fdc651056 ("workqueue: account nr_active by the backing pool")
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Tejun Heo <tj@kernel.org>

wq->cpu_pwq holds RCU-protected pwq pointers, but the percpu allocation
path fills it in with plain loads and stores, which sparse flags:

  kernel/workqueue.c:5682:57: sparse: incorrect type in initializer (different address spaces) @@     expected struct pool_workqueue **pwq_p @@     got struct pool_workqueue [noderef] __rcu ** @@

Allocate the array as __rcu pointers and publish each pwq with
rcu_assign_pointer() once it is initialized and linked, the order
install_unbound_pwq() uses.

The warnings are not new: commit 79f23600bc7b ("workqueue: factor out
get_percpu_pool()") only turned the flagged assignment into an
initializer.

Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202608120931.tvTzq1gD-lkp@intel.com/
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Tejun Heo <tj@kernel.org>