linux-stable.git/include/linux/blk-cgroup.h, branch v5.3.3

blkcg: allow blkcg_policy->pd_stat() to print non-debug info too

2019-07-16T16:06:39+00:00

Currently, ->pd_stat() is called only when moduleparam
blkcg_debug_stats is set which prevents it from printing non-debug
policy-specific statistics.  Let's move debug testing down so that
->pd_stat() can print non-debug stat too.  This patch doesn't cause
any visible behavior change.

Signed-off-by: Tejun Heo 
Cc: Josef Bacik 
Signed-off-by: Jens Axboe

blkcg: implement REQ_CGROUP_PUNT

2019-07-10T15:00:57+00:00

When a shared kthread needs to issue a bio for a cgroup, doing so
synchronously can lead to priority inversions as the kthread can be
trapped waiting for that cgroup.  This patch implements
REQ_CGROUP_PUNT flag which makes submit_bio() punt the actual issuing
to a dedicated per-blkcg work item to avoid such priority inversions.

This will be used to fix priority inversions in btrfs compression and
should be generally useful as we grow filesystem support for
comprehensive IO control.

Cc: Chris Mason 
Reviewed-by: Josef Bacik 
Reviewed-by: Jan Kara 
Signed-off-by: Tejun Heo 
Signed-off-by: Jens Axboe

blk-cgroup: move struct blkg_stat to bfq

2019-06-20T16:32:34+00:00

This structure and assorted infrastructure is only used by the bfq I/O
scheduler.  Move it there instead of bloating the common code.

Acked-by: Tejun Heo 
Acked-by: Paolo Valente 
Signed-off-by: Christoph Hellwig 
Signed-off-by: Jens Axboe

blk-cgroup: introduce a new struct blkg_rwstat_sample

2019-06-20T16:32:34+00:00

When sampling the blkcg counts we don't need atomics or per-cpu
variables.  Introduce a new structure just containing plain u64
counters.

Acked-by: Tejun Heo 
Signed-off-by: Christoph Hellwig 
Signed-off-by: Jens Axboe

blk-cgroup: pass blkg_rwstat structures by reference

2019-06-20T16:32:34+00:00

Returning a structure generates rather bad code, so switch to passing
by reference.  Also don't require the structure to be zeroed and add
to the 0-initialized counters, but actually set the counters to the
calculated value.

Acked-by: Tejun Heo 
Signed-off-by: Christoph Hellwig 
Signed-off-by: Jens Axboe

blk-cgroup: factor out a helper to read rwstat counter

2019-06-20T16:32:34+00:00

Trying to break up the crazy statements to something readable.
Also switch to an unsigned counter as it can't ever turn negative.

Reviewed-by: Chaitanya Kulkarni 
Acked-by: Tejun Heo 
Signed-off-by: Christoph Hellwig 
Signed-off-by: Jens Axboe

blkcg: clean up blkg_tryget_closest()

2018-12-21T15:47:05+00:00

The implementation of blkg_tryget_closest() wasn't super obvious and
became a point of suspicion when debugging [1]. So let's clean it up so
it's obviously not the problem.

Also add missing RCU read locking to bio_clone_blkg_association(), which
got exposed by adding the RCU read lock held check in
blkg_tryget_closest().

[1] https://lore.kernel.org/linux-block/a7e97e4b-0dd8-3a54-23b7-a0f27b17fde8@kernel.dk/

Signed-off-by: Dennis Zhou 
Signed-off-by: Jens Axboe

blkcg: handle dying request_queue when associating a blkg

2018-12-13T00:43:33+00:00

Between v3 [1] and v4 [2] of the blkg association series, the
association point moved from generic_make_request_checks(), which is
called after the request enters the queue, to bio_set_dev(), which is when
the bio is formed before submit_bio(). When the request_queue goes away,
the blkgs supporting the request_queue are destroyed and then the
q->root_blkg is set to %NULL.

This patch adds a %NULL check to blkg_tryget_closest() to prevent the
NPE caused by the above. It also adds a guard to see if the
request_queue is dying when creating a blkg to prevent creating a blkg
for a dead request_queue.

[1] https://lore.kernel.org/lkml/20180911184137.35897-1-dennisszhou@gmail.com/
[2] https://lore.kernel.org/lkml/20181126211946.77067-1-dennis@kernel.org/

Fixes: 5cdf2e3fea5e ("blkcg: associate blkg when associating a device")
Reported-and-tested-by: Ming Lei 
Reviewed-by: Bart Van Assche 
Signed-off-by: Dennis Zhou 
Signed-off-by: Jens Axboe

blkcg: put back rcu lock in blkcg_bio_issue_check()

2018-12-08T05:26:38+00:00

I was a little overzealous in removing the rcu_read_lock() call from
blkcg_bio_issue_check() and it broke blk-throttle. Put it back.

Fixes: e35403a034bf ("blkcg: associate blkg when associating a device")
Signed-off-by: Dennis Zhou 
Signed-off-by: Jens Axboe

blkcg: rename blkg_try_get() to blkg_tryget()

2018-12-08T05:26:38+00:00

blkg reference counting now uses percpu_ref rather than atomic_t. Let's
make this consistent with css_tryget. This renames blkg_try_get to
blkg_tryget and now returns a bool rather than the blkg or %NULL.

Signed-off-by: Dennis Zhou 
Reviewed-by: Josef Bacik 
Acked-by: Tejun Heo 
Signed-off-by: Jens Axboe