linux-stable.git/include/linux/blk_types.h, branch v5.6.2

block: add iostat counters for flush requests

2019-11-21T16:06:47+00:00

Requests that triggers flushing volatile writeback cache to disk (barriers)
have significant effect to overall performance.

Block layer has sophisticated engine for combining several flush requests
into one. But there is no statistics for actual flushes executed by disk.
Requests which trigger flushes usually are barriers - zero-size writes.

This patch adds two iostat counters into /sys/class/block/$dev/stat and
/proc/diskstats - count of completed flush requests and their total time.

Signed-off-by: Konstantin Khlebnikov 
Signed-off-by: Jens Axboe

block: add zone open, close and finish operations

2019-11-07T13:31:48+00:00

Zoned block devices (ZBC and ZAC devices) allow an explicit control
over the condition (state) of zones. The operations allowed are:
* Open a zone: Transition to open condition to indicate that a zone will
  actively be written
* Close a zone: Transition to closed condition to release the drive
  resources used for writing to a zone
* Finish a zone: Transition an open or closed zone to the full
  condition to prevent write operations

To enable this control for in-kernel zoned block device users, define
the new request operations REQ_OP_ZONE_OPEN, REQ_OP_ZONE_CLOSE
and REQ_OP_ZONE_FINISH as well as the generic function
blkdev_zone_mgmt() for submitting these operations on a range of zones.
This results in blkdev_reset_zones() removal and replacement with this
new zone magement function. Users of blkdev_reset_zones() (f2fs and
dm-zoned) are updated accordingly.

Contains contributions from Matias Bjorling, Hans Holmberg,
Dmitry Fomichev, Keith Busch, Damien Le Moal and Christoph Hellwig.

Reviewed-by: Javier González 
Reviewed-by: Christoph Hellwig 
Signed-off-by: Ajay Joshi 
Signed-off-by: Matias Bjorling 
Signed-off-by: Hans Holmberg 
Signed-off-by: Dmitry Fomichev 
Signed-off-by: Keith Busch 
Signed-off-by: Damien Le Moal 
Signed-off-by: Jens Axboe

block: reorder bio::__bi_remaining for better packing

2019-10-25T20:12:50+00:00

Simple reordering of __bi_remaining can reduce bio size by 8 bytes that
are now wasted on padding (measured on x86_64):

struct bio {
        struct bio *               bi_next;              /*     0     8 */
        struct gendisk *           bi_disk;              /*     8     8 */
        unsigned int               bi_opf;               /*    16     4 */
        short unsigned int         bi_flags;             /*    20     2 */
        short unsigned int         bi_ioprio;            /*    22     2 */
        short unsigned int         bi_write_hint;        /*    24     2 */
        blk_status_t               bi_status;            /*    26     1 */
        u8                         bi_partno;            /*    27     1 */

        /* XXX 4 bytes hole, try to pack */

        struct bvec_iter   bi_iter;                      /*    32    24 */

        /* XXX last struct has 4 bytes of padding */

        atomic_t                   __bi_remaining;       /*    56     4 */

        /* XXX 4 bytes hole, try to pack */
[...]
        /* size: 104, cachelines: 2, members: 19 */
        /* sum members: 96, holes: 2, sum holes: 8 */
        /* paddings: 1, sum paddings: 4 */
        /* last cacheline: 40 bytes */
};

Now becomes:

struct bio {
        struct bio *               bi_next;              /*     0     8 */
        struct gendisk *           bi_disk;              /*     8     8 */
        unsigned int               bi_opf;               /*    16     4 */
        short unsigned int         bi_flags;             /*    20     2 */
        short unsigned int         bi_ioprio;            /*    22     2 */
        short unsigned int         bi_write_hint;        /*    24     2 */
        blk_status_t               bi_status;            /*    26     1 */
        u8                         bi_partno;            /*    27     1 */
        atomic_t                   __bi_remaining;       /*    28     4 */
        struct bvec_iter   bi_iter;                      /*    32    24 */

        /* XXX last struct has 4 bytes of padding */
[...]
        /* size: 96, cachelines: 2, members: 19 */
        /* paddings: 1, sum paddings: 4 */
        /* last cacheline: 32 bytes */
};

Signed-off-by: David Sterba 
Signed-off-by: Jens Axboe

blkcg: implement blk-iocost

2019-08-29T03:17:12+00:00

This patchset implements IO cost model based work-conserving
proportional controller.

While io.latency provides the capability to comprehensively prioritize
and protect IOs depending on the cgroups, its protection is binary -
the lowest latency target cgroup which is suffering is protected at
the cost of all others.  In many use cases including stacking multiple
workload containers in a single system, it's necessary to distribute
IO capacity with better granularity.

One challenge of controlling IO resources is the lack of trivially
observable cost metric.  The most common metrics - bandwidth and iops
- can be off by orders of magnitude depending on the device type and
IO pattern.  However, the cost isn't a complete mystery.  Given
several key attributes, we can make fairly reliable predictions on how
expensive a given stream of IOs would be, at least compared to other
IO patterns.

The function which determines the cost of a given IO is the IO cost
model for the device.  This controller distributes IO capacity based
on the costs estimated by such model.  The more accurate the cost
model the better but the controller adapts based on IO completion
latency and as long as the relative costs across differents IO
patterns are consistent and sensible, it'll adapt to the actual
performance of the device.

Currently, the only implemented cost model is a simple linear one with
a few sets of default parameters for different classes of device.
This covers most common devices reasonably well.  All the
infrastructure to tune and add different cost models is already in
place and a later patch will also allow using bpf progs for cost
models.

Please see the top comment in blk-iocost.c and documentation for
more details.

v2: Rebased on top of RQ_ALLOC_TIME changes and folded in Rik's fix
    for a divide-by-zero bug in current_hweight() triggered by zero
    inuse_sum.

Signed-off-by: Tejun Heo 
Cc: Andy Newell 
Cc: Josef Bacik 
Cc: Rik van Riel 
Signed-off-by: Jens Axboe

block: annotate refault stalls from IO submission

2019-08-14T14:50:01+00:00

psi tracks the time tasks wait for refaulting pages to become
uptodate, but it does not track the time spent submitting the IO. The
submission part can be significant if backing storage is contended or
when cgroup throttling (io.latency) is in effect - a lot of time is
spent in submit_bio(). In that case, we underreport memory pressure.

Annotate submit_bio() to account submission time as memory stall when
the bio is reading userspace workingset pages.

Tested-by: Suren Baghdasaryan 
Signed-off-by: Johannes Weiner 
Signed-off-by: Jens Axboe

block: add req op to reset all zones and flag

2019-08-05T03:41:29+00:00

This patch introduces a new request operation REQ_OP_ZONE_RESET_ALL.
This is useful for the applications like mkfs where it needs to reset
all the zones present on the underlying block device. As part for this
patch we also introduce new QUEUE_FLAG_ZONE_RESETALL which indicates the
queue zone reset all capability and corresponding helper macro.

Reviewed-by: Damien Le Moal 
Reviewed-by: Hannes Reinecke 
Signed-off-by: Chaitanya Kulkarni 
Signed-off-by: Jens Axboe

blk-mq: allow REQ_NOWAIT to return an error inline

2019-07-22T03:46:23+00:00

By default, if a caller sets REQ_NOWAIT and we need to block, we'll
return -EAGAIN through the bio->bi_end_io() callback. For some use
cases, this makes it hard to use.

Allow a caller to ask for inline return of errors related to
blocking by also setting REQ_NOWAIT_INLINE.

Signed-off-by: Jens Axboe

blkcg: implement REQ_CGROUP_PUNT

2019-07-10T15:00:57+00:00

When a shared kthread needs to issue a bio for a cgroup, doing so
synchronously can lead to priority inversions as the kthread can be
trapped waiting for that cgroup.  This patch implements
REQ_CGROUP_PUNT flag which makes submit_bio() punt the actual issuing
to a dedicated per-blkcg work item to avoid such priority inversions.

This will be used to fix priority inversions in btrfs compression and
should be generally useful as we grow filesystem support for
comprehensive IO control.

Cc: Chris Mason 
Reviewed-by: Josef Bacik 
Reviewed-by: Jan Kara 
Signed-off-by: Tejun Heo 
Signed-off-by: Jens Axboe

block: remove the bi_phys_segments field in struct bio

2019-06-20T16:29:22+00:00

We only need the number of segments in the blk-mq submission path.
Remove the field from struct bio, and return it from a variant of
blk_queue_split instead of that it can passed as an argument to
those functions that need the value.

This also means we stop recounting segments except for cloning
and partial segments.

To keep the number of arguments in this how path down remove
pointless struct request_queue arguments from any of the functions
that had it and grew a nr_segs argument.

Signed-off-by: Christoph Hellwig 
Signed-off-by: Jens Axboe

block: remove the bi_seg_{front,back}_size fields in struct bio

2019-05-23T16:25:26+00:00

At this point these fields aren't used for anything, so we can remove
them.

Reviewed-by: Ming Lei 
Reviewed-by: Hannes Reinecke 
Signed-off-by: Christoph Hellwig 
Signed-off-by: Jens Axboe