linux-stable.git/block/bio.c, branch v4.2.6

block: Do a full clone when splitting discard bios

2015-07-23T22:21:34+00:00

This fixes a data corruption bug when using discard on top of MD linear,
raid0 and raid10 personalities.

Commit 20d0189b1012 "block: Introduce new bio_split()" permits sharing
the bio_vec between the two resulting bios. That is fine for read/write
requests where the bio_vec is immutable. For discards, however, we need
to be able to attach a payload and update the bio_vec so the page can
get mapped to a scatterlist entry. Therefore the bio_vec can not be
shared when splitting discards and we must do a full clone.

Signed-off-by: Martin K. Petersen 
Reported-by: Seunguk Shin 
Tested-by: Seunguk Shin 
Cc: Seunguk Shin 
Cc: Jens Axboe 
Cc: Kent Overstreet 
Cc:  # v3.14+
Reviewed-by: Christoph Hellwig 
Signed-off-by: Jens Axboe

block: export bio_associate_*() and wbc_account_io()

2015-07-23T19:36:44+00:00

bio_associate_blkcg(), bio_associate_current() and wbc_account_io()
are used to implement cgroup writeback support for filesystems and
thus need to be exported.  Export them.

Signed-off-by: Tejun Heo 
Reported-by: Stephen Rothwell 
Signed-off-by: Jens Axboe

blkcg: implement bio_associate_blkcg()

2015-06-02T14:33:34+00:00

Currently, a bio can only be associated with the io_context and blkcg
of %current using bio_associate_current().  This is too restrictive
for cgroup writeback support.  Implement bio_associate_blkcg() which
associates a bio with the specified blkcg.

bio_associate_blkcg() leaves the io_context unassociated.
bio_associate_current() is updated so that it considers a bio as
already associated if it has a blkcg_css, instead of an io_context,
associated with it.

Signed-off-by: Tejun Heo 
Cc: Jens Axboe 
Cc: Vivek Goyal 
Signed-off-by: Jens Axboe

cgroup, block: implement task_get_css() and use it in bio_associate_current()

2015-06-02T14:33:34+00:00

bio_associate_current() currently open codes task_css() and
css_tryget_online() to find and pin $current's blkcg css.  Abstract it
into task_get_css() which is implemented from cgroup side.  As a task
is always associated with an online css for every subsystem except
while the css_set update is propagating, task_get_css() retries till
css_tryget_online() succeeds.

This is a cleanup and shouldn't lead to noticeable behavior changes.

Signed-off-by: Tejun Heo 
Cc: Li Zefan 
Cc: Jens Axboe 
Cc: Vivek Goyal 
Signed-off-by: Jens Axboe

block: remove management of bi_remaining when restoring original bi_end_io

2015-05-22T14:58:55+00:00

Commit c4cf5261 ("bio: skip atomic inc/dec of ->bi_remaining for
non-chains") regressed all existing callers that followed this pattern:
 1) saving a bio's original bi_end_io
 2) wiring up an intermediate bi_end_io
 3) restoring the original bi_end_io from intermediate bi_end_io
 4) calling bio_endio() to execute the restored original bi_end_io

The regression was due to BIO_CHAIN only ever getting set if
bio_inc_remaining() is called.  For the above pattern it isn't set until
step 3 above (step 2 would've needed to establish BIO_CHAIN).  As such
the first bio_endio(), in step 2 above, never decremented __bi_remaining
before calling the intermediate bi_end_io -- leaving __bi_remaining with
the value 1 instead of 0.  When bio_inc_remaining() occurred during step
3 it brought it to a value of 2.  When the second bio_endio() was
called, in step 4 above, it should've called the original bi_end_io but
it didn't because there was an extra reference that wasn't dropped (due
to atomic operations being optimized away since BIO_CHAIN wasn't set
upfront).

Fix this issue by removing the __bi_remaining management complexity for
all callers that use the above pattern -- bio_chain() is the only
interface that _needs_ to be concerned with __bi_remaining.  For the
above pattern callers just expect the bi_end_io they set to get called!
Remove bio_endio_nodec() and also remove all bio_inc_remaining() calls
that aren't associated with the bio_chain() interface.

Also, the bio_inc_remaining() interface has been moved local to bio.c.

Fixes: c4cf5261 ("bio: skip atomic inc/dec of ->bi_remaining for non-chains")
Reviewed-by: Christoph Hellwig 
Reviewed-by: Jan Kara 
Signed-off-by: Mike Snitzer 
Signed-off-by: Jens Axboe

bio: skip atomic inc/dec of ->bi_cnt for most use cases

2015-05-05T19:32:49+00:00

Struct bio has a reference count that controls when it can be freed.
Most uses cases is allocating the bio, which then returns with a
single reference to it, doing IO, and then dropping that single
reference. We can remove this atomic_dec_and_test() in the completion
path, if nobody else is holding a reference to the bio.

If someone does call bio_get() on the bio, then we flag the bio as
now having valid count and that we must properly honor the reference
count when it's being put.

Tested-by: Robert Elliott 
Signed-off-by: Jens Axboe

bio: skip atomic inc/dec of ->bi_remaining for non-chains

2015-05-05T19:32:47+00:00

Struct bio has an atomic ref count for chained bio's, and we use this
to know when to end IO on the bio. However, most bio's are not chained,
so we don't need to always introduce this atomic operation as part of
ending IO.

Add a helper to elevate the bi_remaining count, and flag the bio as
now actually needing the decrement at end_io time. Rename the field
to __bi_remaining to catch any current users of this doing the
incrementing manually.

For high IOPS workloads, this reduces the overhead of bio_endio()
substantially.

Tested-by: Robert Elliott 
Acked-by: Kent Overstreet 
Reviewed-by: Jan Kara 
Signed-off-by: Jens Axboe

block: rewrite and split __bio_copy_iov()

2015-02-05T16:30:44+00:00

Rewrite __bio_copy_iov using the copy_page_{from,to}_iter helpers, and
split it into two simpler functions.

This commit should contain only literal replacements, without
functional changes.

Cc: Kent Overstreet 
Cc: Jens Axboe 
Cc: Al Viro 
Signed-off-by: Dongsu Park 
[hch: removed the __bio_copy_iov wrapper]
Signed-off-by: Christoph Hellwig 
Reviewed-by: Ming Lei 
Signed-off-by: Jens Axboe

block: merge __bio_map_user_iov into bio_map_user_iov

2015-02-05T16:30:43+00:00

And also remove the unused bdev argument.

Signed-off-by: Christoph Hellwig 
Reviewed-by: Ming Lei 
Signed-off-by: Jens Axboe

block: merge __bio_map_kern into bio_map_kern

2015-02-05T16:30:42+00:00

This saves a little code, and allow to simplify the error handling.

Signed-off-by: Christoph Hellwig 
Reviewed-by: Ming Lei 
Signed-off-by: Jens Axboe