linux.git/include/linux/blk-mq.h, branch v5.2-rc5

blk-mq: always free hctx after request queue is freed

2019-05-04T13:24:08+00:00

In normal queue cleanup path, hctx is released after request queue
is freed, see blk_mq_release().

However, in __blk_mq_update_nr_hw_queues(), hctx may be freed because
of hw queues shrinking. This way is easy to cause use-after-free,
because: one implicit rule is that it is safe to call almost all block
layer APIs if the request queue is alive; and one hctx may be retrieved
by one API, then the hctx can be freed by blk_mq_update_nr_hw_queues();
finally use-after-free is triggered.

Fixes this issue by always freeing hctx after releasing request queue.
If some hctxs are removed in blk_mq_update_nr_hw_queues(), introduce
a per-queue list to hold them, then try to resuse these hctxs if numa
node is matched.

Cc: Dongli Zhang 
Cc: James Smart 
Cc: Bart Van Assche 
Cc: linux-scsi@vger.kernel.org,
Cc: Martin K . Petersen ,
Cc: Christoph Hellwig ,
Cc: James E . J . Bottomley ,
Reviewed-by: Hannes Reinecke 
Tested-by: James Smart 
Signed-off-by: Ming Lei 
Signed-off-by: Jens Axboe

blk-mq: introduce blk_mq_complete_request_sync()

2019-04-10T15:57:33+00:00

In NVMe's error handler, follows the typical steps of tearing down
hardware for recovering controller:

1) stop blk_mq hw queues
2) stop the real hw queues
3) cancel in-flight requests via
	blk_mq_tagset_busy_iter(tags, cancel_request, ...)
cancel_request():
	mark the request as abort
	blk_mq_complete_request(req);
4) destroy real hw queues

However, there may be race between #3 and #4, because blk_mq_complete_request()
may run q->mq_ops->complete(rq) remotelly and asynchronously, and
->complete(rq) may be run after #4.

This patch introduces blk_mq_complete_request_sync() for fixing the
above race.

Cc: Sagi Grimberg 
Cc: Bart Van Assche 
Cc: James Smart 
Cc: linux-nvme@lists.infradead.org
Reviewed-by: Keith Busch 
Reviewed-by: Christoph Hellwig 
Signed-off-by: Ming Lei 
Signed-off-by: Jens Axboe

block: Unexport blk_mq_add_to_requeue_list()

2019-03-20T20:19:36+00:00

This function is not used outside the block layer core. Hence unexport it.

Cc: Christoph Hellwig 
Cc: Ming Lei 
Signed-off-by: Bart Van Assche 
Signed-off-by: Jens Axboe

blk-mq: remove unused 'nr_expired' from blk_mq_hw_ctx

2019-03-19T15:04:06+00:00

There is no usage of 'nr_expired'.

The 'nr_expired' was introduced by commit 1d9bd5161ba3 ("blk-mq: replace
timeout synchronization with a RCU and generation based scheme"). Its usage
was removed since commit 12f5b9314545 ("blk-mq: Remove generation
seqeunce").

Signed-off-by: Dongli Zhang 
Signed-off-by: Jens Axboe

block: kill BLK_MQ_F_SG_MERGE

2019-02-15T15:40:12+00:00

QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too.

Reviewed-by: Christoph Hellwig 
Reviewed-by: Omar Sandoval 
Signed-off-by: Ming Lei 
Signed-off-by: Jens Axboe

block: make request_to_qc_t public

2018-12-18T16:50:47+00:00

block consumers will need it for polling requests that
are sent with blk_execute_rq_nowait. Also, get rid of
blk_tag_to_qc_t and open-code it instead.

Reviewed-by: Jens Axboe 
Signed-off-by: Sagi Grimberg 
Signed-off-by: Christoph Hellwig

blk-mq: change blk_mq_queue_busy() to blk_mq_queue_inflight()

2018-12-18T04:31:42+00:00

There's a single user of this function, dm, and dm just wants
to check if IO is inflight, not that it's just allocated.

This fixes a hang with srp/002 in blktests with dm, where it tries
to suspend but waits for inflight IO to finish first. As it checks
for just allocated requests, this fails.

Tested-by: Mike Snitzer 
Signed-off-by: Jens Axboe

block: move queues types to the block layer

2018-12-04T18:38:17+00:00

Having another indirect all in the fast path doesn't really help
in our post-spectre world.  Also having too many queue type is just
going to create confusion, so I'd rather manage them centrally.

Note that the queue type naming and ordering changes a bit - the
first index now is the default queue for everything not explicitly
marked, the optional ones are read and poll queues.

Reviewed-by: Sagi Grimberg 
Signed-off-by: Christoph Hellwig 
Signed-off-by: Jens Axboe

blk-mq: add mq_ops->commit_rqs()

2018-11-29T17:11:56+00:00

blk-mq passes information to the hardware about any given request being
the last that we will issue in this sequence. The point is that hardware
can defer costly doorbell type writes to the last request. But if we run
into errors issuing a sequence of requests, we may never send the request
with bd->last == true set. For that case, we need a hook that tells the
hardware that nothing else is coming right now.

For failures returned by the drivers ->queue_rq() hook, the driver is
responsible for flushing pending requests, if it uses bd->last to
optimize that part. This works like before, no changes there.

Reviewed-by: Omar Sandoval 
Reviewed-by: Ming Lei 
Reviewed-by: Christoph Hellwig 
Signed-off-by: Jens Axboe

blk-mq: Simplify request completion state

2018-11-26T17:34:27+00:00

There are no more users relying on blk-mq request states to prevent
double completions, so replace the relatively expensive cmpxchg operation
with WRITE_ONCE.

Reviewed-by: Christoph Hellwig 
Signed-off-by: Keith Busch 
Signed-off-by: Jens Axboe