linux.git/drivers/scsi/scsi_error.c, branch v3.15 Linux kernel source tree [SCSI] More USB deadlock fixes 2014-04-21T21:28:40+00:00 James Bottomley JBottomley@Parallels.com 2014-04-10T20:36:11+00:00 c69e6f812bab0d5442b40e2f1bfbca48d40bc50b This patch fixes a corner case in the previous USB Deadlock fix patch (12023e7 [SCSI] Fix USB deadlock caused by SCSI error handling). The scenario is abort command, set flag, abort completes, send TUR, TUR doesn't return, so we now try to abort the TUR, but scsi_abort_eh_cmnd() will skip the abort because the flag is set and move straight to reset. Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: James Bottomley <JBottomley@Parallels.com> 
This patch fixes a corner case in the previous USB Deadlock fix patch (12023e7
[SCSI] Fix USB deadlock caused by SCSI error handling).

The scenario is abort command, set flag, abort completes, send TUR, TUR
doesn't return, so we now try to abort the TUR, but scsi_abort_eh_cmnd()
will skip the abort because the flag is set and move straight to reset.

Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
[SCSI] Fix USB deadlock caused by SCSI error handling 2014-04-21T21:28:26+00:00 Hannes Reinecke hare@suse.de 2014-03-31T14:37:34+00:00 7daf480483e60898f30e0a2a84fecada7a7cfac0 USB requires that every command be aborted first before we escalate to reset. In particular, USB will deadlock if we try to reset first before aborting the command. Unfortunately, the flag we use to tell if a command has already been aborted: SCSI_EH_ABORT_SCHEDULED is not cleared properly leading to cases where we can requeue a command with the flag set and proceed immediately to reset if it fails (thus causing USB to deadlock). Fix by clearing the SCSI_EH_ABORT_SCHEDULED flag if it has been set. Which means this will be the second time scsi_abort_command() has been called for the same command. IE the first abort went out, did its thing, but now the same command has timed out again. So this flag gets cleared, and scsi_abort_command() returns FAILED, and _no_ asynchronous abort is being scheduled. scsi_times_out() will then proceed to call scsi_eh_scmd_add(). But as we've cleared the SCSI_EH_ABORT_SCHEDULED flag the SCSI_EH_CANCEL_CMD flag will continue to be set, and the command will be aborted with the main SCSI EH routine. Reported-by: Alan Stern <stern@rowland.harvard.edu> Tested-by: Andreas Reis <andreas.reis@gmail.com> Signed-off-by: Hannes Reinecke <hare@suse.de> Signed-off-by: James Bottomley <JBottomley@Parallels.com> 
USB requires that every command be aborted first before we escalate to reset.
In particular, USB will deadlock if we try to reset first before aborting the
command.

Unfortunately, the flag we use to tell if a command has already been aborted:
SCSI_EH_ABORT_SCHEDULED is not cleared properly leading to cases where we can
requeue a command with the flag set and proceed immediately to reset if it
fails (thus causing USB to deadlock).

Fix by clearing the SCSI_EH_ABORT_SCHEDULED flag if it has been set.  Which
means this will be the second time scsi_abort_command() has been called for
the same command.  IE the first abort went out, did its thing, but now the
same command has timed out again.

So this flag gets cleared, and scsi_abort_command() returns FAILED, and _no_
asynchronous abort is being scheduled.  scsi_times_out() will then proceed to
call scsi_eh_scmd_add().  But as we've cleared the SCSI_EH_ABORT_SCHEDULED
flag the SCSI_EH_CANCEL_CMD flag will continue to be set, and the command will
be aborted with the main SCSI EH routine.

Reported-by: Alan Stern <stern@rowland.harvard.edu>
Tested-by: Andreas Reis <andreas.reis@gmail.com>
Signed-off-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
[SCSI] Fix command result state propagation 2014-04-21T21:27:26+00:00 Alan Stern stern@rowland.harvard.edu 2014-03-28T17:51:15+00:00 644373a4219add42123df69c8b7ce6a918475ccd We're seeing a case where the contents of scmd->result isn't being reset after a SCSI command encounters an error, is resubmitted, times out and then gets handled. The error handler acts on the stale result of the previous error instead of the timeout. Fix this by properly zeroing the scmd->status before the command is resubmitted. Signed-off-by: Alan Stern <stern@rowland.harvard.edu> Signed-off-by: James Bottomley <JBottomley@Parallels.com> 
We're seeing a case where the contents of scmd->result isn't being reset after
a SCSI command encounters an error, is resubmitted, times out and then gets
handled.  The error handler acts on the stale result of the previous error
instead of the timeout.  Fix this by properly zeroing the scmd->status before
the command is resubmitted.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
[SCSI] Fix spurious request sense in error handling 2014-04-21T21:27:05+00:00 James Bottomley JBottomley@Parallels.com 2014-03-28T17:50:17+00:00 d555a2abf3481f81303d835046a5ec2c4fb3ca8e We unconditionally execute scsi_eh_get_sense() to make sure all failed commands that should have sense attached, do. However, the routine forgets that some commands, because of the way they fail, will not have any sense code ... we should not bother them with a REQUEST_SENSE command. Fix this by testing to see if we actually got a CHECK_CONDITION return and skip asking for sense if we don't. Tested-by: Alan Stern <stern@rowland.harvard.edu> Signed-off-by: James Bottomley <JBottomley@Parallels.com> 
We unconditionally execute scsi_eh_get_sense() to make sure all failed
commands that should have sense attached, do.  However, the routine forgets
that some commands, because of the way they fail, will not have any sense code
... we should not bother them with a REQUEST_SENSE command.  Fix this by
testing to see if we actually got a CHECK_CONDITION return and skip asking for
sense if we don't.

Tested-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
[SCSI] do not manipulate device reference counts in scsi_get/put_command 2014-03-15T17:19:24+00:00 Christoph Hellwig hch@infradead.org 2014-02-20T22:20:55+00:00 0479633686d370303e3430256ace4bd5f7f138dc Many callers won't need this and we can optimize them away. In addition the handling in the __-prefixed variants was inconsistant to start with. Based on an earlier patch from Bart Van Assche. [jejb: fix kerneldoc probelm picked up by Fengguang Wu] Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: James Bottomley <JBottomley@Parallels.com> 
Many callers won't need this and we can optimize them away.  In addition
the handling in the __-prefixed variants was inconsistant to start with.

Based on an earlier patch from Bart Van Assche.

[jejb: fix kerneldoc probelm picked up by Fengguang Wu]
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
[SCSI] Set the minimum valid value of 'eh_deadline' as 0 2013-12-19T15:39:02+00:00 Ren Mingxin renmx@cn.fujitsu.com 2013-11-11T12:44:56+00:00 bb3b621a33d60fc2baddf31597ade01243e00a2c The former minimum valid value of 'eh_deadline' is 1s, which means the earliest occasion to shorten EH is 1 second later since a command is failed or timed out. But if we want to skip EH steps ASAP, we have to wait until the first EH step is finished. If the duration of the first EH step is long, this waiting time is excruciating. So, it is necessary to accept 0 as the minimum valid value for 'eh_deadline'. According to my test, with Hannes' patchset 'New EH command timeout handler' as well, the minimum IO time is improved from 73s (eh_deadline = 1) to 43s(eh_deadline = 0) when commands are timed out by disabling RSCN and target port. Signed-off-by: Ren Mingxin <renmx@cn.fujitsu.com> Signed-off-by: Hannes Reinecke <hare@suse.de> Signed-off-by: James Bottomley <JBottomley@Parallels.com> 
The former minimum valid value of 'eh_deadline' is 1s, which means
the earliest occasion to shorten EH is 1 second later since a
command is failed or timed out. But if we want to skip EH steps
ASAP, we have to wait until the first EH step is finished. If the
duration of the first EH step is long, this waiting time is
excruciating. So, it is necessary to accept 0 as the minimum valid
value for 'eh_deadline'.

According to my test, with Hannes' patchset 'New EH command timeout
handler' as well, the minimum IO time is improved from 73s
(eh_deadline = 1) to 43s(eh_deadline = 0) when commands are timed
out by disabling RSCN and target port.

Signed-off-by: Ren Mingxin <renmx@cn.fujitsu.com>
Signed-off-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
[SCSI] Unlock accesses to eh_deadline 2013-12-19T15:39:02+00:00 Hannes Reinecke hare@suse.de 2013-11-11T12:44:55+00:00 76ad3e5956bf0bc8871ebd19ebda03f2287c966a 32bit accesses are guaranteed to be atomic, so we can remove the spinlock when checking for eh_deadline. We only need to make sure to catch any updates which might happened during the call to time_before(); if so we just recheck with the correct value. Signed-off-by: Hannes Reinecke <hare@suse.de> Signed-off-by: James Bottomley <JBottomley@Parallels.com> 
32bit accesses are guaranteed to be atomic, so we can remove
the spinlock when checking for eh_deadline. We only need to
make sure to catch any updates which might happened during
the call to time_before(); if so we just recheck with the
correct value.

Signed-off-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
[SCSI] improved eh timeout handler 2013-12-19T15:39:02+00:00 Hannes Reinecke hare@suse.de 2013-11-11T12:44:54+00:00 e494f6a728394ab0df194342549ee20e6f0752df When a command runs into a timeout we need to send an 'ABORT TASK' TMF. This is typically done by the 'eh_abort_handler' LLDD callback. Conceptually, however, this function is a normal SCSI command, so there is no need to enter the error handler. This patch implements a new scsi_abort_command() function which invokes an asynchronous function scsi_eh_abort_handler() to abort the commands via the usual 'eh_abort_handler'. If abort succeeds the command is either retried or terminated, depending on the number of allowed retries. However, 'eh_eflags' records the abort, so if the retry would fail again the command is pushed onto the error handler without trying to abort it (again); it'll be cleared up from SCSI EH. [hare: smatch detected stray switch fixed] Signed-off-by: Hannes Reinecke <hare@suse.de> Signed-off-by: James Bottomley <JBottomley@Parallels.com> 
When a command runs into a timeout we need to send an 'ABORT TASK'
TMF. This is typically done by the 'eh_abort_handler' LLDD callback.

Conceptually, however, this function is a normal SCSI command, so
there is no need to enter the error handler.

This patch implements a new scsi_abort_command() function which
invokes an asynchronous function scsi_eh_abort_handler() to
abort the commands via the usual 'eh_abort_handler'.

If abort succeeds the command is either retried or terminated,
depending on the number of allowed retries. However, 'eh_eflags'
records the abort, so if the retry would fail again the
command is pushed onto the error handler without trying to
abort it (again); it'll be cleared up from SCSI EH.

[hare: smatch detected stray switch fixed]
Signed-off-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
[SCSI] Fix erratic device offline during EH 2013-12-19T15:39:02+00:00 James Bottomley jbottomley@parallels.com 2013-11-11T12:44:53+00:00 2451079bc2ae1334058be8babd44be03ecfa7041 Commit 18a4d0a22ed6c54b67af7718c305cd010f09ddf8 (Handle disk devices which can not process medium access commands) was introduced to offline any device which cannot process medium access commands. However, commit 3eef6257de48ff84a5d98ca533685df8a3beaeb8 (Reduce error recovery time by reducing use of TURs) reduced the number of TURs by sending it only on the first failing command, which might or might not be a medium access command. So in combination this results in an erratic device offlining during EH; if the command where the TUR was sent upon happens to be a medium access command the device will be set offline, if not everything proceeds as normal. This patch moves the check to the final test, eliminating this problem. Signed-off-by: Hannes Reinecke <hare@suse.de> Signed-off-by: James Bottomley <JBottomley@Parallels.com> 
Commit 18a4d0a22ed6c54b67af7718c305cd010f09ddf8
(Handle disk devices which can not process medium access commands)
was introduced to offline any device which cannot process medium
access commands.
However, commit 3eef6257de48ff84a5d98ca533685df8a3beaeb8
(Reduce error recovery time by reducing use of TURs) reduced
the number of TURs by sending it only on the first failing
command, which might or might not be a medium access command.
So in combination this results in an erratic device offlining
during EH; if the command where the TUR was sent upon happens
to be a medium access command the device will be set offline,
if not everything proceeds as normal.

This patch moves the check to the final test, eliminating
this problem.

Signed-off-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
[SCSI] scsi_error: Escalate to LUN reset if abort fails 2013-10-25T11:18:30+00:00 Hannes Reinecke hare@suse.de 2013-10-23T08:51:22+00:00 6fd046f960354250de3fed411eecdb784601bbad If a command abort fails there is a fair chance that all other aborts will be failing, too. So we should be calling LUN reset directly after the first failed abort and skip aborting the remaining commands. Signed-off-by: Hannes Reinecke <hare@suse.de> Signed-off-by: James Bottomley <JBottomley@Parallels.com> 
If a command abort fails there is a fair chance that all other
aborts will be failing, too.
So we should be calling LUN reset directly after the first failed
abort and skip aborting the remaining commands.

Signed-off-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>