Hi Hannes,
Thanks your detailed comments.
在 2022/5/6 0:19, Hannes Reinecke 写道:
On 5/4/22 19:27, chenxiang (M) wrote:
Hi Hannes and other guys,
For SCSI EH, i have a question (sorry, it is not related to this
patchset): for current flow of SCSI EH, if IOs of one disk is failed
(if there are many disks under the same scsi host), it will block all
the IOs of total scsi host.
So during SCSI EH, all IOs are blocked even if some disks are normal.
That's the place product line sometimes complain about
as it blocks IO bussiness of some normal disks because of just one
bad disk during SCSI EH.
Is it possible to split the SCSI EH into two parts, the process of
recovering the disk and the process of recovering scsi host, at the
beginning
If it were so easy.
The biggest problem we're facing in SCSI EH is that basically _all_
instances I've seen where EH got engaged were due to a command timeout.
Right, currently it is always a command timeout which makes EH got
engaged. The worse situation is that some IOs are failed with response
while other IOs
are timeout. Then when the first IO with response complete, it tries to
enter EH (just mark host SHOST_RECOVERY), then it begins to block IOs.
Normally maybe
after almost 30s, all those IOs are completed (timeout or failed)，then
it enters EH. So the blocking time of this situation is waiting for EH
(max 30s) + EH (serval seconds ~ 10+seconds).
Which means that we've sent a command to the HBA, and never heard from
it again. Now, it were easy if it would just be the command which has
vanished, but the problem is that we don't know what happened.
It might be the command being ln transit, the drive might be
unresponsive, or the HBA has gone off the rails altogether.
So until we've established where the command got lost, we have to
assume the worst and _have_ to treat the HBA as unreliable.
So initially we shouldn't isolate the device, and hope the failure is
restricted to the device.
Instead we have to stop I/O to the HBA, establish communication
(typically by sending a TMF), and only restart operations once we get
a response back from the HBA.
Ok, but what we see is that hard disk is more easily broken than HBA,
and usually error handling is due to a bad disk though the other disks
are normal.
Current SCSI EH is based on scsi host (there is a EH thread for every
scsi host), I think if SCSI EH is based on scsi device (there is a EH
thread for every scsi device),
when one IO of one disk is failed or timeout, we just mark the disk as
RECOVERY and trigger EH of the disk. Only when recovery operation of the
device also is failed, then
trigger EH of scsi host. Maybe it can alleviate the issue.
Even if there is something wrong with HBA, once IO of a disk is fialed
or timeout, it will also stop IOs of the disk immediately and
separately, and i think maybe it doesn't make much difference.
(In current SCSI EH, i think it also the situation that many IOs are
still sent to broken HBA, if previous IOs are all timeout).
This is especially true for old SCSI parallel HBA, where quite some
state is being kept in the HBA structure itself. So if we were to send
another command we would loas the state of the failed command, and
wouldn't be able to figure out the root cause on why the command had
failed.
Cheers,
Hannes
