The SATA disks work ok if you use smartctl to set the SCTERC timeout
as low as possible (smartctl -l scterc,20,20 /dev/${drive} ). I have
a set of commands that starts high and sets it lower with the idea
that each different manufactuers disk will have a different min value
and I simply want it is as low as I can go.
Desktop/Green disks do not have a settable timeout and timeout at 60+
seconds or more.
Red/NAS/Video/Enterprise/Purple SATA and such typically timeout at
10sec but can be set lower. SAS timeouts are typically 1sec or lower
on a bad sector.
And I have personally dealt with "enterprise" vendors that when using
SATA leave the timeout at default (10 seconds) rather than lowering it
so that the disks work reasonably when bad sectors are happening.
On Tue, Jun 4, 2024 at 9:50 AM Jaco Kroon <jaco@xxxxxxxxx> wrote:
>
> Hi,
>
> On 2024/06/04 15:46, Stuart D Gathman wrote:
> > On Tue, 4 Jun 2024, Roger Heflin wrote:
> >
> >> My experience is that heavy disk io/batch disk io systems work better
> >> with these values being smallish.
> >
> >> I don't see a use case for having large values. It seems to have no
> >> real upside and several downsides. Get the buffer size small enough
> >> and you will still get pauses to clear the writes the be pauses will
> >> be short enough to not be a problem.
> >
> > Not a normal situation, but I should mention my recent experience.
> > One of the disks in an underlying RAID was going bad. It still worked,
> > but the disk struggled manfully with multiple retries and recalibrates
> > to complete many reads/writes - i.e. it was extremely slow. I was
> > running into all kinds of strange boundary conditions because of this.
> > E.g. VMs were getting timeouts on their virtio disk devices, leading
> > to file system corruption and other issues.
> >
> > I was not modifying any LVM volumes, so did not run into any problems
> > with LVM - but that is a boundary condition to keep in mind. You
> > don't necessarily need to fully work under such conditions, but need
> > to do something sane.
>
> On SAS or NL-SAS drives?
>
> I've seen this before on SATA drives, and is probably the single biggest
> reason why I have a major dislike for deploying SATA drives to any kind
> of high-reliability environment.
>
> Regardless, we do monitor all cases using smartd and it *usually* picks
> up a bad drive before it gets to the above point of pain but with SATA
> drives this isn't always the case, and the drive will simply
> indefinitely keep retrying, blocking request slot numbers over time
> (SATA protocol handles 32 requests IIRC, but after how long can the
> Linux kernel re-use a number it has never received a response on kind of
> problem) and getting slower and slower until you power cycle the drive,
> after which it's fine again for a while. Never had that crap with
> NL-SAS drives.
>
> Specific host is all NL-SAS. No VM involvement here.
>
> Kind regards,
> Jaco
>
