On Tue, 5 Nov 2019, Jens Axboe wrote: > On 11/5/19 11:31 AM, Alan Stern wrote: > > On Tue, 5 Nov 2019, Andrea Vai wrote: > > > >> Il giorno lun, 04/11/2019 alle 13.20 -0500, Alan Stern ha scritto: > > > >>> You should be able to do something like this: > >>> > >>> cd linux > >>> patch -p1 </path/to/patch2 > >>> > >>> and that should work with no errors. You don't need to use git to > >>> apply a patch. > >>> > >>> In case that patch2 file was mangled somewhere along the way, I > >>> have > >>> attached a copy to this message. > >> > >> Ok, so the "patch" command worked, the kernel compiled and ran, but > >> the test still failed (273, 108, 104, 260, 177, 236, 179, 1123, 289, > >> 873 seconds to copy a 500MB file, vs. ~30 seconds with the "good" > >> kernel). > >> > >> Let me know what else could I do, > > > > I'm out of suggestions. If anyone else knows how to make a kernel with > > no legacy queuing support -- only multiqueue -- issue I/O requests > > sequentially, please speak up. > > Do we know for a fact that the device needs strictly serialized requests > to not stall? Not exactly, but that is far and away the most likely explanation for the device's behavior. We tried making a bunch of changes, some of which helped a little bit, but all of them left a very large performance gap. I/O monitoring showed that the only noticeable difference in the kernel-device interaction caused by the $SUBJECT commit was the non-sequential access pattern. > And writes in particular? Andrea has tested only the write behavior. Possibly reading will be affected too, but my guess is that it won't be. > I won't comment on how broken > that is, just trying to establish this as the problem that's making this > particular device be slow? It seems reasonable that the access pattern could make a significant difference. The device's behavior suggests that it buffers incoming data and pauses from time to time to write the accumulated data into non-volatile storage. If its algorithm for allocating, erasing, and writing data blocks is optimized for the sequential case, you can easily imagine that non-sequential accesses would cause it to pause more often and for longer times -- which is exactly what we observed. These extra pauses are what resulted in the overall performance decrease. So far we have had no way to perform a direct test. That is, we don't know of any setting that would change a single kernel between sequential and non-sequential access. If you can suggest a simple way to force a kernel without the $SUBJECT commit to do non-sequential writes, I'm sure Andrea will be happy to try it out and see if it causes a slowdown. > I've lost track of this thread, but has mq-deadline been tried as the > IO scheduler? We do have support for strictly serialized (writes) > since that's required for zoned device, wouldn't be hard at all to make > this cover a blacklisted device like this one. Please spell out the exact procedure in detail so that Andrea can try it. He's not a kernel hacker, and I know very little about the block layer. Alan Stern
