On Mon, Mar 01, 2021 at 09:09:01AM +0000, Christoph Hellwig wrote: > On Fri, Feb 26, 2021 at 07:47:55AM +1100, Dave Chinner wrote: > > Sorry, I/O wait might matter for what? > > Think you have a SAS hard drive, WCE=0, typical queue depth of a few > dozend commands. Yup, so typical IO latency of 2-3ms, assuming 10krpm, assuming we aren't doing metadata writeback which will blow this out. I've tested on slower iSCSI devices than this (7-8ms typical av. seek time), and it didn't show up any performance anomalies. > Before that we'd submit a bunch of iclogs, which are generally > sequential except of course for the log wrap around case. The drive > can now easily take all the iclogs and write them in one rotation. Even if we take the best case for your example, this still means we block on every 8 iclogs waiting 2-3ms for the spindle to rotate and complete the IOs. Hence for a checkpoint of 32MB with 256kB iclogs, we're blocking for 2-3ms at least 16 times before we get to the commit iclog. With default iclog size of 32kB, we'll block a couple of hundred times waiting on iclog IO... IOWs, we're already talking about a best case checkpoint commit latency of 30-50ms here. [ And this isn't even considering media bandwidth there - 32MB on a drive that can do maybe 200MB/s in the middle of the spindle where the log is. That's another 150ms of data transfer time to physical media. So if the drive is actually writing to physical media because WCE=0, then we're taking *at least* 200ms per 32MB checkpoint. ] > Now if we wait for the previous iclogs before submitting the > commit_iclog we need at least one more additional full roundtrip. So we add an average of 2-3ms to what is already taking, in the best case, 30-50ms. And these are mostly async commits this overhead is added to, so there's rarely anything waiting on it and hence the extra small latency is almost always lost in the noise. Even if the extra delay is larger, there is rarely anything waiting on it so it's still noise... I just don't see anything relevant that stands out from the noise on my systems. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx
