On Thu, Mar 2, 2023 at 11:44 AM Gionatan Danti <g.danti@xxxxxxxxxx> wrote: > > Il 2023-03-02 01:51 Roger Heflin ha scritto: > > A spinning raid6 array is slow on writes (see raid6 write penalty). > > Because of that the array can only do about 100 write operattions/sec. > > True. But does flushing cached data really proceed in random LBA order > (as seen by HDDs), rather than trying to coalesce writes in linear > fashion? > It is a 100G cache over 16TB, so even if it flushes in order the may not be that close to each other (1 in 160). Also if pieces are decided and added to the cached then the cache is not in order on the ssd and proper coalescing would require reading the entire cache and sorting the 3,000,000 location entries before starting the de-stage. And that complication of a de-stage is likely not been coded yet if I was just guessing, the de-stage starts at the beginning and continues to the end of the cache. Even coded though, if the you have enough blocks cached and if the blocks spread say one or 2 on each track it would break down to having to write a tiny bit on each track with seeks between mostly breaking down to the time required to simply read/write the HD end to end. At 150MB/sec (should be about the platter speed) that would take 3.5 hours. > > If the disk is doing other work then it only has the extra capacity so > > it could destage slower. > > > > A lot depends on how big each chunk is. The lvmcache indicates the > > smallest chunksize is 32k. > > > > 100G / 32k = 3 million, and at 100seeks/sec that comes to at least an > > hour. > > You are off an order of magnitude: 3 millions IOP at 100 IOPs means > ~30000s, so about 9 hours. Right, I did the calc in my head and screwed it up. I thought it should have been higher but did not re-check it. > > > Lvm bookkeeping has to also be written to the spinning disks I would > > think, so 2 hours if the array were idle. > > > > Throw in a 50% baseload on the disks and you get 4 hours. > > > > Hours is reasonable. > > If flushing happens in random disk order, than yes, you are bound to > wait several hours indeed. > _______________________________________________ linux-lvm mailing list linux-lvm@xxxxxxxxxx https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
