On Thu 30-05-24 21:19:55, Sagi Grimberg wrote: > > > btw, I think you meant that *slower* devices may need a larger queue to > > > saturate, > > > because if the device is fast, 256MB inflight is probably enough... So > > > you are solving > > > for the "consumer grade contemporary disks". > > No, faster devices. The context here is random writes. The more of > > those that are in the queue on the server, the more chance it has to > > re-order or coalesce requests. > > Umm, what would reorder help with if writes are random? Well, even though writes are random, they are still from a limited space so the more of them you have, the larger contiguous chunks you are actually going to get. But that's not really the main helping factor here. The main benefit of allowing more than 256MB to be fed into the server is indeed that the writeback on the server can be more contiguous. With 256MB limit what I see is that the server gets the data, then commit which triggers writeout of the data on the server, the server then signals completion and idles before the it gets more data from the client. With say 1GB or larger limit, the server has some data to write most of the time so overall throughput and disk utilization is much better. So it is all about how latency of the network, the speed of the storage and batching of writeback completions plays together. And yes, we can just increase the limit but my opinion still is that limiting the number of MB of writeback we have outstanding against the server is a wrong way of throttling things because that number is difficult to get right without knowing storage speeds, network speeds, load from other clients and internals of page writeback... I understand Trond's concerns about latency increase as well as your concerns about overloading the server with RPC traffic so perhaps we need *some* mechanism by which the client can detect contention. But I think we need to come up with something better than fixed number of MB... Honza -- Jan Kara <jack@xxxxxxxx> SUSE Labs, CR
