Jeff Darcy wrote:
Unfortunately, I/O traffic shaping is still in its infancy
compared to what's available for networking - or perhaps even "infancy"
is too generous. As far as the I/O stack is concerned, all of the
traffic is coming from the glusterfsd process(es) without
differentiation, so even if the functionality to apportion I/O amongst
tasks existed it wouldn't be usable without more information. Maybe
some day...
I don't think this would even be useful. It sounds like seeking more
finely grained (sub-process level!) control over disk I/O prioritisation
without there even being a clearly presented case about the current
functionality (ionice) not being sufficient.
Does such a case really need to be made explicitly? Dividing processes
into classes is all well and good, but there can still be contention
between processes in the same class. Being able to resolve that
contention in a fair and/or deterministic way is still useful, and still
unaddressed.
Hmm, perhaps. I never found that the degree of granularity that ionice
provides is insufficient.
In any case, that might be a moot point. I interpreted Ran's problem as
VMs running on GlusterFS *clients* causing contention at the GlusterFS
*servers*. Maybe that was incorrect, but even if Ran doesn't face that
problem others do. I certainly see and hear about it a lot from where I
sit at Red Hat, and no amount of tweaking at the hypervisor (i.e.
GlusterFS client) level will solve it.
So the problem is that a small number of clients can overwhelm the
servers and the other clients then get very little throughput to the
servers? If that is the case, it would imply that there is a vastly
different "cost" top operations in glfs, and that a few clients
hammering the expensive operations can result in the more conventional
workload clients slowing down. If that really is the case, then, OK, I
agree - that is something that perhaps needs to be looked into at glfs
protocol level.
But that is now what my interpretation of the problem was. The way I
read it, the complaint was that glusterfsd causes a lot of disk I/O,
which in turn makes the VM containing it eat all of the host's disk I/O
and other VMs on that host slow down. Hence why I was suggesting
ionice-ing the qemu process containing the server.
Gordan
