On 01/31/2010 09:06 AM, Ran wrote: > You guys are talking about network IO im taking about the gluster server disk IO > the idea to shape the trafic does make sence seens the virt machines > server do use network to get to the disks(gluster) > but what about if there are say 5 KVM servers(with VPS's) all on > gluster what do you do then ? its not quite fair share seens every > server has its own fair share and doesnt see the others . > > Also there are other applications that uses gluster like mail etc.. > and i see that gluster IO is very high very often cousing the all > storage not to work . > Its very disturbing . You bring up a good set of points. Some of these problems can be addressed at the hypervisor (i.e. GlusterFS client) level, some can be addressed by GlusterFS itself, and some can be addressed only at the level of the local-filesystem or block-device level on the GlusterFS servers. Unfortunately, I/O traffic shaping is still in its infancy compared to what's available for networking - or perhaps even "infancy" is too generous. As far as the I/O stack is concerned, all of the traffic is coming from the glusterfsd process(es) without differentiation, so even if the functionality to apportion I/O amongst tasks existed it wouldn't be usable without more information. Maybe some day... What you can do now at the GlusterFS level, though, is make sure that traffic is distributed across many servers and possibly across many volumes per server to take advantage of multiple physical disks and/or interconnects for one server. That way, a single VM will only use a small subset of the servers/volumes and will not starve other clients that are using different servers/volumes (except for network bottlenecks which are a separate issue). That's what the "distribute" translator is for, and it can be combined with replicate or stripe to provide those functions as well. Perhaps it would be useful to create and publish some up-to-date recipes for these sorts of combinations.
