Hi,
I am in the process of evaluation of Gluster for major BI company,
but I was surprised by very small write performance on Amazon EBS.
Our setup is Gluster 3.1.2, distributed replica 2x2 on 64-bit m1.large
instances. Every server node has 1 EBS volume attached to it.
The configuration of the distributed replica is a default one, my
small attemps to improve performance (io-threads, disabled io-stats
and latency-measurement):
volume EBSVolume-posix
type storage/posix
option directory /mnt/ebs
end-volume
volume EBSVolume-access-control
type features/access-control
subvolumes EBSVolume-posix
end-volume
volume EBSVolume-locks
type features/locks
subvolumes EBSVolume-access-control
end-volume
volume EBSVolume-io-threads
type performance/io-threads
option thread-count 4
subvolumes EBSVolume-locks
end-volume
volume /mnt/ebs
type debug/io-stats
option log-level NONE
option latency-measurement off
subvolumes EBSVolume-io-threads
end-volume
volume EBSVolume-server
type protocol/server
option transport-type tcp
option auth.addr./mnt/ebs.allow *
subvolumes /mnt/ebs
end-volume
In our test, all clients starts writing to different 1GB file at the same time.
The measured write bandwidth, with 2x2 servers:
1 client: 6.5 MB/s
2 clients: 4.1 MB/s
3 clients: 2.4 MB/s
4 clients: 4.3 MB/s
This is not acceptable for our needs. With PVFS2 (I know it's
stripping which is very different from replica) we can get up to 35
MB/s.
2-3 times slower than that would be understandable. But 5-15 times
slower is not, and I would like to know whether there is something we
could try out.
Could anybody publish their write speeds on similar setup, and tips
how to achieve better performance?
Thank you,
Karol
