Il 2020-02-17 03:59 Markus Kern ha scritto:
Greetings! I am currently evaluating our options to replace our old mixture of IBM SAN storage boxes. This will be a strategic decision for the next years. One of the solutions I am reviewing is a GlusterFS installation. Planned usage: - Central NFS server for around 25 systems providing around 400 docker containers - Central storage for a small VMWare vCenter cluster and a RedHat virtualization cluster. In total maybe around 15 machines The following requirements ensue from this: - Fast storage - High availability After reading all kind of tutorials and documentation, I came to the conclusion that for the expected traffic a "Distributed Replicate Volume" is the proper setup. Nothing has been purchased but I think about following small setup for the beginning (call it PoC): 4 x server, each with 8 x 1.8TB 10k SAS disks in a RAID60 Two 10 GBit interfaces per server: One for communication betweens the 4 systems only (separate VLAN), the other one for regular traffic between clients and servers. Does this all make sense? Generally speaking: Is such a setup capable of providing fast enough storage for a virtualization cluster? Do you have any hints? Thanks Markus
I evaluated such a setup, but I decided against it when using a small number of nodes/brick.
The key reason was bad sync performance even when using ramdisks *and* two local bricks (ie: minimal network overhead). You can read more here: https://lists.gluster.org/pipermail/gluster-users/2020-January/037601.html
The interesting thing is that when increasing the number of bricks, performance scaled well. So it seems gluster *can* be good at virtualization, but it need a large number of bricks (eg: an entire server rack or one-brick-for-physical-disk approach). This matches the experiences shared by other sysadmin.
Moreover, in order to have efficient resync/healing after a node reboot, you need to enable sharding (ie: the virtual disks will be divided in many small chunks). I was somewhat unconfortable doing that, as any problem with gluster incapable to mount the share would lead to quite trickly "file reconstruction puzzle".
So I ended with local storage (and a hot-standby server) rather than Gluster. If anyone has some different stories to share, I really am all ears.
Regards. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it [1] email: g.danti@xxxxxxxxxx - info@xxxxxxxxxx GPG public key ID: FF5F32A8 ________ Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/441850968 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
