> It is very hard to compare them because they are structurally very different. For example, GlusterFS performance will depend *a lot* on the underlying file system performance. Ceph eliminated that factor by using Bluestore. > Ceph is very well performing for VM storage, since it's block based and as such optimized for that. I haven't tested CephFS a lot (I used it but only for very small storage) so I cannot speak for its performance, but I am guessing it's not ideal. For large amount of files thus GlusterFS is still a good choice. Was your experience above based on using a sharded volume or a normal one? When we worked with virtual machine images, we followed the volume sharding advice. I don't have a comparison for Ceph handy. I was just curious. It worked so well for us (but maybe our storage is "too good") that we found it hard to imagine it could be improved much. This was a simple case though of a single VM, 3 gluster servers, a sharded volume, and a raw virtual machine image. Probably a simpler case than yours. Thank you for writing this and take care, Erik > > One *MAJOR* advantage of Ceph over GlusterFS is tooling. Ceph's self-analytics, status reporting and problem fixing toolset is just so far beyond GlusterFS that it's really hard for me to recommend GlusterFS for any but the most experienced sysadmins. It does come with the type of implementation Ceph has chosen that they have to have such good tooling (because honestly, poking around in binary data structures really wouldn't be practical for most users), but whenever I had a problem with Ceph the solution was just a couple of command line commands (even if it meant to remove a storage device, wipe it and add it back), where with GlusterFS it means poking around in the .glusterfs directory, looking up inode numbers, extended attributes etc. which is a real pain if you have a multi-million-file filesystem to work on. And that's not even with sharding or distributed volumes. > > Also, Ceph has been a lot more stable that GlusterFS for us. The amount of hand-holding GlusterFS needs is crazy. With Ceph, there is this one bug (I think in certain Linux kernel versions) where it sometimes reads only zeroes from disk and complains about that and then you have to restart that OSD to not run into problems, but that's one "swatch" process on each machine that will do that automatically for us. I have run some Ceph clusters for several years now and only once or twice I had to deal with problems. The several GlusterFS clusters we operate constantly run into troubles. We now shut down all GlusterFS clients before we reboot any GlusterFS node because it was near impossible to reboot a single node without running into unrecoverable troubles (heal entries that will not heal etc.). With Ceph we can achieve 100% uptime, we regularly reboot our hosts one by one and some minutes later the Ceph cluster is clean again. > > If others have more insights I'd be very happy to hear them. > > Stefan > > > ----- Original Message ----- > > Date: Tue, 16 Jun 2020 20:30:34 -0700 > > From: Artem Russakovskii <archon810@xxxxxxxxx> > > To: Strahil Nikolov <hunter86_bg@xxxxxxxxx> > > Cc: gluster-users <gluster-users@xxxxxxxxxxx> > > Subject: Re: State of Gluster project > > Message-ID: > > <CAD+dzQdf_TiPBSDj57hY=t8AQ=mACrxinPX7iU4hmuxNMo+omg@xxxxxxxxxxxxxx> > > Content-Type: text/plain; charset="utf-8" > > > > Has anyone tried to pit Ceph against gluster? I'm curious what the ups and > > downs are. > > > > On Tue, Jun 16, 2020, 4:32 PM Strahil Nikolov <hunter86_bg@xxxxxxxxx> wrote: > > > >> Hey Mahdi, > >> > >> For me it looks like Red Hat are focusing more on CEPH than on Gluster. > >> I hope the project remains active, cause it's very difficult to find a > >> Software-defined Storage as easy and as scalable as Gluster. > >> > >> Best Regards, > >> Strahil Nikolov > >> > >> ?? 17 ??? 2020 ?. 0:06:33 GMT+03:00, Mahdi Adnan <mahdi@xxxxxxxxx> ??????: > >> >Hello, > >> > > >> > I'm wondering what's the current and future plan for Gluster project > >> >overall, I see that the project is not as busy as it was before "at > >> >least > >> >this is what I'm seeing" Like there are fewer blogs about what the > >> >roadmap > >> >or future plans of the project, the deprecation of Glusterd2, even Red > >> >Hat > >> >Openshift storage switched to Ceph. > >> >As the community of this project, do you feel the same? Is the > >> >deprecation > >> >of Glusterd2 concerning? Do you feel that the project is slowing down > >> >somehow? Do you think Red Hat is abandoning the project or giving fewer > >> >resources to Gluster? > >> ________ > >> > >> > >> > >> Community Meeting Calendar: > >> > >> Schedule - > >> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > >> Bridge: https://bluejeans.com/441850968 > >> > >> Gluster-users mailing list > >> Gluster-users@xxxxxxxxxxx > >> https://lists.gluster.org/mailman/listinfo/gluster-users > >> > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: > > <http://lists.gluster.org/pipermail/gluster-users/attachments/20200616/a1d0f142/attachment-0001.html > > ________ > > > > Community Meeting Calendar: > > Schedule - > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > https://lists.gluster.org/mailman/listinfo/gluster-users ________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
