David C. Miller <millerdc@...> writes:
>
>
> ----- Original Message -----
> > From: "John Doe" <jdmls@...>
> > To: "Cent O Smailinglist" <centos@...>
> > Sent: Tuesday, August 28, 2012 3:14:29 AM
> > Subject: Is glusterfs ready?
> >
> > Hey,
> >
> > since RH took control of glusterfs, I've been looking to convert our
> > old independent RAID storage servers to several non RAID glustered
> > ones.
> >
> > The thing is that I, here and there, heard a few frightening stories
> > from some users (even with latest release).
> > Any one has experienced with it long enough to think one can blindly
> > trust it or if it is almost there but not yet ready?
> >
Heya,
Well I guess I'm one of the frightening stories, or at least a
previous employer was. They had a mere 0.1 petabyte store over 6
bricks yet they had incredible performance and reliability
difficulties. I'm talking about a mission critical system being
unavailable for weeks at a time. At least it wasn't customer
facing (there was another set of servers for that).
The system was down more than it was up. Reading was generally
OK (but very slow) but multiple threads writing caused mayhem -
I'm talking lost files and file system accesses going into the
multiple minutes.
In the end I implemented a 1-Tb store to be fuse-unioned over the top
of the thing to take the impact of multiple threads writing to it. A
single thread (overnight) brought the underlying glusterfs up to date.
That got us more or less running but the darned thing spent most of
its time re-indexing and balancing rather than serving files.
To be fair, some of the problems were undoubtedly of their own making
as 2 nodes were centos and 4 were fedora-12 - apparently the engineer
couldn't find the installation CD for the 2 new nodes and 'made do'
with what he had! I recall that a difference in the system 'sort'
command gave all sorts of grief until it was discovered, never mind
different versions of the gluster drivers.
I'd endorse Johnny's comments about it not handling large numbers of
small files well (ie <~ 10 Mb). I believe it was designed for large
multi-media files such as clinical X-Rays. ie a small number of large
files.
Another factor is that the available space is the physical space
divided by 4 due to the replication across the nodes on top of the
nodes being RAID'd themselves.
Lesse now - that was all of 6 months ago - unlike most of my war
stories, it's not ancient history!!
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos
