On Thu, 15 Nov 2007, Kevan Benson wrote:
Chris Johnson wrote:
Hi again,
Ok, now have CentOS5 on the two DELL front ends to this SATA
Beast thingy. I have gluster enhanced fuse on both and glusterfs
1.3.5 on both.
Performance is still way below NFS. I have turned on
write-behind and read-ahead. And with fuse it's a little faster. but
not significantly. To get more out of this it's pretty ovious I need
to do unify if I want BIG space, maybe striping. If I want high
availability I need AFR (that works with unify?). And if I want to
use the second DELL front end I'm probably going to need locking if
they share drives. Self healing works, right? Wasn't there discussion
about a potential problem not long ago?
Sound about right?
Pretty much. The self heal discussion was about edge cases. It works for
most situations you'll see it in.
AFR and Unify (and striping) are stackable, layer as many as you want (AFR
dual unified AFR's if you want.
I might if I get another one.
I think I'm seeing from the glusterfs wiki that the order
in which things are defined in the config files matters. It's a bit
unclear to me what the real order of things should be.
It's doesn't REALLY matter too much, unless it's a client config and you
aren't explicitly defining the share to load (then it grabs the last
specified).
Most translators specified that rely on other volumes have you specifically
state which volumes they are using. The only way I see order mattering there
is that the subvolumes used by a translator may need to be defined above the
location they are used in a translator. I.e. define share1 and share2 before
you include them in an AFR/Unify. I'm not sure this is required, but it's
probably easier to read the configs if you do it anyways.
About what I thought but the wiki sounded otherwise.
Oh, this is ext3 with xattr turned on. But we could use Reiserfs
if that would help. I may run tests on that as well.
I don't know about others, but I wouldn't use reiserfs if you care about your
data and aren't supplying redundancy above teh file system level (with
glusterfs, for example). I've seen multiple reports of how Reiserfs has a
tendency to have unrecoverable errors in the file system when it gets
sufficiently screwed up, basically necessitating a reformat of the partition.
Huh. Now that's interesting. We've been running our mail server
off Reiserfs for a few years now and never an issue. It's servived
power outs and dropped drives. We're using software RAID on it.
Can we have a discussion on whether I'm heading in the right
direction and what order things go in for the config files?
That depends on your goals. What's important here, speed, redundancy, or a
mix of both?
Both of course. Are the mutually exclusive? Please, I know
they can be to some extent. I'm talking about the real world,
whatever that is. I need to get the performance up,
redundancy/failover would be be real good too. NFS has a few problems
with that.
Also, any operational notes on running gluster on anything this
big will be appreciated.
--
-Kevan Benson
-A-1 Networks
-------------------------------------------------------------------------------
Chris Johnson |Internet: johnson@xxxxxxxxxxxxxxxxxxx
Systems Administrator |Web: http://www.nmr.mgh.harvard.edu/~johnson
NMR Center |Voice: 617.726.0949
Mass. General Hospital |FAX: 617.726.7422
149 (2301) 13th Street |Do not meddle in the affairs of wizards, for
Charlestown, MA., 02129 USA |they are subtle and quick to anger.
-------------------------------------------------------------------------------
