This doesn't seem like a good way to do what I think you want to do. 1 - /scratch should be as fast as possible, so putting it on a distributed fs, unless that fs is optimized for speed (mumble.Lustre.mumble), is a mistake. 2 - if you insist on doing this with gluster (perhaps because none of your individual /scratch partitions is large enough), making a dist & replicated /scratch is making a bad decision worse as replication will slow the process down even more. (Why replicate what is a temp data store?) 3 - integrating the gluster server into the rocks environment (on a per-node basis) seems like a recipe for .. well, migraines, at least If you need a relatively fast, simple, large, reliable, aggressively caching fs for /scratch, NFS to a large RAID0/10 has some attractions, unless the gluster server fanout IO overwhelms the aforementioned attractions. IMHO... On Thursday, November 15, 2012 09:30:52 AM Jerome wrote: > Dear all > > I'm testing Gluster in a cluster of compute nodes, based on Rocks. The > idea is to use the scratch of each nodes as a big volume for scratch. It > permit to access to this scratch system file on all the nodes of the > cluster. > For the moment, i have installed this gluster system on 4 nodes, ona > distributed replica of 2, like this: > > # gluster volume info > > Volume Name: scratch1 > Type: Distributed-Replicate > Volume ID: c8c3e3fe-c785-4438-86eb-0b84c7c29123 > Status: Started > Number of Bricks: 2 x 2 = 4 > Transport-type: tcp > Bricks: > Brick1: compute-2-0:/state/partition1/scratch > Brick2: compute-2-1:/state/partition1/scratch > Brick3: compute-2-6:/state/partition1/scratch > Brick4: compute-2-9:/state/partition1/scratch > > # gluster volume status > Status of volume: scratch1 > Gluster process Port Online Pid > ---------------------------------------------------------------------------- > -- Brick compute-2-0:/state/partition1/scratch 24009 Y 16464 > Brick compute-2-1:/state/partition1/scratch 24009 Y 3848 > Brick compute-2-6:/state/partition1/scratch 24009 Y 511 > Brick compute-2-9:/state/partition1/scratch 24009 Y 2086 > NFS Server on localhost 38467 N 4060 > Self-heal Daemon on localhost N/A N 4065 > NFS Server on compute-2-0 38467 Y 16470 > Self-heal Daemon on compute-2-0 N/A Y 16476 > NFS Server on compute-2-9 38467 Y 2092 > Self-heal Daemon on compute-2-9 N/A Y 2099 > NFS Server on compute-2-6 38467 Y 517 > Self-heal Daemon on compute-2-6 N/A Y 524 > NFS Server on compute-2-1 38467 Y 3854 > Self-heal Daemon on compute-2-1 N/A Y 3860 > > > All of this run correctly, i used some stress file to advise that > configuration could be runnable. > My problem is when a node reboot accidentaly, or for some administration > task: the node reinstall itself, and the gluster volume begin to > fail.... I detect taht the UUID of a machine is generated during the > instalation, so i develop some script to get back the original UUID of > the node. Despote this, the node could not get back in the volume. I > miss some special task to do. So, it is possible to do a such system > with gluster? or maybe i have to reconfigure all of the voluem when a > node reinstall? > > Best regards. -- Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine [m/c 2225] / 92697 Google Voice Multiplexer: (949) 478-4487 415 South Circle View Dr, Irvine, CA, 92697 [shipping] MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps) -- Passive-Aggressive Supporter of the The Canada Party: <http://www.americabutbetter.com/>
