Just set up and mounted a glusterfs consisting of 6 nodes:
$ gluster volume info
Volume Name: g6
Type: Distribute
Status: Started
Number of Bricks: 6
Transport-type: tcp
Bricks:
Brick1: pbs1:/data2
Brick2: pbs2:/data2
Brick3: pbs3:/data2
Brick4: pbs3:/data
Brick5: dabrick:/data2
Brick6: hef:/data2
Options Reconfigured:
auth.allow: 128.*
It's mounted on 9 similar (tho not identical) ubuntu 10.04.3 nodes
(64b Kubuntu Dektop) in a cluster (claw[1-9]), which are running a
self-compiled and installed version of 3.3b1. The gluster server
bricks are also running 10.04.3 (64b server).
However, the glusterfs mounts differentially on the nodes:
root at bduc-login:~
576 $ for ITER in $(seq 1 9); do echo -n "claw${ITER}:"; ssh
claw${ITER} 'df -h |egrep "gluz|pbs"' ; done;
claw1:pbs3:/g6 21T 114G 21T 1% /mnt/gluz
claw2:pbs3.calit2.uci.edu:/g6
21T 114G 21T 1% /mnt/gluz
claw3:pbs3.calit2.uci.edu:/g6
21T 114G 21T 1% /mnt/gluz
claw4:pbs3.calit2.uci.edu:/g6
21T 114G 21T 1% /mnt/gluz
claw5:pbs3.calit2.uci.edu:/g6
5.5T 2.7G 5.5T 1% /mnt/gluz <---
claw6:pbs3.calit2.uci.edu:/g6
21T 112G 21T 1% /mnt/gluz
claw7:pbs3.calit2.uci.edu:/g6
21T 114G 21T 1% /mnt/gluz
claw8:pbs3.calit2.uci.edu:/g6
5.5T 2.7G 5.5T 1% /mnt/gluz <---
claw9:pbs3.calit2.uci.edu:/g6
21T 112G 21T 1% /mnt/gluz
Obviously claw5 and claw8 are ... different. It looks like they're
mounting a component of the glusterfs (from the numbers, it's the
contribution of "Brick5: dabrick:/data2"), but not the whole thing.
The glusterfs was mounted on all of them with a script:
mount -t glusterfs pbs3.calit2.uci.edu:/g6 /mnt/gluz
Ive unmounted and remounted the glusterfs on claw8 with no difference.
I've also unmounted all instances of the glusterfs, then gluster-
stopped and restarted the volume on the gluster server node and then
remounted the fs as well. No diff.
When mounted, claw[58] can see some of the files not all of them (the
same ones that are on the 'dabrick' volume, as you might guess) but
cannot write to the mounted filesystem, even tho the user has
permissions to write.
...?!
The nodes that are having the problem have never had a gluster fs
mounted on them before.
--
Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine
[ZOT 2225] / 92697 Google Voice Multiplexer: (949) 478-4487
MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)
--
This signature has been OCCUPIED!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gluster.org/pipermail/gluster-users/attachments/20111028/d589fa81/attachment.htm>
