Hi everyone, I just wanted to give you a little feedback on mainline--2.5. I am using my 3 test servers, and will soon add 2 more. The main problem I am experiencing is : self-heal repairs directory inconsitencies but not file inconcistencies Here is the current state: tbs-lab1 (client) tbs-lab2 (brick1) tbs-lab3 (brick2) on the client, afr = brick1 + brick2. for the test, brick1 has whole data and brick2 is blanked (for now). brick1 and brick2 are both on a ext3 FS with extended attributes : /dev/sda2 /glusterfs ext3 defaults,user_xattr 0 2 I am currently using glusterfs--mainline--2.5--patch-220 on each machine. client : CFLAGS="-O3" ./configure --prefix=/usr/local --sysconfdir=/etc --disable-server --disable-ibverbs servers : CFLAGS="-O3" ./configure --prefix=/usr/local --sysconfdir=/etc --disable-fuse-client --disable-ibverbs here is the servers spec file (X is the number of the brick, so 1 or 2, here) : volume brickX type storage/posix option directory /glusterfs end-volume volume locksX type features/posix-locks subvolumes brickX end-volume volume serverX type protocol/server option transport-type tcp/server subvolumes locksX option auth.ip.locksX.allow 192.168.0.5 # client IP end-volume volume traceX type debug/trace subvolumes serverX option debug on end-volume launching both bricks with this command : glusterfsd -f /etc/glusterfs/glusterfs-server.vol --log-file=/var/log/glusterfs/glusterfsd.log --log-level=DEBUG Now here is the client spec file : volume brick1c type protocol/client option transport-type tcp/client option remote-host 192.168.0.6 # brick1 IP option remote-subvolume locks1 end-volume volume brick2c type protocol/client option transport-type tcp/client option remote-host 192.168.0.7 # brick2 IP option remote-subvolume locks2 end-volume volume afr type cluster/afr subvolumes brick1c brick2c option replicate *:2 end-volume volume writebehind type performance/write-behind option aggregate-size 131072 subvolumes afr end-volume And I mounting the volume as such : glusterfs -f /etc/glusterfs/glusterfs-client.vol -l /var/log/glusterfs/glusterfs.log /mnt/glusterfs --log-level=DEBUG DEBUG is here to give as much information as needed to correct the issue. So, at this state, we have : brick1:~# ls -l /glusterfs total 4 drwxr-xr-x 2 www-data www-data 4096 2007-05-11 11:27 apache2-default and : brick2:~# ls -l /glusterfs total 0 If I do a 'ls -l' on the client, it instantaneously creates apache2-default directory on brick2 (but not subdirs, which haven't been accessed yet) Issue is here : let's try to access a file from the client, let's say : cp /mnt/glusterfs/apache2-default/apache_pb2_ani.gif /dev/null brick1:~# ls -l /glusterfs total 4 drwxr-xr-x 2 www-data www-data 4096 2007-05-11 11:27 apache2-default brick1:~# ls -l /glusterfs/apache2-default/ total 151 -rw-r--r-- 1 www-data www-data 2160 2007-05-11 11:27 apache_pb2_ani.gif -rw-r--r-- 1 www-data www-data 2414 2007-05-11 11:27 apache_pb2.gif .... bla bla bla *snip* ... -rw-r--r-- 1 www-data www-data 26 2007-05-11 11:27 robots.txt brick2:~# ls -l /glusterfs total 4 drwxr-xr-x 2 root root 4096 2007-06-26 10:03 apache2-default brick2:~# ls -l /glusterfs/apache2-default/ total 0 Can you see where I might have forgotten something, or done something wrong ? log files are too big to be put on attachement (limit is 40kB) I can provide them to anyone who wants Cheers, Sebastien LELIEVRE slelievre@xxxxxxxxxxxxxxxx Services to ISP TBS-internet http://www.TBS-internet.com
