Hi all I'm running some initial sanity and performance checks on this: root at h06 /root # glusterd -V glusterfs 3.4.0beta4 built on Jul 10 2013 15:14:50 The setup is a two node replicating cluster (h06 and h65) the client is on h06. The network is limited to 100Mbits/second and rtt is 0.6ms so performance is not spectacular but that is not the problem which currently concerns me. The test was to take a typical linux rootfs with 52K files of various sizes totaling 1.9GByte and copy it to a gluster mount using tar: root at h06 /root/mnt/h06 # sync;time (tar -c *|(cd /gluster/mnt/20 && tar -x);sync) real 20m24.836s user 0m3.517s sys 0m39.814s Times are not brilliant but it will be OK for the usage scenario. Extracts from 'mount': /dev/mapper/vs-h06 on /root/mnt/h06 type ext4 (rw,relatime,barrier=1,data=ordered) /var/lib/glusterd/vols/gl/gl-fuse.vol on /gluster/mnt type fuse.glusterfs (rw,nosuid,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) Among other tests I wanted to see that the copy in /gluster/mnt/20 was identical to the original in /root/mnt/h06 The original has 5261 symbolic links: root at h06 /root/mnt/h06 # find -type l |wc 5261 5261 182824 The copy has only 4089: root at h65 /gluster/mnt/20 # find -type l |wc 4089 4089 141644 Here is an example: root at h06 /root # ls -ld ~/mnt/h06/bin/netcat lrwxrwxrwx 1 root root 24 Jun 30 09:03 /root/mnt/h06/bin/netcat -> /etc/alternatives/netcat root at h06 /root # ls -ld /gluster/mnt/20/bin/netcat ---------- 1 root root 0 Jul 14 06:37 /gluster/mnt/20/bin/netcat A scan with md5sum on the original and the copy in gluster shows only these links as being different. All normal files checksum the same. The mirrored gluster filesystem on h65 (no surprise) shows the identical result - some symbolic links have been changed in empty files. To the best of my knowledge the gluster filesystems are identical on the two nodes but differ from the original. To me it appears that the command: (cd source && tar -c *)|(cd gluster && tar -x) has changed some symbolic links in 'source' into empty files in 'gluster'. I have used commands like the above tar everyday for years on other filessytems and this is the first time I've come across something like this. Any clues what I've done wrong or where I should look next? All the best Allan
