2 server using afr (client side afr), 1 client 1) 10 file[0-9].tar.bz2 @ 45M on /mnt/raid/gfs on server1 2) scp file* on server1 to /mnt/raid/gfs on server2 3) on server1 find /mnt/raid/gfs -exec setfattr trusted.glusterfs.version -v 1 {} \; 4) on server1 find /mnt/raid/gfs -exec setfattr trusted.glusterfs.created -v 1210048518 {} \; 5) on server2 find /mnt/raid/gfs -exec setfattr trusted.glusterfs.version -v 1 {} \; 6) on server2 find /mnt/raid/gfs -exec setfattr trusted.glusterfs.createtime -v 1210048518 {} \; Ok so at this point i have two copies of identical data on both servers, and i manually set both of the xattr on all files on BOTH servers to be the same. 7) start glusterfsd on server1, but NOT on server2 8) on client mount /mnt/gfs 9) cd /mnt/gfs 10) > file0.tar.bz2 (truncate? set to null?) 11) echo "" > file9.tar.bz2 Ok here I > file0 and echo a blank string to file9. When I perform ls -l on file0.tar.bz2 is shows 0, on file9.tar.bz2 it shows 1. Remember that at this point over on server2 both of these files are still 45 megs each. 12) start glusterfsd on server2 13) head -c 1 file9.tar.bz2, the 1 byte? file is copied to server1 14) head -c 1 file file1.tar.bz2, it does not update server2! 14) cat file0.tar.bz2, it does not update server2! Apparently if you do "> filename" the afr translator does not understand this? 15) echo "" >> file0.tar.bz2, server1 updates with 1 byte, server2 still says 45 megs! 16) ls -al on client, now file0.tar.bz2 on the client shows as 45 megs, server1 reports 1 byte, server2 still says 45 megs 17) echo "" > file0.tar.bz2, all is well, both servers now have a 1 byte file