FYI, I tracked this down to the cluster/ha translator. If I remove it,
the problem goes away; if I add it back, the problem comes back.
Thanks,
Brent
On Wed, 24 Jun 2009, Brent A Nelson wrote:
At least with 2.0.2, my GlusterFS filesystems
(io-threads=>distribute=>replicate=>ha=>client=>server=>io-threads=>posix-locks=>posix)
seem to have a serious problem with file renaming:
root@node1:/data# mkdir test
root@node1:/data# cd test
root@node1:/data/test# cp /bin/ls .
root@node1:/data/test# mv ls ls.bak
mv: cannot move `ls' to a subdirectory of itself, `ls.bak'
root@node1:/data/test# ls -al
total 240
drwxr-xr-x 2 root root 32768 2009-06-24 10:45 .
drwxr-xr-x 4 root root 32768 2009-06-24 10:45 ..
-rwxr-xr-x 1 root root 106176 2009-06-24 10:45 ls
root@node1:/data/test# cp ls ls.bak2
root@node1:/data/test# ls -al
total 352
drwxr-xr-x 2 root root 32768 2009-06-24 10:46 .
drwxr-xr-x 4 root root 32768 2009-06-24 10:45 ..
-rwxr-xr-x 1 root root 106176 2009-06-24 10:45 ls
-rwxr-xr-x 1 root root 106176 2009-06-24 10:46 ls.bak2
This leaves residue on the underlying shares (eonly odd nodes are listed,
as the even ones are replicas and show the same contents):
root@node1:/data/test# ls -la /disk?/glusterfs/test
/disk3/glusterfs/test:
total 128
drwxr-xr-x 2 root root 4096 2009-06-24 10:46 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..
-rwxr-xr-x 1 root root 106176 2009-06-24 10:46 ls.bak2
/disk4/glusterfs/test:
total 16
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..
root@node3:~# ls -la /disk?/glusterfs/test
/disk3/glusterfs/test:
total 20
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..
---------T 1 root root 0 2009-06-24 10:45 ls
/disk4/glusterfs/test:
total 16
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..
root@node5:~# ls -la /disk?/glusterfs/test
/disk3/glusterfs/test:
total 16
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..
/disk4/glusterfs/test:
total 128
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..
-rwxr-xr-x 1 root root 106176 2009-06-24 10:45 ls
root@node7:~# ls -la /disk?/glusterfs/test
/disk3/glusterfs/test:
total 16
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..
/disk4/glusterfs/test:
total 16
drwxr-xr-x 2 root root 4096 2009-06-24 10:45 .
drwxr-xr-x 4 root root 4096 2009-06-24 10:45 ..
The file I tried to rename left an extra, empty version of the file with a+T
permissions, in addition to the file itself. When I end up with these a+T
files, GlusterFS does not show them, but operations can indicate their
presence (e.g., you can't delete a directory containing these files from
GlusterFS, as GlusterFS won't delete the file, and then the directory is not
empty and can't be unlinked).
I know I have seen these a+T files in 2.0.1 and earlier, as well (I believe I
mentioned them when I was doing NFS testing).
Thanks,
Brent
