Further insight: the initial mtimes are correct (and probably are for
rsync, as well); the mtimes are getting modified by self-heal. I'm not
sure why self-heal did anything at all, as the mirrors should have been in
sync, but it triggers on ls and quickly changes the mtimes on the second
half of the mirror.
Thanks,
Brent
On Wed, 13 May 2009, Brent A Nelson wrote:
I take that back; it doesn't seem to help, at least for the initial rsync.
Shouldn't it, though? If a file on 1/2 of the mirror has a different mtime
than the file on the other half, shouldn't self-heal fix it?
The early indication was that it did seem to help in the case that I manually
removed a file from the second half of the mirror. With the
metadata-change-log option on, the file self-healed with the correct mtime;
without the metadata-change-log option, it did not. I only tried it once,
though, so it might have just been coincidence.
I also saw the mtime issue with "cp -a". It appeared to occur in one brief
burst, and this burst spanned multiple mirrors (a few files created at
4:46pm, all in the same specific directory, for two different mirrors and
different server nodes, have bad mtimes on the second half of each mirror).
This was far less common than rsync, however.
Thanks,
Brent
On Wed, 13 May 2009, Brent A Nelson wrote:
Early indications are that setting "option metadata-change-log on" for
cluster/replicate is a likely workaround for the mtime issue. I'll start
over, and see if the issue is truly gone with this option in place...
It might be worth considering defaulting this option to "on".
Thanks,
Brent
On Wed, 13 May 2009, Brent A Nelson wrote:
On Wed, 13 May 2009, Brent A Nelson wrote:
With regards to the incorrect modification time appearing on some files,
I note the following:
ls -l on Node1 in a mirror:
-r--r--r-- 1 root root 40280 2008-03-17 12:03
/disk1/glusterfs/tftpboot/hardy64/pool/main/t/tasksel/tasksel-data_2.70ubuntu4_all.deb
Node 2 in the same mirror:
-r--r--r-- 1 root root 40280 2009-05-12 19:40
/disk1/glusterfs/tftpboot/hardy64/pool/main/t/tasksel/tasksel-data_2.70ubuntu4_all.deb
It appears that 1/2 of the mirror set the modification time correctly,
but the other half did not.
Just a bit of additional info. It appears that the first half of the
mirror has correct mtimes. The second half of the mirror has wrong mtimes
on all files, but directory mtimes are fine. When you go to view the
mtimes on the GlusterFS, sometimes you will get the mtime from one node,
sometimes the other, hence the seeming randomness.
Also, I see that zero-length files have correct mtimes on both halves of a
mirror.
Thanks,
Brent
