I'm now running gluster 3.1.1 on Debian. A directory that was running under 3.0.4 had duplicate files, but I've remounted things now that we're running 3.1.1 in hopes it would fix things, but so far it has not: # ls -l /mnt/glusterfs/www/0/0descriptionofta581unittotal 37992 -rwxr-xr-x 1 www-data www-data 796343 Jun 23 2010 0descriptionofta581unit_bw.pdf -rwxr-xr-x 1 www-data www-data 796343 Jun 23 2010 0descriptionofta581unit_bw.pdf ---------T 1 root root 1497 Jun 24 2010 0descriptionofta581unit_dc.xml ---------T 1 root root 1497 Jun 24 2010 0descriptionofta581unit_dc.xml ---------T 1 www-data www-data 577050 Jun 24 2010 0descriptionofta581unit.djvu ---------T 1 www-data www-data 577050 Jun 24 2010 0descriptionofta581unit.djvu -rwxr-xr-x 1 www-data www-data 33272 Jun 22 2010 0descriptionofta581unit_djvu.txt -rwxr-xr-x 1 www-data www-data 33272 Jun 22 2010 0descriptionofta581unit_djvu.txt -rwxr-xr-x 1 www-data www-data 4445 Jun 23 2010 0descriptionofta581unit_files.xml -rwxr-xr-x 1 www-data www-data 4445 Jun 23 2010 0descriptionofta581unit_files.xml -rwxr-xr-x 1 www-data www-data 5011 Jun 22 2010 0descriptionofta581unit_marc.xml -rwxr-xr-x 1 www-data www-data 5011 Jun 22 2010 0descriptionofta581unit_marc.xml -rwxr-xr-x 1 www-data www-data 360 Jun 23 2010 0descriptionofta581unit_metasource.xml -rwxr-xr-x 1 www-data www-data 360 Jun 23 2010 0descriptionofta581unit_metasource.xml -rwxr-xr-x 1 www-data www-data 2848 Jun 22 2010 0descriptionofta581unit_meta.xml -rwxr-xr-x 1 www-data www-data 2848 Jun 22 2010 0descriptionofta581unit_meta.xml -rwxr-xr-x 1 www-data www-data 16916480 Jun 22 2010 0descriptionofta581unit_orig_jp2.tar -rwxr-xr-x 1 www-data www-data 16916480 Jun 22 2010 0descriptionofta581unit_orig_jp2.tar -rwxr-xr-x 1 www-data www-data 1051810 Jun 22 2010 0descriptionofta581unit.pdf -rwxr-xr-x 1 www-data www-data 1051810 Jun 22 2010 0descriptionofta581unit.pdf While running the latest, 3.1.1, I noticed some log files that said: [..] [2011-01-12 15:24:33.325546] I [afr-common.c:613:afr_lookup_self_heal_check] bhl-volume-replicate-69: size differs for /www/0/0descriptionofta581unit/0descriptionofta581unit.djvu [2011-01-12 15:24:33.325558] I [afr-common.c:716:afr_lookup_done] bhl-volume-replicate-69: background meta-data data self-heal triggered. path: /www/0/0descriptionofta581unit/0descriptionofta581unit.djvu [2011-01-12 15:24:33.364501] I [afr-self-heal-common.c:1526:afr_self_heal_completion_cbk] bhl-volume-replicate-66: background meta-data data self-heal completed on /www/0/0descriptionofta581unit/0descriptionofta581unit.djvu [2011-01-12 15:24:33.364881] I [afr-self-heal-common.c:1526:afr_self_heal_completion_cbk] bhl-volume-replicate-69: background meta-data data self-heal completed on /www/0/0descriptionofta581unit/0descriptionofta581unit.djvu I assumed it was fixing that, but it didn't. Here's the full logs that include all the gluster.log work it did in this directory: http://pastebin.com/8X52Em7Y Question: how can I 'fix' this, or is the best bet to remove everything and start over? It's going to set us back, but I'd rather do it now that keep banging on this without any resolution. Thanks for the help, really like the new gluster command, very nice! P -- http://philcryer.com