On 01/28/2011 12:49 PM, Steve Wilson wrote: > I'm running a pair of replicated/distributed GlusterFS 3.1.2 servers, > each with 8 bricks. Here's the command I used to create the data volume: > > gluster volume create post replica 2 transport tcp > pablo:/gluster/01/brick stanley:/gluster/01/brick \ > pablo:/gluster/02/brick stanley:/gluster/02/brick \ > pablo:/gluster/03/brick stanley:/gluster/03/brick \ > pablo:/gluster/04/brick stanley:/gluster/04/brick \ > pablo:/gluster/05/brick stanley:/gluster/05/brick \ > pablo:/gluster/06/brick stanley:/gluster/06/brick \ > pablo:/gluster/07/brick stanley:/gluster/07/brick \ > pablo:/gluster/08/brick stanley:/gluster/08/brick > > The clients mount the volume using the gluster FUSE client, again > version 3.1.2. Since GlusterFS doesn't yet support exporting > sub-directories directly (it does now for NFS, though, I believe), I > mount the volume as: > /net/.post-volume > and then bind mount a couple of lower sub-directories as: > /net/post (bound to /net/.post-volume/home) > /net/post-scratch (bound to /net/.post-volume/scratch) > > I am using this volume for user home directories. > > That's the background information... now for the problem. There are > certain files that are almost guaranteed to go missing after less than > a few hours of a user starting up a login session. We use Ubuntu with > GDM as a login greeter and both KDE and Gnome desktops. Here are a > few of the files that usually go missing: > > ls: .dmrc: No such file or directory > ls: .pine-debug2: Invalid argument > ls: .xsession-errors.old: No such file or directory > ls: .recently-used.xbel: No such file or directory > > Taking .dmrc, for example, I can stop the volume, then remove all > occurrences of this file on all the bricks on both servers. Then > start up the volume again, start the user session again, and after a > period of time (I'm not sure how long), the file comes up missing > again. When the file is missing, this is what it looks like on the > bricks themselves: > > server 1: > > root at stanley:/gluster# ls -l ./07/brick/home/stevew/.dmrc > ---------T 1 stevew sysmgr 0 2011-01-28 08:24 > ./07/brick/home/stevew/.dmrc > root at stanley:/gluster# ls -l ./02/brick/home/stevew/.dmrc > -rw-r--r-- 1 stevew sysmgr 25 2011-01-28 08:25 > ./02/brick/home/stevew/.dmrc > > server 2: > > root at pablo:/gluster# ls -l ./07/brick/home/stevew/.dmrc > ---------T 1 stevew sysmgr 0 2011-01-28 08:24 > ./07/brick/home/stevew/.dmrc > root at pablo:/gluster# ls -l ./02/brick/home/stevew/.dmrc > -rw-r--r-- 1 stevew sysmgr 25 2011-01-28 08:25 > ./02/brick/home/stevew/.dmrc > > > Pertinent log messages are: > > server 1: > > net-.post-volume.log:[2011-01-28 08:16:28.899544] W > [fuse-bridge.c:2432:fuse_xattr_cbk] glusterfs-fuse: 6318974: > GETXATTR() /home/stevew/.dmrc => -1 (Invalid argument) > > > server 2: > > net-.post-volume.log:[2011-01-28 08:10:58.74848] W > [fuse-bridge.c:2432:fuse_xattr_cbk] glusterfs-fuse: 2428788: > GETXATTR() /home/stevew/.dmrc => -1 (No such file or directory) > > > Any thoughts or ideas? > > Thanks! > > Steve > > Yesterday I disabled the performance.stat-prefetch volume option and this has improved the situation greatly. I can now do a "ls -alR > /dev/null" and get no errors on the mounted volume. Steve
