Hi all - We're running a 3-node distributed volume, using gluster 3.3.1. We're seeing a rare but repeated issue, where files are written to the volume, appears to be written OK, but are not accessible via NFS or the gluster client. These files appear when we inspect the bricks directly. For example, the path in question is /site/927/Volumes/zero/job/ZY/client_projects/site/58/project. Via NFS: $ ls /projects/site/927/Volumes/zero/job/ZY/client_projects/site/58/project/ ls: cannot access /projects/site/927/Volumes/zero/job/ZY/client_projects/site/58/project/: Invalid argument $ ls /projects/site/927/Volumes/zero/job/ZY/client_projects/site/58 ls: cannot access /projects/site/927/Volumes/zero/job/ZY/client_projects/site/58/project: Invalid argument project $ ls /projects/site/927/Volumes/zero/job/ZY/client_projects/site/58/project/ workspace.mel The directory doesn't show up at all, then eventually displays workspace.mel. On gluster01, via gluster client: $ ls /projects/site/927/Volumes/zero/job/ZY/client_projects/site/58/project/ workspace.mel Many more files were written to that directory. When I inspect the bricks I see them (brick0002 is one of the bricks in this volume): $ ls /gluster/brick0002/site/927/Volumes/zero/job/ZY/client_projects/site/58/project/sourceimages/ ball environment lights paper_package.jpg pinbal_upper.jpg room_v317projection_capture scratches This has happened for a few (2-5) directories, but most directories are working fine. This is a fairly high-activity volume with thousands of written files per day, so the rate of incidence is not very high, but still concerning. Any idea what's going on, any suggestions? Any more info I can provide? Thanks, -- Brian Cipriano ZERO VFX Head of Technology bcipriano at zerovfx.com
