Thanks to Patrick Ruckstuhl for the "me too" response. Surely this must be a problem for other gluster users too. I didn't bother trying to find Patrick's past messages since I vaguely recall reading them previously and the gluster mailing list archive is not easily searchable as far as I know. Anyway, what Patrick said below was enough for me to come up with a procedure for getting us out of this mess. I wrote a little script to walk through the directory trees on the various local filesystems that comprise the nufa volume, find files that do not exist in the nufa volume, and run a mv command for each of those files. As Patrick said, for each of these mv commands I got a "file exists" response and then the file was visible in the gluster nufa volume. Other than what I've done with this script, is there any more automated way of keeping a gluster nufa volume from getting into this state or repairing it when it does? I assume that the only way this can happen is when files are copied to a subvolume filesystem instead of to the nufa volume. Is that correct? Patrick, does that agree with your experience? -- Todd Pfaff <pfaff at mcmaster.ca> http://www.rhpcs.mcmaster.ca/ On Tue, 19 May 2009, Patrick Ruckstuhl wrote: > Hi, > >> We're using gluster-2.0.1 and a nufa volume comprised of thirteen >> subvolumes across thirteen hosts. >> >> We've found today that there are some files in the local filesystem >> associated with the subvolume from one of the hosts that are not >> being seen in the nufa volume on any gluster client. >> >> I don't know how or when this happened, but now we have to do some >> work to get this gluster volume back into a sane state. >> >> Is there a recommended procedure for fixing this particular problem? >> That is, making all files that are found in all subvolume filesystems >> be seen within the gluster nufa volume? > > That looks exactly like the same problem I have. In my case it results > in a "Can not move, file exists" > > (See my earlier report for details including config and setup) > > > Gruss > Patrick > >