hi Pat, It seems that brick is disconnecting frequently: pranithk @ ~/Downloads/pat-mit/old 14:48:39 :) $ grep -w disconnected gdata.log | awk '{print $5, $6}' | sort | uniq -c 5 0-gdata-client-0: disconnected 6 0-gdata-client-1: disconnected 308 0-gdata-client-2: disconnected <<----- It disconnected 308 times where as the others < 10 2 gdata-client-0: disconnected 1 gdata-client-1: disconnected 8 gdata-client-2: disconnected You were mentioning that you tried doing self-heal, but I dont see any replicate configuration in your setup. It is a plain distribute setup. Am I missing something?. This is the file I looked into http://mseas.mit.edu/download/phaley/GlusterUsers/Old/client_logs/gdata.log Pranith. On 01/30/2012 11:47 PM, Pat Haley wrote: > > Hi, > > We recently upgraded our version of gluster from 3.1.4 to > 3.2.5. Shortly after I noticed that gluster was not serving > up all of its files, that is if I log onto one of the individual > bricks and do an ls of the underlying nfs directories, I can > see files that I do not see if do an ls from a client node > in the equivalent gluster directory. > > We may have caused this by also running a script that went > through and removed pointer files (this had been how we had > been dealing with bad pointers under gluster 3.1.4). > > So far, we have tried > - running a gluster self-heal under version 3.2.5 > - rolling back to version 3.1.4 > - running a gluster self-heal under version 3.1.4 > - running a rebalance under version 3.1.4 > None of these have solved the problem. The one other > piece of data we have is that all the files which are > not being served appear to reside on a single brick > (at least every file we missed so far has been on > that server and we have not found one missing from > the other servers). > > Any advice you can give us would be greatly appreciated. > > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- > Pat Haley Email: phaley at mit.edu > Center for Ocean Engineering Phone: (617) 253-6824 > Dept. of Mechanical Engineering Fax: (617) 253-8125 > MIT, Room 5-222B http://web.mit.edu/phaley/www/ > 77 Massachusetts Avenue > Cambridge, MA 02139-4301 > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users