On Wed, 2014-06-04 at 10:19 +0530, Pranith Kumar Karampuri wrote: > On 06/04/2014 08:07 AM, Susant Palai wrote: > > Pranith can you send the client and bricks logs. > I have the logs. But I believe for this issue of directory not listing > entries, it would help more if we have the contents of that directory on > all the directories in the bricks + their hash values in the xattrs. Strange thing is, all the invisible files are on the one server (nas6), the other seems ok. I did rm -Rf of /data2/franco/dir* and was left with this one directory - there were many hundreds which were removed successfully. I've attached listings and xattr dumps. Cheers, Volume Name: data2 Type: Distribute Volume ID: d958423f-bd25-49f1-81f8-f12e4edc6823 Status: Started Number of Bricks: 8 Transport-type: tcp Bricks: Brick1: nas5-10g:/data17/gvol Brick2: nas5-10g:/data18/gvol Brick3: nas5-10g:/data19/gvol Brick4: nas5-10g:/data20/gvol Brick5: nas6-10g:/data21/gvol Brick6: nas6-10g:/data22/gvol Brick7: nas6-10g:/data23/gvol Brick8: nas6-10g:/data24/gvol Options Reconfigured: nfs.drc: on cluster.min-free-disk: 5% network.frame-timeout: 10800 nfs.export-volumes: on nfs.disable: on cluster.readdir-optimize: on Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick nas5-10g:/data17/gvol 49152 Y 6553 Brick nas5-10g:/data18/gvol 49153 Y 6564 Brick nas5-10g:/data19/gvol 49154 Y 6575 Brick nas5-10g:/data20/gvol 49155 Y 6586 Brick nas6-10g:/data21/gvol 49160 Y 20608 Brick nas6-10g:/data22/gvol 49161 Y 20613 Brick nas6-10g:/data23/gvol 49162 Y 20614 Brick nas6-10g:/data24/gvol 49163 Y 20621 Task Status of Volume data2 ------------------------------------------------------------------------------ There are no active volume tasks > > Pranith > > > > Thanks, > > Susant~ > > > > ----- Original Message ----- > > From: "Pranith Kumar Karampuri" <pkarampu@xxxxxxxxxx> > > To: "Franco Broi" <franco.broi@xxxxxxxxxx> > > Cc: gluster-users@xxxxxxxxxxx, "Raghavendra Gowdappa" <rgowdapp@xxxxxxxxxx>, spalai@xxxxxxxxxx, kdhananj@xxxxxxxxxx, vsomyaju@xxxxxxxxxx, nbalacha@xxxxxxxxxx > > Sent: Wednesday, 4 June, 2014 7:53:41 AM > > Subject: Re: glusterfsd process spinning > > > > hi Franco, > > CC Devs who work on DHT to comment. > > > > Pranith > > > > On 06/04/2014 07:39 AM, Franco Broi wrote: > >> On Wed, 2014-06-04 at 07:28 +0530, Pranith Kumar Karampuri wrote: > >>> Franco, > >>> Thanks for providing the logs. I just copied over the logs to my > >>> machine. Most of the logs I see are related to "No such File or > >>> Directory" I wonder what lead to this. Do you have any idea? > >> No but I'm just looking at my 3.5 Gluster volume and it has a directory > >> that looks empty but can't be deleted. When I look at the directories on > >> the servers there are definitely files in there. > >> > >> [franco@charlie1 franco]$ rmdir /data2/franco/dir1226/dir25 > >> rmdir: failed to remove `/data2/franco/dir1226/dir25': Directory not empty > >> [franco@charlie1 franco]$ ls -la /data2/franco/dir1226/dir25 > >> total 8 > >> drwxrwxr-x 2 franco support 60 May 21 03:58 . > >> drwxrwxr-x 3 franco support 24 Jun 4 09:37 .. > >> > >> [root@nas6 ~]# ls -la /data*/gvol/franco/dir1226/dir25 > >> /data21/gvol/franco/dir1226/dir25: > >> total 2081 > >> drwxrwxr-x 13 1348 200 13 May 21 03:58 . > >> drwxrwxr-x 3 1348 200 3 May 21 03:58 .. > >> drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13017 > >> drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13018 > >> drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13020 > >> drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13021 > >> drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13022 > >> drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13024 > >> drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13027 > >> drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13028 > >> drwxrwxr-x 2 1348 200 2 May 16 12:06 dir13029 > >> drwxrwxr-x 2 1348 200 2 May 16 12:06 dir13031 > >> drwxrwxr-x 2 1348 200 3 May 16 12:06 dir13032 > >> > >> /data22/gvol/franco/dir1226/dir25: > >> total 2084 > >> drwxrwxr-x 13 1348 200 13 May 21 03:58 . > >> drwxrwxr-x 3 1348 200 3 May 21 03:58 .. > >> drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13017 > >> drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13018 > >> drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13020 > >> drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13021 > >> drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13022 > >> ..... > >> > >> Maybe Gluster is losing track of the files?? > >> > >>> Pranith > >>> > >>> On 06/02/2014 02:48 PM, Franco Broi wrote: > >>>> Hi Pranith > >>>> > >>>> Here's a listing of the brick logs, looks very odd especially the size > >>>> of the log for data10. > >>>> > >>>> [root@nas3 bricks]# ls -ltrh > >>>> total 2.6G > >>>> -rw------- 1 root root 381K May 13 12:15 data12-gvol.log-20140511 > >>>> -rw------- 1 root root 430M May 13 12:15 data11-gvol.log-20140511 > >>>> -rw------- 1 root root 328K May 13 12:15 data9-gvol.log-20140511 > >>>> -rw------- 1 root root 2.0M May 13 12:15 data10-gvol.log-20140511 > >>>> -rw------- 1 root root 0 May 18 03:43 data10-gvol.log-20140525 > >>>> -rw------- 1 root root 0 May 18 03:43 data11-gvol.log-20140525 > >>>> -rw------- 1 root root 0 May 18 03:43 data12-gvol.log-20140525 > >>>> -rw------- 1 root root 0 May 18 03:43 data9-gvol.log-20140525 > >>>> -rw------- 1 root root 0 May 25 03:19 data10-gvol.log-20140601 > >>>> -rw------- 1 root root 0 May 25 03:19 data11-gvol.log-20140601 > >>>> -rw------- 1 root root 0 May 25 03:19 data9-gvol.log-20140601 > >>>> -rw------- 1 root root 98M May 26 03:04 data12-gvol.log-20140518 > >>>> -rw------- 1 root root 0 Jun 1 03:37 data10-gvol.log > >>>> -rw------- 1 root root 0 Jun 1 03:37 data11-gvol.log > >>>> -rw------- 1 root root 0 Jun 1 03:37 data12-gvol.log > >>>> -rw------- 1 root root 0 Jun 1 03:37 data9-gvol.log > >>>> -rw------- 1 root root 1.8G Jun 2 16:35 data10-gvol.log-20140518 > >>>> -rw------- 1 root root 279M Jun 2 16:35 data9-gvol.log-20140518 > >>>> -rw------- 1 root root 328K Jun 2 16:35 data12-gvol.log-20140601 > >>>> -rw------- 1 root root 8.3M Jun 2 16:35 data11-gvol.log-20140518 > >>>> > >>>> Too big to post everything. > >>>> > >>>> Cheers, > >>>> > >>>> On Sun, 2014-06-01 at 22:00 -0400, Pranith Kumar Karampuri wrote: > >>>>> ----- Original Message ----- > >>>>>> From: "Pranith Kumar Karampuri" <pkarampu@xxxxxxxxxx> > >>>>>> To: "Franco Broi" <franco.broi@xxxxxxxxxx> > >>>>>> Cc: gluster-users@xxxxxxxxxxx > >>>>>> Sent: Monday, June 2, 2014 7:01:34 AM > >>>>>> Subject: Re: glusterfsd process spinning > >>>>>> > >>>>>> > >>>>>> > >>>>>> ----- Original Message ----- > >>>>>>> From: "Franco Broi" <franco.broi@xxxxxxxxxx> > >>>>>>> To: "Pranith Kumar Karampuri" <pkarampu@xxxxxxxxxx> > >>>>>>> Cc: gluster-users@xxxxxxxxxxx > >>>>>>> Sent: Sunday, June 1, 2014 10:53:51 AM > >>>>>>> Subject: Re: glusterfsd process spinning > >>>>>>> > >>>>>>> > >>>>>>> The volume is almost completely idle now and the CPU for the brick > >>>>>>> process has returned to normal. I've included the profile and I think it > >>>>>>> shows the latency for the bad brick (data12) is unusually high, probably > >>>>>>> indicating the filesystem is at fault after all?? > >>>>>> I am not sure if we can believe the outputs now that you say the brick > >>>>>> returned to normal. Next time it is acting up, do the same procedure and > >>>>>> post the result. > >>>>> On second thought may be its not a bad idea to inspect the log files of the bricks in nas3. Could you post them. > >>>>> > >>>>> Pranith > >>>>> > >>>>>> Pranith > >>>>>>> On Sun, 2014-06-01 at 01:01 -0400, Pranith Kumar Karampuri wrote: > >>>>>>>> Franco, > >>>>>>>> Could you do the following to get more information: > >>>>>>>> > >>>>>>>> "gluster volume profile <volname> start" > >>>>>>>> > >>>>>>>> Wait for some time, this will start gathering what operations are coming > >>>>>>>> to > >>>>>>>> all the bricks" > >>>>>>>> Now execute "gluster volume profile <volname> info" > > >>>>>>>> /file/you/should/reply/to/this/mail/with > >>>>>>>> > >>>>>>>> Then execute: > >>>>>>>> gluster volume profile <volname> stop > >>>>>>>> > >>>>>>>> Lets see if this throws any light on the problem at hand > >>>>>>>> > >>>>>>>> Pranith > >>>>>>>> ----- Original Message ----- > >>>>>>>>> From: "Franco Broi" <franco.broi@xxxxxxxxxx> > >>>>>>>>> To: gluster-users@xxxxxxxxxxx > >>>>>>>>> Sent: Sunday, June 1, 2014 9:02:48 AM > >>>>>>>>> Subject: glusterfsd process spinning > >>>>>>>>> > >>>>>>>>> Hi > >>>>>>>>> > >>>>>>>>> I've been suffering from continual problems with my gluster filesystem > >>>>>>>>> slowing down due to what I thought was congestion on a single brick > >>>>>>>>> being caused by a problem with the underlying filesystem running slow > >>>>>>>>> but I've just noticed that the glusterfsd process for that particular > >>>>>>>>> brick is running at 100%+, even when the filesystem is almost idle. > >>>>>>>>> > >>>>>>>>> I've done a couple of straces of the brick and another on the same > >>>>>>>>> server, does the high number of futex errors give any clues as to what > >>>>>>>>> might be wrong? > >>>>>>>>> > >>>>>>>>> % time seconds usecs/call calls errors syscall > >>>>>>>>> ------ ----------- ----------- --------- --------- ---------------- > >>>>>>>>> 45.58 0.027554 0 191665 20772 futex > >>>>>>>>> 28.26 0.017084 0 137133 readv > >>>>>>>>> 26.04 0.015743 0 66259 epoll_wait > >>>>>>>>> 0.13 0.000077 3 23 writev > >>>>>>>>> 0.00 0.000000 0 1 epoll_ctl > >>>>>>>>> ------ ----------- ----------- --------- --------- ---------------- > >>>>>>>>> 100.00 0.060458 395081 20772 total > >>>>>>>>> > >>>>>>>>> % time seconds usecs/call calls errors syscall > >>>>>>>>> ------ ----------- ----------- --------- --------- ---------------- > >>>>>>>>> 99.25 0.334020 133 2516 epoll_wait > >>>>>>>>> 0.40 0.001347 0 4090 26 futex > >>>>>>>>> 0.35 0.001192 0 5064 readv > >>>>>>>>> 0.00 0.000000 0 20 writev > >>>>>>>>> ------ ----------- ----------- --------- --------- ---------------- > >>>>>>>>> 100.00 0.336559 11690 26 total > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Cheers, > >>>>>>>>> > >>>>>>>>> _______________________________________________ > >>>>>>>>> Gluster-users mailing list > >>>>>>>>> Gluster-users@xxxxxxxxxxx > >>>>>>>>> http://supercolony.gluster.org/mailman/listinfo/gluster-users > >>>>>>>>> >
/data17/gvol/franco/dir1226/dir25: total 2051 drwxrwxr-x 2 1348 200 2 May 21 03:58 . drwxrwxr-x 3 1348 200 3 Jun 4 09:37 .. /data18/gvol/franco/dir1226/dir25: total 2048 drwxrwxr-x 2 1348 200 2 May 21 03:58 . drwxrwxr-x 3 1348 200 3 May 21 03:58 .. /data19/gvol/franco/dir1226/dir25: total 2053 drwxrwxr-x 2 1348 200 2 May 21 03:58 . drwxrwxr-x 3 1348 200 3 May 21 03:58 .. /data20/gvol/franco/dir1226/dir25: total 2048 drwxrwxr-x 2 1348 200 2 May 21 03:58 . drwxrwxr-x 3 1348 200 3 May 21 03:58 .. # file: data17/gvol/franco/dir1226/dir25 trusted.gfid=0s9OYWmrHlQlu0O3bUvCfVKA== trusted.glusterfs.dht=0sAAAAAQAAAADf///5/////w== # file: data18/gvol/franco/dir1226/dir25 trusted.gfid=0s9OYWmrHlQlu0O3bUvCfVKA== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAH////g== # file: data19/gvol/franco/dir1226/dir25 trusted.gfid=0s9OYWmrHlQlu0O3bUvCfVKA== trusted.glusterfs.dht=0sAAAAAQAAAAAf////P////Q== # file: data20/gvol/franco/dir1226/dir25 trusted.gfid=0s9OYWmrHlQlu0O3bUvCfVKA== trusted.glusterfs.dht=0sAAAAAQAAAAA////+X////A==
/data21/gvol/franco/dir1226/dir25: total 2081 drwxrwxr-x 13 1348 200 13 May 21 03:58 . drwxrwxr-x 3 1348 200 3 May 21 03:58 .. drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13017 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13018 drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13020 drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13021 drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13022 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13024 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13027 drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13028 drwxrwxr-x 2 1348 200 2 May 16 12:06 dir13029 drwxrwxr-x 2 1348 200 2 May 16 12:06 dir13031 drwxrwxr-x 2 1348 200 3 May 16 12:06 dir13032 /data21/gvol/franco/dir1226/dir25/dir13017: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data21/gvol/franco/dir1226/dir25/dir13018: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data21/gvol/franco/dir1226/dir25/dir13020: total 286038 drwxrwxr-x 2 1348 200 3 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. -rw-rw-r-- 2 1348 200 291201024 May 16 12:05 file0 /data21/gvol/franco/dir1226/dir25/dir13021: total 250893 drwxrwxr-x 2 1348 200 3 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. -rw-rw-r-- 2 1348 200 255164416 May 16 12:05 file0 /data21/gvol/franco/dir1226/dir25/dir13022: total 757896 drwxrwxr-x 2 1348 200 3 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. -rw-rw-r-- 2 1348 200 775151616 May 16 12:05 file0 /data21/gvol/franco/dir1226/dir25/dir13024: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data21/gvol/franco/dir1226/dir25/dir13027: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data21/gvol/franco/dir1226/dir25/dir13028: total 616414 drwxrwxr-x 2 1348 200 3 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. -rw-rw-r-- 2 1348 200 630005760 May 16 12:06 file0 /data21/gvol/franco/dir1226/dir25/dir13029: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data21/gvol/franco/dir1226/dir25/dir13031: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data21/gvol/franco/dir1226/dir25/dir13032: total 663958 drwxrwxr-x 2 1348 200 3 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. -rw-rw-r-- 2 1348 200 678756352 May 16 12:06 file0 /data22/gvol/franco/dir1226/dir25: total 2084 drwxrwxr-x 13 1348 200 13 May 21 03:58 . drwxrwxr-x 3 1348 200 3 May 21 03:58 .. drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13017 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13018 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13020 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13021 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13022 drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13024 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13027 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13028 drwxrwxr-x 2 1348 200 3 May 16 12:06 dir13029 drwxrwxr-x 2 1348 200 2 May 16 12:06 dir13031 drwxrwxr-x 2 1348 200 2 May 16 12:06 dir13032 /data22/gvol/franco/dir1226/dir25/dir13017: total 2051 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data22/gvol/franco/dir1226/dir25/dir13018: total 2051 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data22/gvol/franco/dir1226/dir25/dir13020: total 2051 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data22/gvol/franco/dir1226/dir25/dir13021: total 2051 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data22/gvol/franco/dir1226/dir25/dir13022: total 2051 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data22/gvol/franco/dir1226/dir25/dir13024: total 494108 drwxrwxr-x 2 1348 200 3 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. -rw-rw-r-- 2 1348 200 504602624 May 16 12:05 file0 /data22/gvol/franco/dir1226/dir25/dir13027: total 2051 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data22/gvol/franco/dir1226/dir25/dir13028: total 2051 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data22/gvol/franco/dir1226/dir25/dir13029: total 487204 drwxrwxr-x 2 1348 200 3 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. -rw-rw-r-- 2 1348 200 497475584 May 16 12:06 file0 /data22/gvol/franco/dir1226/dir25/dir13031: total 2051 drwxrwxr-x 2 1348 200 2 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data22/gvol/franco/dir1226/dir25/dir13032: total 2051 drwxrwxr-x 2 1348 200 2 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data23/gvol/franco/dir1226/dir25: total 2086 drwxrwxr-x 13 1348 200 13 May 21 03:58 . drwxrwxr-x 3 1348 200 3 May 21 03:58 .. drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13017 drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13018 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13020 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13021 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13022 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13024 drwxrwxr-x 2 1348 200 3 May 16 12:05 dir13027 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13028 drwxrwxr-x 2 1348 200 2 May 16 12:06 dir13029 drwxrwxr-x 2 1348 200 3 May 16 12:06 dir13031 drwxrwxr-x 2 1348 200 2 May 16 12:06 dir13032 /data23/gvol/franco/dir1226/dir25/dir13017: total 156066 drwxrwxr-x 2 1348 200 3 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. -rw-rw-r-- 2 1348 200 157917184 May 16 12:05 file0 /data23/gvol/franco/dir1226/dir25/dir13018: total 661280 drwxrwxr-x 2 1348 200 3 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. -rw-rw-r-- 2 1348 200 675979264 May 16 12:05 file0 /data23/gvol/franco/dir1226/dir25/dir13020: total 2053 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data23/gvol/franco/dir1226/dir25/dir13021: total 2053 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data23/gvol/franco/dir1226/dir25/dir13022: total 2053 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data23/gvol/franco/dir1226/dir25/dir13024: total 2053 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data23/gvol/franco/dir1226/dir25/dir13027: total 133064 drwxrwxr-x 2 1348 200 3 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. -rw-rw-r-- 2 1348 200 134307840 May 16 12:05 file0 /data23/gvol/franco/dir1226/dir25/dir13028: total 2053 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data23/gvol/franco/dir1226/dir25/dir13029: total 2053 drwxrwxr-x 2 1348 200 2 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data23/gvol/franco/dir1226/dir25/dir13031: total 627668 drwxrwxr-x 2 1348 200 3 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. -rw-rw-r-- 2 1348 200 641515520 May 16 12:06 file0 /data23/gvol/franco/dir1226/dir25/dir13032: total 2053 drwxrwxr-x 2 1348 200 2 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data24/gvol/franco/dir1226/dir25: total 2081 drwxrwxr-x 13 1348 200 13 May 21 03:58 . drwxrwxr-x 3 1348 200 3 May 21 03:58 .. drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13017 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13018 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13020 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13021 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13022 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13024 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13027 drwxrwxr-x 2 1348 200 2 May 16 12:05 dir13028 drwxrwxr-x 2 1348 200 2 May 16 12:06 dir13029 drwxrwxr-x 2 1348 200 2 May 16 12:06 dir13031 drwxrwxr-x 2 1348 200 2 May 16 12:06 dir13032 /data24/gvol/franco/dir1226/dir25/dir13017: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data24/gvol/franco/dir1226/dir25/dir13018: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data24/gvol/franco/dir1226/dir25/dir13020: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data24/gvol/franco/dir1226/dir25/dir13021: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data24/gvol/franco/dir1226/dir25/dir13022: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data24/gvol/franco/dir1226/dir25/dir13024: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data24/gvol/franco/dir1226/dir25/dir13027: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data24/gvol/franco/dir1226/dir25/dir13028: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:05 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data24/gvol/franco/dir1226/dir25/dir13029: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data24/gvol/franco/dir1226/dir25/dir13031: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. /data24/gvol/franco/dir1226/dir25/dir13032: total 2048 drwxrwxr-x 2 1348 200 2 May 16 12:06 . drwxrwxr-x 13 1348 200 13 May 21 03:58 .. # file: data21/gvol/franco/dir1226/dir25 trusted.gfid=0s9OYWmrHlQlu0O3bUvCfVKA== trusted.glusterfs.dht=0sAAAAAQAAAABf///9f///+w== # file: data21/gvol/franco/dir1226/dir25/dir13032 trusted.gfid=0s5/XbT5BXRKSWCX1xC+CAeA== trusted.glusterfs.dht=0sAAAAAQAAAAB////+v////A== # file: data21/gvol/franco/dir1226/dir25/dir13032/file0 trusted.gfid=0swFeNsYCvQ7qTsGVXhLwEQw== # file: data21/gvol/franco/dir1226/dir25/dir13021 trusted.gfid=0sppOgGfHiQ1mjZ99XJoMntA== trusted.glusterfs.dht=0sAAAAAQAAAAB////+v////A== # file: data21/gvol/franco/dir1226/dir25/dir13021/file0 trusted.gfid=0s5be6o2IjTeuxYv/FuUCODQ== # file: data21/gvol/franco/dir1226/dir25/dir13017 trusted.gfid=0smGnsSlC6TtG12MvaiR6s0g== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAP////g== # file: data21/gvol/franco/dir1226/dir25/dir13031 trusted.gfid=0s+1rpUzb3Q4WZqeqUOhDSHA== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAP////g== # file: data21/gvol/franco/dir1226/dir25/dir13022 trusted.gfid=0su+dCGvxbTK+grv2tt1+cWQ== trusted.glusterfs.dht=0sAAAAAQAAAAB////+v////A== # file: data21/gvol/franco/dir1226/dir25/dir13022/file0 trusted.gfid=0s9H/U8CWjRmGzUrXSW3c9bQ== # file: data21/gvol/franco/dir1226/dir25/dir13028 trusted.gfid=0s5DAyskivQxCbV0ODsjY/DQ== trusted.glusterfs.dht=0sAAAAAQAAAAB////+v////A== # file: data21/gvol/franco/dir1226/dir25/dir13028/file0 trusted.gfid=0slTPOwFWmQ3SswVTWn3tbyw== # file: data21/gvol/franco/dir1226/dir25/dir13018 trusted.gfid=0sO1xMdnQ8Q0aCNEKN/ZzhUQ== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAP////g== # file: data21/gvol/franco/dir1226/dir25/dir13020 trusted.gfid=0spzhST5s+ShiMHqI9HeN9ng== trusted.glusterfs.dht=0sAAAAAQAAAAB////+v////A== # file: data21/gvol/franco/dir1226/dir25/dir13020/file0 trusted.gfid=0sgBomFwwfR8SEc51ExDIsqA== # file: data21/gvol/franco/dir1226/dir25/dir13027 trusted.gfid=0skJ+xvLSOQPGLtEIowyERDA== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAP////g== # file: data21/gvol/franco/dir1226/dir25/dir13029 trusted.gfid=0smsAi6mM4Q8WjpbcHzTQM6g== trusted.glusterfs.dht=0sAAAAAQAAAAA/////f////Q== # file: data21/gvol/franco/dir1226/dir25/dir13024 trusted.gfid=0stIckXZDUQWWrAR7llTxYPQ== trusted.glusterfs.dht=0sAAAAAQAAAAA/////f////Q== # file: data22/gvol/franco/dir1226/dir25 trusted.gfid=0s9OYWmrHlQlu0O3bUvCfVKA== trusted.glusterfs.dht=0sAAAAAQAAAAB////8n///+g== # file: data22/gvol/franco/dir1226/dir25/dir13029 trusted.gfid=0smsAi6mM4Q8WjpbcHzTQM6g== trusted.glusterfs.dht=0sAAAAAQAAAAB////+v////A== # file: data22/gvol/franco/dir1226/dir25/dir13029/file0 trusted.gfid=0sCr2NLe8EQy6g59ldr6aCpw== # file: data22/gvol/franco/dir1226/dir25/dir13024 trusted.gfid=0stIckXZDUQWWrAR7llTxYPQ== trusted.glusterfs.dht=0sAAAAAQAAAAB////+v////A== # file: data22/gvol/franco/dir1226/dir25/dir13024/file0 trusted.gfid=0sl3Bd57MjQcetMfg9dmcBzw== # file: data22/gvol/franco/dir1226/dir25/dir13020 trusted.gfid=0spzhST5s+ShiMHqI9HeN9ng== trusted.glusterfs.dht=0sAAAAAQAAAAC////9/////w== # file: data22/gvol/franco/dir1226/dir25/dir13018 trusted.gfid=0sO1xMdnQ8Q0aCNEKN/ZzhUQ== trusted.glusterfs.dht=0sAAAAAQAAAAA/////f////Q== # file: data22/gvol/franco/dir1226/dir25/dir13027 trusted.gfid=0skJ+xvLSOQPGLtEIowyERDA== trusted.glusterfs.dht=0sAAAAAQAAAAA/////f////Q== # file: data22/gvol/franco/dir1226/dir25/dir13017 trusted.gfid=0smGnsSlC6TtG12MvaiR6s0g== trusted.glusterfs.dht=0sAAAAAQAAAAA/////f////Q== # file: data22/gvol/franco/dir1226/dir25/dir13031 trusted.gfid=0s+1rpUzb3Q4WZqeqUOhDSHA== trusted.glusterfs.dht=0sAAAAAQAAAAA/////f////Q== # file: data22/gvol/franco/dir1226/dir25/dir13022 trusted.gfid=0su+dCGvxbTK+grv2tt1+cWQ== trusted.glusterfs.dht=0sAAAAAQAAAAC////9/////w== # file: data22/gvol/franco/dir1226/dir25/dir13028 trusted.gfid=0s5DAyskivQxCbV0ODsjY/DQ== trusted.glusterfs.dht=0sAAAAAQAAAAC////9/////w== # file: data22/gvol/franco/dir1226/dir25/dir13032 trusted.gfid=0s5/XbT5BXRKSWCX1xC+CAeA== trusted.glusterfs.dht=0sAAAAAQAAAAC////9/////w== # file: data22/gvol/franco/dir1226/dir25/dir13021 trusted.gfid=0sppOgGfHiQ1mjZ99XJoMntA== trusted.glusterfs.dht=0sAAAAAQAAAAC////9/////w== # file: data23/gvol/franco/dir1226/dir25 trusted.gfid=0s9OYWmrHlQlu0O3bUvCfVKA== trusted.glusterfs.dht=0sAAAAAQAAAACf///7v///+Q== # file: data23/gvol/franco/dir1226/dir25/dir13024 trusted.gfid=0stIckXZDUQWWrAR7llTxYPQ== trusted.glusterfs.dht=0sAAAAAQAAAAC////9/////w== # file: data23/gvol/franco/dir1226/dir25/dir13029 trusted.gfid=0smsAi6mM4Q8WjpbcHzTQM6g== trusted.glusterfs.dht=0sAAAAAQAAAAC////9/////w== # file: data23/gvol/franco/dir1226/dir25/dir13027 trusted.gfid=0skJ+xvLSOQPGLtEIowyERDA== trusted.glusterfs.dht=0sAAAAAQAAAAB////+v////A== # file: data23/gvol/franco/dir1226/dir25/dir13027/file0 trusted.gfid=0sDU0WYCOZR5WeBf6sCHLt0A== # file: data23/gvol/franco/dir1226/dir25/dir13018 trusted.gfid=0sO1xMdnQ8Q0aCNEKN/ZzhUQ== trusted.glusterfs.dht=0sAAAAAQAAAAB////+v////A== # file: data23/gvol/franco/dir1226/dir25/dir13018/file0 trusted.gfid=0sYzdmPyowRFCRLYVIsBlKYQ== # file: data23/gvol/franco/dir1226/dir25/dir13020 trusted.gfid=0spzhST5s+ShiMHqI9HeN9ng== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAP////g== # file: data23/gvol/franco/dir1226/dir25/dir13028 trusted.gfid=0s5DAyskivQxCbV0ODsjY/DQ== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAP////g== # file: data23/gvol/franco/dir1226/dir25/dir13022 trusted.gfid=0su+dCGvxbTK+grv2tt1+cWQ== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAP////g== # file: data23/gvol/franco/dir1226/dir25/dir13031 trusted.gfid=0s+1rpUzb3Q4WZqeqUOhDSHA== trusted.glusterfs.dht=0sAAAAAQAAAAB////+v////A== # file: data23/gvol/franco/dir1226/dir25/dir13031/file0 trusted.gfid=0sAX37CO8zQPebxYsBKK/1WQ== # file: data23/gvol/franco/dir1226/dir25/dir13017 trusted.gfid=0smGnsSlC6TtG12MvaiR6s0g== trusted.glusterfs.dht=0sAAAAAQAAAAB////+v////A== # file: data23/gvol/franco/dir1226/dir25/dir13017/file0 trusted.gfid=0s3K5Oc7Z5T3utIL0RjZZGLw== # file: data23/gvol/franco/dir1226/dir25/dir13021 trusted.gfid=0sppOgGfHiQ1mjZ99XJoMntA== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAP////g== # file: data23/gvol/franco/dir1226/dir25/dir13032 trusted.gfid=0s5/XbT5BXRKSWCX1xC+CAeA== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAP////g== # file: data24/gvol/franco/dir1226/dir25 trusted.gfid=0s9OYWmrHlQlu0O3bUvCfVKA== trusted.glusterfs.dht=0sAAAAAQAAAAC////63///+A== # file: data24/gvol/franco/dir1226/dir25/dir13032 trusted.gfid=0s5/XbT5BXRKSWCX1xC+CAeA== trusted.glusterfs.dht=0sAAAAAQAAAAA/////f////Q== # file: data24/gvol/franco/dir1226/dir25/dir13021 trusted.gfid=0sppOgGfHiQ1mjZ99XJoMntA== trusted.glusterfs.dht=0sAAAAAQAAAAA/////f////Q== # file: data24/gvol/franco/dir1226/dir25/dir13017 trusted.gfid=0smGnsSlC6TtG12MvaiR6s0g== trusted.glusterfs.dht=0sAAAAAQAAAAC////9/////w== # file: data24/gvol/franco/dir1226/dir25/dir13031 trusted.gfid=0s+1rpUzb3Q4WZqeqUOhDSHA== trusted.glusterfs.dht=0sAAAAAQAAAAC////9/////w== # file: data24/gvol/franco/dir1226/dir25/dir13028 trusted.gfid=0s5DAyskivQxCbV0ODsjY/DQ== trusted.glusterfs.dht=0sAAAAAQAAAAA/////f////Q== # file: data24/gvol/franco/dir1226/dir25/dir13022 trusted.gfid=0su+dCGvxbTK+grv2tt1+cWQ== trusted.glusterfs.dht=0sAAAAAQAAAAA/////f////Q== # file: data24/gvol/franco/dir1226/dir25/dir13018 trusted.gfid=0sO1xMdnQ8Q0aCNEKN/ZzhUQ== trusted.glusterfs.dht=0sAAAAAQAAAAC////9/////w== # file: data24/gvol/franco/dir1226/dir25/dir13020 trusted.gfid=0spzhST5s+ShiMHqI9HeN9ng== trusted.glusterfs.dht=0sAAAAAQAAAAA/////f////Q== # file: data24/gvol/franco/dir1226/dir25/dir13027 trusted.gfid=0skJ+xvLSOQPGLtEIowyERDA== trusted.glusterfs.dht=0sAAAAAQAAAAC////9/////w== # file: data24/gvol/franco/dir1226/dir25/dir13029 trusted.gfid=0smsAi6mM4Q8WjpbcHzTQM6g== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAP////g== # file: data24/gvol/franco/dir1226/dir25/dir13024 trusted.gfid=0stIckXZDUQWWrAR7llTxYPQ== trusted.glusterfs.dht=0sAAAAAQAAAAAAAAAAP////g==
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://supercolony.gluster.org/mailman/listinfo/gluster-users
