Re: Nearly full OSDs with very little (apparent) FS utilization

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sat, Oct 26, 2013 at 2:05 AM, Gregory Farnum <greg@xxxxxxxxxxx> wrote:
> Are you sure you're using only CephFS? Do you have any snapshots?
> -Greg
> Software Engineer #42 @ http://inktank.com | http://ceph.com
>
>
> On Fri, Oct 25, 2013 at 2:59 AM, Miguel Afonso Oliveira
> <m.a.oliveira@xxxxxxxxxxx> wrote:
>> Hi,
>>
>> I have a recent ceph deployment with version:
>>
>> ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7)
>>
>> on 4 12TB OSDs:
>>
>> GLOBAL:
>>     SIZE       AVAIL     RAW USED     %RAW USED
>>     49143G     8285G     40858G       83.14
>>
>> POOLS:
>>     NAME         ID     USED       %USED     OBJECTS
>>     data         0      20396G     41.50     7342052
>>     metadata     1      276M       0         81826
>>     rbd          2      0          0         0
>>
>> and this morning I started to get a warning about a full OSD:
>>
>>   cluster 14320bfb-8b8c-4280-afee-df63172b1d0c
>>    health HEALTH_WARN 1 near full osd(s)
>>    monmap e3: 3 mons at
>> {gridio1=10.112.0.148:6789/0,gridio2=10.112.0.149:6789/0,gridio3=10.112.0.150:6789/0},
>> election epoch 44, quorum 0,1,2 gridio1,gridio2,gridio3
>>    osdmap e498: 4 osds: 4 up, 4 in
>>     pgmap v485463: 6144 pgs: 6142 active+clean, 2
>> active+clean+scrubbing+deep; 20396 GB data, 40858 GB used, 8285 GB / 49143
>> GB avail; 2252B/s wr, 0op/s
>>    mdsmap e54: 1/1/1 up {0=gridio4=up:active}
>>
>> However when I use a du on the mount point I get:
>>
>> [root@ce01 /]# du -bsh grid/
>> 31G    grid/

what is the output of 'getfattr -d -m -  grid/' ?

>>
>> This number is reasonable for the utilization of the FS... Even entering
>> into account with replication (2 in this case) there is no way I can
>> understand an underlying raw utilization
>> of over 40TB....
>>
>> What is going on here? What can cause such discrepancies? I know ceph uses
>> delayed deletion but the discrepancy is too big...
>>
>> Cheers,
>>
>> MAO
>>
>>
>>
>>
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.co
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux