Yes, we do use customized layout settings for most of our folders. We have some long running backup jobs which require high-throughput writes in order to finish in a reasonable amount of time. ________________________________________ From: Florent B <florent@xxxxxxxxxxx> Sent: Wednesday, April 20, 2016 11:07 To: Yan, Zheng; Simion Rad Cc: ceph-users@xxxxxxxxxxxxxx Subject: Re: cephfs does not seem to properly free up space That seems to be the bug we have for years now with CephFS. We always used customized layout. On 04/20/2016 02:20 AM, Yan, Zheng wrote: > have you ever used fancy layout? > > see http://tracker.ceph.com/issues/15050 > > > On Wed, Apr 20, 2016 at 3:17 AM, Simion Rad <Simion.Rad@xxxxxxxxx> wrote: >> Mounting and unmount doesn't change anyting. >> The used space reported by df command is nearly the same as the values returned by ceph -s command. >> >> Example 1, df output: >> ceph-fuse 334T 134T 200T 41% /cephfs >> >> Example 2, ceph -s output: >> health HEALTH_WARN >> mds0: Many clients (22) failing to respond to cache pressure >> noscrub,nodeep-scrub,sortbitwise flag(s) set >> monmap e1: 5 mons at {r730-12=10.103.213.12:6789/0,r730-4=10.103.213.4:6789/0,r730-5= >> 10.103.213.5:6789/0,r730-8=10.103.213.8:6789/0,r730-9=10.103.213.9:6789/0} >> election epoch 132, quorum 0,1,2,3,4 r730-4,r730-5,r730-8,r730-9,r730-12 >> mdsmap e14637: 1/1/1 up {0=ceph2-mds-2=up:active} >> osdmap e6549: 68 osds: 68 up, 68 in >> flags noscrub,nodeep-scrub,sortbitwise >> pgmap v4394151: 896 pgs, 3 pools, 54569 GB data, 56582 kobjects >> 133 TB used, 199 TB / 333 TB avail >> 896 active+clean >> client io 47395 B/s rd, 1979 kB/s wr, 388 op/s >> >> >> ________________________________________ >> From: John Spray <jspray@xxxxxxxxxx> >> Sent: Tuesday, April 19, 2016 22:04 >> To: Simion Rad >> Cc: ceph-users@xxxxxxxxxxxxxx >> Subject: Re: cephfs does not seem to properly free up space >> >> On Tue, Apr 19, 2016 at 2:40 PM, Simion Rad <Simion.Rad@xxxxxxxxx> wrote: >>> Hello, >>> >>> >>> At my workplace we have a production cephfs cluster (334 TB on 60 OSDs) >>> which was recently upgraded from Infernalis 9.2.0 to Infernalis 9.2.1 on >>> Ubuntu 14.04.3 (linux 3.19.0-33). >>> >>> It seems that cephfs still doesn't free up space at all or at least that's >>> what df command tells us. >> Hmm, historically there were bugs with the purging code, but I thought >> we fixed them before Infernalis. >> >> Does the space get freed after you unmount the client? Some issues >> have involved clients holding onto references to unlinked inodes. >> >> John >> >>> Is there a better way of getting a df-like output with other command for >>> cephfs ? >>> >>> >>> Thank you, >>> >>> Marius Rad >>> >>> SysAdmin >>> >>> www.propertyshark.com >>> >>> >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@xxxxxxxxxxxxxx >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |