On Mon, Oct 10, 2016 at 9:05 AM, Hauke Homburg <hhomburg@xxxxxxxxxxxxxx> wrote: > Am 07.10.2016 um 17:37 schrieb Gregory Farnum: >> On Fri, Oct 7, 2016 at 7:15 AM, Hauke Homburg <hhomburg@xxxxxxxxxxxxxx> wrote: >>> Hello, >>> >>> I have a Ceph Cluster with 5 Server, and 40 OSD. Aktual on this Cluster >>> are 85GB Free Space, and the rsync dir has lots of Pictures and a Data >>> Volume of 40GB. >>> >>> The Linux is a Centos 7 and the Last stable Ceph. The Client is a Debian >>> 8 with Kernel 4 and the Cluster is with cephfs mounted. >>> >>> When i sync the Directory i see often the Message rsync mkstemp no space >>> left on device (28). At this Point i can touch a File in anotherDiretory >>> in the Cluster. In the Diretory i have ~ 630000 Files. Are this too much >>> Files? >> Yes, in recent releases CephFS limits you to 100k dentries in a single >> directory fragment. This *includes* the "stray" directories that files >> get moved into when you unlink them, and is intended to prevent issues >> with very large folders. It will stop being a problem once we enable >> automatic fragmenting (soon, hopefully). >> You can change that by changing the "mds bal fragment size max" >> config, but you're probably better off by figuring out if you've got >> an over-large directory or if you're deleting files faster than the >> cluster can keep up. There was a thread about this very recently and >> John included some details about tuning if you check the archives. :) >> -Greg > Hello, > > Thanks for the answer. > I enabled on the Cluster the mds bal frag = true Options. > > Today i read that i have to enable this option on the Client, too. With > a Fuse mount i can do it with the ceph Binary. I use the Kernel Module. > How can i do it there? mds_bal_frag is only a server side thing. You do also need to do the "ceph fs set <name> allow_dirfrags true", which you can run from any client with an admin key (but again, this is a server side thing, not a client setting). Note that the reason directory fragmentation is not enabled by default is that it wasn't thoroughly tested ahead of Jewel, so there's a reason it requires a --yes-i-really-mean-it. John > > Regards > > Hauke > > -- > www.w3-creative.de > > www.westchat.de > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |