On Thu, Sep 15, 2016 at 4:19 PM, Burkhard Linke <Burkhard.Linke@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx> wrote: > Hi, > > > On 09/15/2016 12:00 PM, John Spray wrote: >> >> On Thu, Sep 15, 2016 at 2:20 PM, Burkhard Linke >> <Burkhard.Linke@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx> wrote: >>> >>> Hi, >>> >>> does CephFS impose an upper limit on the number of files in a directory? >>> >>> >>> We currently have one directory with a large number of subdirectories: >>> >>> $ ls | wc -l >>> 158141 >>> >>> Creating a new subdirectory fails: >>> >>> $ touch foo >>> touch: cannot touch 'foo': No space left on device > > thansk for the fast reply. >> >> This limit was added recently: it's a limit on the size of a directory >> fragment. >> >> Previously folks were hitting nasty OSD issues with very large >> directory fragments, so we added this limit to give a clean failure >> instead. > > I remember seeing a thread on the devel mailing list about this issue. >> >> >> Directory fragmentation (mds_bal_frag setting) is turned off by >> default in Jewel: I was planning to get this activated by default in >> Kraken, but haven't quite got there yet. Once fragmentation is >> enabled you should find that the threshold for splitting dirfrags is >> hit well before you hit the safety limit that gives you ENOSPC. > > Does enabling directory fragmentation require a MDS restart? And are > directories processed at restart or on demand during the first access? Are > there known problems with fragmentation? You could change it at runtime using `injectargs` as with other configs. Directories are considered for fragmentation when they are touched by a client operation, so you won't see any fragmentation happen until the client does something. I would recommend that you don't enable fragmenation unless you have a 100% need for an unbounded number of files in one directory. I just tried a test run with frags enabled and saw a new failure! John >> >> Note that if you set mds_bal_frag then you also need to use the "ceph >> fs set <myfs> allow_dirfrags true" (that command from memory so check >> the help if it's wrong), or the MDSs will ignore the setting. > > So its allowing fragmentation first and changing the MDS configuration > afterwards. > > > Regards, > Burkhard > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |