Re: cephfs(Kraken 11.2.1), Unable to write more file when one dir more than 100000 files, mds_bal_fragment_size_max = 5000000

"donglifecomm@xxxxxxxxx" <donglifecomm@xxxxxxxxx> · Mon, 11 Sep 2017 15:46:19 +0800

ZhengYan,

kernel client is 4.12.0.

[root@yj43959-ceph-dev ~]# uname -a
Linux yj43959-ceph-dev.novalocal 4.12.0-1.el7.elrepo.x86_64 #1 SMP Sun Jul 2 20:38:48 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

Thanks a lot.

donglifecomm@xxxxxxxxx

From: Yan, Zheng
Date: 2017-09-11 15:24
To: donglifecomm@xxxxxxxxx
CC: ceph-users
Subject: Re: cephfs(Kraken 11.2.1),  Unable to write more file when one dir more than 100000 files,  mds_bal_fragment_size_max = 5000000

> On 11 Sep 2017, at 14:07, donglifecomm@xxxxxxxxx wrote:
> 
> ZhengYan,
> 
> I set "mds_bal_fragment_size_max = 100000, mds_bal_frag = true", then  I write  100000 files named 512k.file$i, but there are still some file is missing. such as :
> [root@yj43959-ceph-dev cephfs]# find ./volumes/ -type f | wc -l
> 91070

which client do you use? kernel client or fuse client and version?

Yan, Zheng

> 
> <InsertPic_.png>
> 
> donglifecomm@xxxxxxxxx
>  
> From: Yan, Zheng
> Date: 2017-09-08 15:20
> To: donglifecomm@xxxxxxxxx
> CC: ceph-users; marcus.haarmann
> Subject: Re: cephfs(Kraken 11.2.1), Unable to write more file when one dir more than 100000 files, mds_bal_fragment_size_max = 5000000
>  
> > On 8 Sep 2017, at 13:54, donglifecomm@xxxxxxxxx wrote:
> >
> > ZhengYan,
> >
> > I'm sorry, just a description of some questions.
> >
> > when one dir more than 100000 files, I can continue to write it , but I don't find file which wrote in the past. for example:
> > 1.  I write  100000 files named 512k.file$i
> >        
> > 2. I continue to write  10000 files named aaa.file$i
> >
> > 3. I continue to write  10000 files named bbb.file$i
> >
> > 4.  I continue to write  10000 files named ccc.file$i
> >
> > 5. I continue to write  10000 files named ddd.file$i
> >
> > 6. I can't find all ddd.file$i, some ddd.file$i missing. such as:
> >
> > [root@yj43959-ceph-dev scripts]# find /mnt/cephfs/volumes -type f  |  grep 512k.file | wc -l
> > 100000
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/aaa.file* | wc -l
> > 10000
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/bbb.file* | wc -l
> > 10000
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ccc.file* | wc -l
> > 10000
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ddd.file* | wc -l    // some files missing
> > 1072
>  
> It’s likely caused by http://tracker.ceph.com/issues/18314.  To support very large directory, you should enable directory fragment instead of enlarge mds_bal_fragment_size_max.
>  
> Regards
> Yan, Zheng
>  
> >
> > <InsertPic_.png>
> >
> > donglifecomm@xxxxxxxxx
> >  
> > From: donglifecomm@xxxxxxxxx
> > Date: 2017-09-08 13:30
> > To: zyan
> > CC: ceph-users
> > Subject: cephfs(Kraken 11.2.1), Unable to write more file when one dir more than 100000 files, mds_bal_fragment_size_max = 5000000
> > ZhengYan,
> >
> > I test cephfs(Kraken 11.2.1),  I can't write more files when one dir more than 100000 files, I have already set up "mds_bal_fragment_size_max = 5000000".
> >
> > why is this case? Is it a bug?
> >
> > Thanks a lot.
> >
> > donglifecomm@xxxxxxxxx

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com