Re: cephfs(Kraken 11.2.1), Unable to write more file when one dir more than 100000 files, mds_bal_fragment_size_max = 5000000

"donglifecomm@xxxxxxxxx" <donglifecomm@xxxxxxxxx> · Mon, 11 Sep 2017 14:07:38 +0800

ZhengYan,

I set "mds_bal_fragment_size_max = 100000, mds_bal_frag = true", then  I write  100000 files named 512k.file$i, but there are still some file is missing. such as :
[root@yj43959-ceph-dev cephfs]# find ./volumes/ -type f | wc -l
91070

donglifecomm@xxxxxxxxx

From: Yan, Zheng
Date: 2017-09-08 15:20
To: donglifecomm@xxxxxxxxx
CC: ceph-users; marcus.haarmann
Subject: Re: cephfs(Kraken 11.2.1),  Unable to write more file when one dir more than 100000 files,  mds_bal_fragment_size_max = 5000000

> On 8 Sep 2017, at 13:54, donglifecomm@xxxxxxxxx wrote:
> 
> ZhengYan,
> 
> I'm sorry, just a description of some questions.
> 
> when one dir more than 100000 files, I can continue to write it , but I don't find file which wrote in the past. for example:
> 1.  I write  100000 files named 512k.file$i
>        
> 2. I continue to write  10000 files named aaa.file$i
> 
> 3. I continue to write  10000 files named bbb.file$i
> 
> 4.  I continue to write  10000 files named ccc.file$i
> 
> 5. I continue to write  10000 files named ddd.file$i
> 
> 6. I can't find all ddd.file$i, some ddd.file$i missing. such as:
> 
> [root@yj43959-ceph-dev scripts]# find /mnt/cephfs/volumes -type f  |  grep 512k.file | wc -l
> 100000
> [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/aaa.file* | wc -l
> 10000
> [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/bbb.file* | wc -l
> 10000
> [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ccc.file* | wc -l
> 10000
> [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ddd.file* | wc -l    // some files missing
> 1072

It’s likely caused by http://tracker.ceph.com/issues/18314.  To support very large directory, you should enable directory fragment instead of enlarge mds_bal_fragment_size_max.

Regards
Yan, Zheng

> 
> <InsertPic_.png>
> 
> donglifecomm@xxxxxxxxx
>  
> From: donglifecomm@xxxxxxxxx
> Date: 2017-09-08 13:30
> To: zyan
> CC: ceph-users
> Subject: cephfs(Kraken 11.2.1), Unable to write more file when one dir more than 100000 files, mds_bal_fragment_size_max = 5000000
> ZhengYan,
> 
> I test cephfs(Kraken 11.2.1),  I can't write more files when one dir more than 100000 files, I have already set up "mds_bal_fragment_size_max = 5000000".
> 
> why is this case? Is it a bug?
> 
> Thanks a lot.
> 
> donglifecomm@xxxxxxxxx

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com