> 21 апр. 2017 г., в 17:08, Bernd Schubert <bschubert@xxxxxxx> написал(а): >> >> Initial analyze say about several problems >> 0) CPU load isn’t high, and perf top say ldiskfs functions isn’t hot (2%-3% >> cpu), most spent for dir entry checking function. >> >> 1) lookup have a large time to read a directory block to verify file not >> exist. I think it because a block fragmentation. [root@pink03 ~]# cat >> /proc/100993/stack >> [<ffffffff81211b1e>] sleep_on_buffer+0xe/0x20 >> [<ffffffff812130da>] __wait_on_buffer+0x2a/0x30 >> [<ffffffffa0899e6c>] ldiskfs_bread+0x7c/0xc0 [ldiskfs] >> [<ffffffffa088ee4a>] __ldiskfs_read_dirblock+0x4a/0x400 [ldiskfs] >> [<ffffffffa08915af>] ldiskfs_dx_find_entry+0xef/0x200 [ldiskfs] >> [<ffffffffa0891b8b>] ldiskfs_find_entry+0x4cb/0x570 [ldiskfs] >> [<ffffffffa08921d5>] ldiskfs_lookup+0x75/0x230 [ldiskfs] >> [<ffffffff811e8e7d>] lookup_real+0x1d/0x50 >> [<ffffffff811e97f2>] __lookup_hash+0x42/0x60 >> [<ffffffff811ee848>] filename_create+0x98/0x180 >> [<ffffffff811ef6e1>] user_path_create+0x41/0x60 >> [<ffffffff811f084a>] SyS_mknodat+0xda/0x220 >> [<ffffffff811f09ad>] SyS_mknod+0x1d/0x20 >> [<ffffffff81645549>] system_call_fastpath+0x16/0x1b >> [<ffffffffffffffff>] 0xffffffffffffffff > > I wrote patches for ext4 a long time ago to get a better caching for that > > https://patchwork.ozlabs.org/patch/101200/ > > > For FhGFS/BeeGFS we then decided to use a totally different directory layout, > which totally eliminated the underlying issue for the main requirement or > large dirs at all. (Personally I would recommend to do the something similar > for Lustre - using hash dirs to store objects has a much too random access > pattern once the file system gets used with many files...). > > Also, a caching issue has been fixed by Mel Gorman in 3.11 (I didn't check if > these patches are backported to any vendor kernel). > > Bernd, Thanks to point we to patches, I will test with it on my next test loop. As about a different layout - it’s exist as separate option. Alex
