On 5/17/21 11:54 AM, Jim Guo wrote: > Hello, Mr. Li. > Recently I was experiencing frequent io hang when testing with fio > with 4K random write. Fio iops dropped to 0 for about 20 seconds > every several minutes. > After some debugging, I discovered that it is the incremental gc that > cause this problem. > My cache disk is relatively small (375GiB with 4K block size and 512K > bucket size), backing hdds are 4 x 1 TiB. I cannot reproduce this on > another environment with bigger cache disk. > When running 4K random write fio bench, the buckets are consumed very > quickly and soon it has to invalidate some bucket (this happens quite > often). Since the cache disk is small, a lot of write io will soon > reach sectors_to_gc and trigger gc thread. Write io will also increase > search_inflight, which cause gc thread to sleep for 100ms. This will > cause gc procedure to execute for a long time, and invalidating bucket > for the write io will wait for the whole gc procedure. > After removing the 100ms sleep from the incremental gc patch, the io > never hang any more. What is the kernel version in your system? And where the kernel package is from? > I think for small ssd, sleeping for 100ms seems too long or maybe > write io should not trigger gc thread to sleep for 100ms? > Thank you very much. > Do you have a testing result on this idea? Thanks. Coly Li
