Hi Jeffle & Xiang I have tested your fscache,erofs: fscache-based on-demand read semantics v9 patches sets https://www.spinics.net/lists/linux-fsdevel/msg216178.html. For now , it works fine with the nydus image-service. After the image data is fully loaded to local storage, it does have great IO performance gain compared with nydus V5 which is based on fuse. For 4K random read , fscache-based erofs can get the same performance with the original local filesystem. But I still saw a performance drop in the 4K sequential read case. And I found the root cause is in erofs_fscache_readahead() we use synchronous IO , which may stall the readahead pipelining. I have tried to change to use asynchronous io during erofs fscache readahead procedure, as what netfs did. Then I saw a great performance gain. Here are my test steps and results: - generate nydus v6 format image , in which stored a large file for IO test. - launch nydus image-service , and make image data fully loaded to local storage (ext4). - run fio with below cmd. fio -ioengine=psync -bs=4k -size=5G -direct=0 -thread -rw=read -filename=./test_image -name="test" -numjobs=1 -iodepth=16 -runtime=60 v9 patches: 202654 KB/s v9 patches + async readahead patch: 407213 KB/s ext4: 439912 KB/s Xin Yin (1): erofs: change to use asynchronous io for fscache readahead fs/erofs/fscache.c | 256 ++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 245 insertions(+), 11 deletions(-) -- 2.11.0
