Dear Experts, We are facing a performance degradation on the XFS partition. We was trying to copy a big file(200GB ~ 250GB) from a path to /dev/null, when performing cp command to 60s ~ 90s, the reading speed was suddenly down. At the beginning, the reading speed was around 1080MB/s, 60s later the speed was down to around 350MB/s. This problem is only found with XFS + Thick LUN. The test environment: Storage Model: Dell unity XT 380 Think/Thin LUN Linux Version: 4.12.14 The steps to run test: 1) Create a xfs partition with following commands parted -a opt /dev/sdb mklabel gpt mkpart sdb xfs 0% 100% mkfs.xfs /dev/sdbx mount /dev/sdbx /xfs 2) Create a ~200GB file named fileA in the partition. 3) Run cp command to copy the file created in step 2. Meanwhile, run iostat vmstat and blktrace to capture logs. cp /xfs/fileA /dev/null To narrow down this issue, we also did some experiments below to compare: 1) Run the test with dd command with XFS + Thick LUN dd if=/xfs/fileA of=/dev/null bs=32k status=progress Result: also meet performance degradation Speed: around 650MB/S Speed has changed to around 350MB/S since the 60s ~ 90s of cp run. 2) Run the test with dd command with raw device with XFS + Thick LUN dd if=/dev/sdbx of=/dev/null bs=32k status=progress Results: No performance degradation Speed: around 520MB/s 3) Run run test with ext4 + Think LUN cp /xfs/fileA /dev/null Results: No performance degradation Speed: around 1080MB/s 4) Run the test with cp with XFS + Thin LUN cp /xfs/fileA /dev/null Result: No performance degradation Speed: around 500MB/s 5) Run the test with dd with XFS + Thin LUN dd if=/xfs/fileA of=/dev/null bs=32k status=progress Result: No performance degradation Speed: around 500MB/s It seems the issue only can be triggered with XFS + Thick LUN, no matter dd or cp to read the test file. We would like to learn if there is something special with XFS in this test situation? Is it known? Do you have any thoughts or suggestions? Also, do you need vmstat or iostat logs or blktrace or any other logs to address this issue? Thank you in advance. Regards, Bruce
