Hi,
On 6/19/20 10:15 AM, Hou Tao wrote:
Hi,
Recently we encountered a read performance problem on LSI SAS-3 3108 RAID controller.
The read performance is much better when nr_requests is set as 192 compared with 256,
but after disabling the NoRA (No Readahead) feature of the hardware RAID1, there will
be no difference between nr_requests=192 and nr_requests=256.
Do you know if RAID10 mode could be affected or not? And how about MegaRAID
2208 controller? Thanks.
One scene is the direct read of one ext4 file with 8GB size. The ext4 fs is stacked
on hardware RAID1 with 3.7TB size, and the RAID1 is composed of two HDDs.
The queue_depth of RAID1 device is set as 256 by driver.
fio --direct=1 --ioengine=libaio --group_reporting=1 --runtime=30 --bs=4k \
--name=1 --numjobs=1 --iodepth=512 --filesize=8G --directory=/tmp/sdd
one ext4 file:
IOPS | nr_requests=128 | nr_requests=192 | nr_requests=256 |
RA | 51k | 51k | 48k |
NoRA | 47k | 46k | 47k |
Another scene is the direct read of two ext4 files with 4GB size.
fio --direct=1 --ioengine=libaio --group_reporting=1 --runtime=30 --bs=4k \
--numjobs=2 --iodepth=256 --filesize=4G --directory=/tmp/sdd
two ext4 files:
IOPS | nr_requests=128 | nr_requests=192 | nr_requests=256 |
RA | 95.7k | 94.5k | 30.7k |
NoRA | 27.3k | 27.1k | 27.2k |
These results show RA feature can boost the read performance, but when nr_requests
is increased, performance is degraded.
However when using a JBOD setup on HDD which has the same model as the HDD
in RAID1 setup, there is no such problem.
one ext4 file:
IOPS | nr_requests=128 | nr_requests=192 | nr_requests=256 |
| 50k | 50.4k | 50.5 |
two ext4 files:
IOPS | nr_requests=128 | nr_requests=192 | nr_requests=256 |
| 46.5k | 46.3k | 46.0k |
So is the performance degradation a known issue of the RAID1 RA feature,
or is there other explanation for it ?
Regards,
Tao
---
Other details of test environment:
(1) linux kernel version
5.6.15
(2) block setup
device/queue_depth:256
queue/scheduler:[mq-deadline] kyber bfq none
(3) driver version
[...]
(4) megaraid firmware version
Product Name = SAS3108
FW Package Build = 24.16.0-0106
BIOS Version = 6.32.02.0_4.17.08.00_0x06150500
FW Version = 4.660.00-8102
Driver Name = megaraid_sas
Driver Version = 07.713.01.00-rc1
Current Personality = RAID-Mode
Not sure the behavior could be different with the latest firmware
version 24.21.0-0126.
Thanks,
Guoqing
