On Thu, 2013-10-10 at 18:33 -0500, Xianghua Xiao wrote:
>
<SNIP>
> I switched to fileio and now READ is at full speed, i.e. wirespeed of
> 10Gbps, plus all cores are being used(though still only core0 takes
> the MSI-X interrupts). However, WRITE slows down dramatically from
> 415MB/s to 150MB/s, that really puzzles me.
>
>From the Mode: O_DSYNC output below, the FILEIO device is *not* running
in buffered I/O mode for WRITEs.
Also, can you confirm using iostat -xm that the READs are coming from
the backend device, and not from the buffer cache itself..?
FYI, iostat -xm output also shosw the size of the I/Os being submitted
to both the MD RAID devices, as well as to the underlying mpt2sas SCSI
devices. This would be useful to see if the size of I/Os is different
between backend drivers.
>
> I had the same setting, including :
> echo 1 > /sys/kernel/config/target/core/fileio_$i/my_fileio
> $i/attrib/emulate_write_cache for all endpoints.
> At WRITE all cores are equally involved, they're just not fully loaded
> at all.
>
Setting emulate_write_cache is the per device attribute for exposing the
SCSI WriteCacheEnabled bit to the initiator, and does not directly
enable buffered I/O operation for FILEIO backends.
>
> Here is the output of fileio:
>
> Rounding down aligned max_sectors from 32767 to 32760
> Status: DEACTIVATED Max Queue Depth: 0 SectorSize: 512 HwMaxSectors: 32760
> TCM FILEIO ID: 0 File: /dev/md12 Size: 0 Mode: O_DSYNC
>
> I'm still trying to get the optimal throughput for sequential
> READ/WRITE at the moment. Reliability is secondary as I'm trying to
> stress the system performance for now. Is there anything else that I
> can tune for FILEIO (or IBLOCK)?
Buffered FILEIO can be enabled with targetcli + rtslib, but not during
setup with the legacy lio-utils code.
The easiest way to do this is modifying /etc/target/tcm_start.sh after
the configuration has been saved, and then including the
',fd_buffered_io=1' parameter for each FILEIO device like so:
tcm_node --establishdev fileio_0/test fd_dev_name=/tmp/test,fd_dev_size=2147483648,fd_buffered_io=1
>From there restart the target, and verify the same output from
/sys/kernel/config/target/core/$FILEIO_HBA/$FILEIO_DEV/info as above, which
should now read:
Status: DEACTIVATED Max Queue Depth: 0 SectorSize: 512 HwMaxSectors: 32760
TCM FILEIO ID: 0 File: /dev/md12 Size: 0 Mode: Buffered-WCE
--nab
--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
