On 13-09-18 03:58 AM, Jack Wang wrote:
On 09/18/2013 08:41 AM, Alireza Haghdoost wrote:
Hi
I am working on a high throughput and low latency application which
does not tolerate block layer overhead to send IO request directly to
fiber channel lower layer SCSI driver. I used to work with libaio but
currently I am looking for a way to by pass the block layer and send
SCSI commands from the application layer directly to the SCSI driver
using /dev/sgX device and ioctl() system call.
I have noticed that sending IO request through sg device even with
nonblocking and direct IO flags is quite slow and does not fill up
lower layer SCSI driver TCQ queue. i.e IO depth or
/sys/block/sdX/in_flight is always ZERO. Therefore the application
throughput is even lower that sending IO request through block layer
with libaio and io_submit() system call. In both cases I used only one
IO context (or fd) and single threaded.
Hi Alireza,
I think what you want is in_flight command scsi dispatch to low level
device.
I submit a simple patch to export device_busy
http://www.spinics.net/lists/linux-scsi/msg68697.html
I also notice fio sg engine will not fill queue properly, but haven't
look into deeper.
Cheers
Jack
I have noticed that some well known benchmarking tools like fio does
not support IO depth for sg devices as well. Therefore, I was
wondering if it is feasible to bypass block layer and achieve higher
throughput and lower latency (for sending IO request only).
Any comment on my issue is highly appreciated.
I'm not sure if this is relevant to your problem but by
default both the bsg and sg drivers "queue at head"
when they inject SCSI commands into the block layer.
The bsg driver has a BSG_FLAG_Q_AT_TAIL flag to change
that queueing to what may be preferable for your purposes.
The sg driver could, but does not, support that flag.
Doug Gilbert
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
