I believe kernel 4.8 offers an NVMe over fabrics driver that supports SGL descriptors. In the case of a 16KB IO that is described as 512B, 3.5KB, 4KB, and 8KB, there would be separate data transfers to the device under test (such as NVMe over fabrics device). In the case of a 16KB IO allocated as contiguous pages, one large data transfer could be sufficient. Is it possible to force one way or the other with fio (assuming the driver has support)? We could simply use bssplit as 512B/3.5KB/4KB/8KB, but then the 8KB might actually be on noncontiguous pages, in which case it could really end up as 512B/3.5KB/4KB/4KB/4KB. Thanks. Regards, Jeff -----Original Message----- From: Jens Axboe [mailto:axboe@xxxxxxxxx] Sent: Tuesday, September 20, 2016 9:03 PM To: Jeff Furlong <jeff.furlong@xxxxxxxx>; fio@xxxxxxxxxxxxxxx Subject: Re: fio SGL support On 09/20/2016 12:03 PM, Jeff Furlong wrote: > Hi All, > Is it possible to add Scatter Gather List (SGL) support to fio? > Suppose we have a 16KB write to our device. I believe fio will > allocate memory in 4KB pages (depending upon machine architecture). > For SGL use, it would be valuable to define the SGL descriptors, such > that we could control the 16KB write as (example): 512B, 3.5KB, 4KB, > 8KB. Similarly for reads, we may define the SGL descriptors for how > to place the IO in memory. Fio will allocate buffers in multiples of the maximum blocksize specified in the job. If you ask for bs=16k or similar, then fio will allocate a virtually contig block of memory that is 16k in length and properly aligned, if needed. > If fio SGL support is possible, then the descriptors would be passed > down to the kernel and driver level, where SGL supported drivers would > be required. However, the latest Linux kernels are offering support. What interface is this? > If fio SGL support is possible, adding an option such as sglsplit > (similar to bssplit) may be an easy way to specify the descriptors. I'm a little puzzled by this. Since fio doesn't deal with physical addresses, any IO that fio performs can be described by a virtual address and a total length. That may map to different physical pages in the case of O_DIRECT, and subsequently an SG list will be needed on the driver side to describe those pages in a single command. So you probably have to be a bit more specific here. -- Jens Axboe -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
