Latest commit seems to help, but found one more issue: # blockdev --getsize64 /dev/nvme2n1 800166076416 800166076416B = 763097.8MB Test1: Round io_size to match bs: PASS # ./fio/fio --name=SW_1MB_QD32 --ioengine=libaio --direct=1 --rw=write --iodepth=32 --size=1% --runtime=60s --time_based --numjobs=1 --bs=1m --overwrite=1 --filename=/dev/nvme2n1 SW_1MB_QD32: (g=0): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32 fio-2.3-26-g19dd Starting 1 process Jobs: 1 (f=1): [W(1)] [100.0% done] [0KB/1380MB/0KB /s] [0/1380/0 iops] [eta 00m:00s] SW_1MB_QD32: (groupid=0, jobs=1): err= 0: pid=80311: Thu Jan 14 10:11:36 2016 write: io=84966MB, bw=1415.1MB/s, iops=1415, runt= 60009msec Test2: Loop around max device size and continue IO with fixed runtime: PASS # ./fio/fio --name=SW_1MB_QD32 --ioengine=libaio --direct=1 --rw=write --iodepth=32 --size=100% --runtime=15m --time_based --numjobs=1 --bs=1m --overwrite=1 --filename=/dev/nvme2n1 SW_1MB_QD32: (g=0): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32 fio-2.3-26-g19dd Starting 1 process Jobs: 1 (f=1): [W(1)] [100.0% done] [0KB/1388MB/0KB /s] [0/1388/0 iops] [eta 00m:00s] SW_1MB_QD32: (groupid=0, jobs=1): err= 0: pid=80377: Thu Jan 14 10:28:27 2016 write: io=1231.9GB, bw=1401.6MB/s, iops=1401, runt=900008msec Test3: Loop around max device size and continue IO with fixed total IO, round total IO to bs: FAIL (does not loop around to start LBA) # ./fio/fio --name=SW_1MB_QD32 --ioengine=libaio --direct=1 --rw=write --iodepth=32 --size=100% --io_size=810000000000 --numjobs=1 --bs=1m --overwrite=1 --filename=/dev/nvme2n1 SW_1MB_QD32: (g=0): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32 fio-2.3-26-g19dd Starting 1 process Jobs: 1 (f=0): [W(1)] [98.9% done] [0KB/1453MB/0KB /s] [0/1453/0 iops] [eta 00m:06s] SW_1MB_QD32: (groupid=0, jobs=1): err= 0: pid=81065: Thu Jan 14 10:49:53 2016 write: io=763097MB, bw=1399.5MB/s, iops=1399, runt=545278msec Test4: Loop around max device size and continue IO with fixed total IO, total IO is already aligned to bs: FAIL (does not loop around to start LBA) # ./fio/fio --name=SW_1MB_QD32 --ioengine=libaio --direct=1 --rw=write --iodepth=32 --size=100% --io_size=810675077120 --numjobs=1 --bs=1m --overwrite=1 --filename=/dev/nvme2n1 SW_1MB_QD32: (g=0): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32 fio-2.3-26-g19dd Starting 1 process Jobs: 1 (f=1): [W(1)] [98.6% done] [0KB/1404MB/0KB /s] [0/1404/0 iops] [eta 00m:08s] SW_1MB_QD32: (groupid=0, jobs=1): err= 0: pid=81339: Thu Jan 14 11:01:57 2016 write: io=763097MB, bw=1399.9MB/s, iops=1399, runt=545142msec Regards, Jeff -----Original Message----- From: Jens Axboe [mailto:axboe@xxxxxxxxx] Sent: Thursday, January 14, 2016 9:33 AM To: Jeff Furlong <jeff.furlong@xxxxxxxx>; Jens Rosenboom <j.rosenboom@xxxxxxxx>; Sitsofe Wheeler <sitsofe@xxxxxxxxx> Cc: fio@xxxxxxxxxxxxxxx Subject: Re: time_based option broken On 01/08/2016 12:35 PM, Jeff Furlong wrote: > Good points. Here is further data: > > # blockdev --getsize64 /dev/nvme2n1 > 800166076416 > > # fio --name=SW_1MB_QD32 --ioengine=libaio --direct=1 --rw=write > --iodepth=32 --size=1000m --runtime=60s --time_based --numjobs=1 > --bs=1m --overwrite=1 --filename=/dev/nvme2n1 > SW_1MB_QD32: (g=0): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, > iodepth=32 fio-2.3-11-g5f3b Starting 1 process > Jobs: 1 (f=1): [W(1)] [100.0% done] [0KB/990.2MB/0KB /s] [0/990/0 > iops] [eta 00m:00s] > SW_1MB_QD32: (groupid=0, jobs=1): err= 0: pid=19511: Fri Jan 8 11:08:55 2016 > write: io=58620MB, bw=976.11MB/s, iops=976, runt= 60001msec > > So the above using a fixed --size that is compatible with --bs is one workaround. > > When trying to write more than 100% of the device (wrapping around after end of device), instead of --size=100% we could use a fixed size (where above 800166076416 bytes is 763097.8 MB), even if time_based is not used (use --io_size instead): > > # fio --name=SW_1MB_QD32 --ioengine=libaio --direct=1 --rw=write > --iodepth=32 --size=763097m --io_size=764000m --numjobs=1 --bs=1m > --overwrite=1 --filename=/dev/nvme2n1 > SW_1MB_QD32: (g=0): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, > iodepth=32 fio-2.3-11-g5f3b Starting 1 process > Jobs: 1 (f=1): [W(1)] [100.0% done] [0KB/567.9MB/0KB /s] [0/567/0 > iops] [eta 00m:00s] > SW_1MB_QD32: (groupid=0, jobs=1): err= 0: pid=129928: Thu Jan 7 12:45:52 2016 > write: io=764000MB, bw=600510KB/s, iops=586, runt=1302786msec > > Alternatively, if we simply back out the commit c82ea3d49aa then we can make the 1% or 100% options (--time_based or --io_size) work fine: > > # fio --name=SW_1MB_QD32 --ioengine=libaio --direct=1 --rw=write > --iodepth=32 --size=1% --runtime=60s --time_based --numjobs=1 --bs=1m > --overwrite=1 --filename=/dev/nvme2n1 > SW_1MB_QD32: (g=0): rw=write, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, > iodepth=32 fio-2.3-11-g5f3b Starting 1 process > Jobs: 1 (f=1): [W(1)] [100.0% done] [0KB/954.2MB/0KB /s] [0/954/0 > iops] [eta 00m:00s] > SW_1MB_QD32: (groupid=0, jobs=1): err= 0: pid=21853: Fri Jan 8 11:20:04 2016 > write: io=56931MB, bw=971606KB/s, iops=948, runt= 60001msec So the bug isn't really in the bisected commit, it just changes how fio backs out and then shows another bug. The real bug is that we don't align the new start properly, so we end up with these weird unaligned start offsets that don't work with O_DIRECT IO. Try current -git, I just committed this fix: http://git.kernel.dk/cgit/fio/commit/?id=19ddc35b9b97be5af371bb65e93a4864d1dce7b6 -- Jens Axboe HGST E-mail Confidentiality Notice & Disclaimer: This e-mail and any files transmitted with it may contain confidential or legally privileged information of HGST and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited. If you have received this e-mail in error, please notify the sender immediately and delete the e-mail in its entirety from your system. ��.n��������+%������w��{.n�������^n�r������&��z�ޗ�zf���h���~����������_��+v���)ߣ�
