Gary, I think the io_replay file wait is referring to a "start_delay" type of wait, not periods between ops, but could be wrong. I had a similar need, and found blktrace replay to produce reasonable timings. So I created a python3 script to generate the blktrace binary replay file. Below is a snippet to create blktrace type binary entries. It's limited to only what was working on, but I'd love to have some help to fill out the known operations and such. Hopefully this will help you proceed though. (Note: there is an fio bug with replaying Trim operations (https://github.com/axboe/fio/issues/771) Regards, Kris ################################################# import struct import binascii from enum import Enum class Ops(Enum): # action value observed in blkparse binary output # unclear on bit meanings, so unclear, but works currently at QD=1 # TODO: find bit meanings and fill out with other ops # TODO: Might be other implications at higher QDs - experiment TRIM = 0x2012 WRITESYNC = 0x001a READ = 0x0011 class BlkParserBinFormat(): magic = 0x65617407 def __init__(self): self.fmt = struct.Struct( # defintion from FIO code: blktrace_api.h # struct blk_io_trace { "I " # __u32 magic; /* MAGIC << 8 | version */ "I " # __u32 sequence; /* event number */ "Q " # __u64 time; /* in nanoseconds */ "Q " # __u64 sector; /* disk offset */ "I " # __u32 bytes; /* transfer length */ "H H " # __u32 action; /* what happened */ Split into action (0x0001 is Q) and command "I " # __u32 pid; /* who did it */ "I " # __u32 device; /* device identifier (dev_t) */ "I " # __u32 cpu; /* on what cpu did it happen */ "H " # __u16 error; /* completion error */ "H " # __u16 pdu_len; /* length of data after this trace */ ) self.curSequence = 0 def newEntry(self, time, sector, xferlen, operation): self.curSequence += 1 logger.debug(Lazy(lambda: pp.pformat((self.magic, self.curSequence, time, sector, xferlen, 0x0001, operation, 0, 0x10310001, 0, 0, 0)))) return self.fmt.pack(self.magic, self.curSequence, time, sector, xferlen, 0x0001, operation, 0, 0x10310001, 0, 0, 0) ######################################## Kris Davis -----Original Message----- From: fio-owner@xxxxxxxxxxxxxxx <fio-owner@xxxxxxxxxxxxxxx> On Behalf Of Gary Little Sent: Saturday, May 18, 2019 4:10 PM To: fio@xxxxxxxxxxxxxxx Subject: Issues with read_iolog trace replay Hello, I am trying to simulate bursty traffic with fo. I know that I can do this by tracing a bursty app and then doing a blktrace replay, but I would like to do something programatic. I had hoped that I could create an io_replay file that looks like this which would read, then pause for 100ms before issuing the next read. Once working I could then vary (lower) the "wait" period to generate the bursts. But what seems to happen is that all the "wait" lines are executed sequentially, followed by all the "reads" sequentially, without the 100ms pause in-between. FWIW, using blktrace input works as expected, and replays the trace at the same iO rate as the traced workload. [root@arches filebench]# /usr/local/bin/fio --version fio-3.13 [root@arches filebench]# /usr/local/bin/fio --name=replay --read_iolog=read100.csv [gary@arches filebench]$ cat read100.csv fio version 2 iolog /dev/sdc add /dev/sdc open /dev/sdc read 16384 4096 /dev/sdc wait 100000 1 /dev/sdc read 16384 4096 /dev/sdc wait 100000 1 /dev/sdc read 16384 4096 /dev/sdc wait 100000 1 /dev/sdc read 16384 4096 /dev/sdc wait 100000 1 /dev/sdc read 16384 4096 /dev/sdc wait 100000 1 /dev/sdc read 16384 4096 /dev/sdc wait 100000 1 /dev/sdc read 16384 4096 /dev/sdc wait 100000 1 /dev/sdc read 16384 4096 /dev/sdc wait 100000 1 /dev/sdc read 16384 4096 /dev/sdc wait 100000 1 /dev/sdc read 16384 4096 /dev/sdc wait 100000 1 /dev/sdc close
