Despite the desire to have fio work with Host Managed devices independent of the engine, a Host Managed device operation is different than a traditional block device, and the kernel also has to manage it differently. Thus, I would still recommend creating a new IO engine for use with Host Managed devices, we would not want the additional overhead associated with SMR to impact the standard aio engine. We have mostly used the fio SG engine along with external operations in testing of SMR. Here is the way we envision a new fio engine might work with Host Managed devices: The SMR device supports report zone operations. Fio engine performs report zones for all zones on the drive prior to testing, and loads the following items to a list or dictionary for each zone: 1. Zone start LBA 2. Type of zone a. Random allowed (conventional HDD format) b. Sequential only (SMR format) c. Sequential preferred (for device managed products) 3. Zone Condition: a. Empty b. Full c. Opened 4. Length of zone (usually is the same for all zones, but would need this information for completeness) 5. Current Write pointer Option for random reads: since the read LBA and length must be less than the current zone write pointer, and cannot cross zone boundaries: Randomly choose from open or full zones. Then choose a random LBA prior to the write pointer within the zone (while accounted for operation size and zone boundary): 1. If zone type is random allowed with a write pointer of 0xffffffffffff then as long as the write pointer does not cross the zone boundary the read is valid 2. Sequential only zones a. Full zones could be read from without restrictions as long as read does not cross zone boundaries b. Opened zones would need to verify the following i. Read + length of command is less then write pointer for zone ii. Does not cross zone boundary Option for random writes: On a write command, Allow the engine to create a write command to a random zone, then choose a somewhat random LBA based on zone type and current write pointer Check zone type: 1. If random allowed, as long as length of command does not cross zone boundary, preform write to random LBA 2. If sequential, check LBA against write pointer and change LBA to write pointer location within same zone a. If length of write crosses zone boundary reset write pointer to zone and perform write at new write pointer b. Update dictionary with new write pointer We may want options for how/when the write pointer is reset after a zone is full: By default, we expect the write pointer to be reset only when a new write is created to a full zone. We're unsure if anyone might wish for other handling. Also need an options for implicit (drives handles) versus explicit (host software handles) open/close of zones Kris Davis -----Original Message----- From: fio-owner@xxxxxxxxxxxxxxx [mailto:fio-owner@xxxxxxxxxxxxxxx] On Behalf Of Phillip Chen Sent: Tuesday, March 6, 2018 12:59 PM To: fio@xxxxxxxxxxxxxxx Subject: ZBC/FLEX FIO addition ideas Hello again, We are interested in making changes to the Seagate branch of FIO that ideally would be widely picked up that would allow FIO to run well on Host Managed/Zoned Block Command (ZBC) drives. Host Managed drives have some simple restrictions which make running workloads on them very challenging. The main hurdles to overcome are all writes must be at its zone write pointer, all reads must be below its zone write pointer, and reads or writes can be over more than one zone. The largest difficulty is to get random performance data. To do this we will need to add the capability to write randomly between zones (but sequentially within zones) and only read what has been written. There seem to be two approaches that could work for this and I'd appreciate any advice as to which approach to pursue. Approach 1: We could let FIO pick I/O offsets as it usually does, but change those offsets into legal ones before they reach the drive. Approach 2: We create a random Host Managed specific workload which will be aware of where it is legal to read and write and will only pick offsets that are valid. Additionally, we would later make some small changes to the existing sequential workloads that would prevent I/O over zone boundaries and do some preparation (writing over zones we are planning to read and resetting the write pointers of zones we want to write over). I am leaning towards this approach because it makes more sense to me. Either way, we will need to implement some way of keeping track of where it is legal to read and write as well as send some new commands like get zone information and reset write pointer(s). Again, it seems like there are two viable approaches. Approach 1: We can create a new I/O engine which handles all the I/O as well as any additional ZBC specific commands like get zone information and resetting write pointers. However, we would like to avoid this option because our customers to use the I/O engine they are comfortable with. Approach 2: We create a layer above the I/O engine which watches all of the I/O and keeps track of the write pointers on the drive as well as issuing any additional ZBC commands as necessary. Thank you, Phillip Chen -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html ��.n��������+%������w��{.n�������^n�r������&��z�ޗ�zf���h���~����������_��+v���)ߣ�
