Looks good, thanks for fixing this! If you find the time to add support to use block addresses in addition to percentage for specifying the zones, I think it would be quite useful. Thanks for the quick update, Phillip Chen On Wed, Nov 29, 2017 at 12:37 PM, Jens Axboe <axboe@xxxxxxxxx> wrote: > OK, I checked that fix, it seems to do the trick. On top of that, > fio does support empty zones. If I run your script and change > the distribution to: > > dist_str = "zoned:50/5:0/90:50/5" > > to says "50% of access to the first 5% of the drive, nothing to > the middle 90% of the drive, and 50% to the last 5% of the drive", > I get: > > histogram percents = [50.02041783178578, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 49.97958216821422] > > which looks pretty spot on to me. > > I'm going to commit the fix. > > On 11/29/2017 12:33 PM, Jens Axboe wrote: >> We can add empty zones, that's fine, if that already doesn't work. >> >> For your case, can you try and kill this line: >> >> qsort(o->zone_split[ddir], o->zone_split_nr[ddir], sizeof(struct zone_split), zone_cmp); >> >> in options.c:zone_split_ddir(), I have a feeling that's screwing us over. >> >> >> On 11/27/2017 04:18 PM, Phillip Chen wrote: >>> I agree that changing the zoned random distribution parameter to allow >>> absolute sizes would be an elegant way to address my use case. I'm not >>> sure if it would be easier to add that functionality to the zoned >>> distribution or create a new distribution named something like >>> zoned_sectors. I'd also like the ability to have zones that no I/O >>> will fall into. I think the easiest way to do that would be to allow 0 >>> as a valid distribution percentage (I.E. something like >>> random_distribution=zoned:10/10:0/50:30/20:8/30:2/40). Currently it >>> seems I can specify 0 as a distribution percentage, but it doesn't >>> create a zero I/O zone like I would like it to. Although that might be >>> addressed just by making the zoned distribution randomize I/O as >>> expected. >>> Thanks for looking into this, >>> Phillip Chen >>> >>> On Mon, Nov 20, 2017 at 6:57 PM, Jens Axboe <axboe@xxxxxxxxx> wrote: >>>> On 11/20/2017 10:17 AM, Phillip Chen wrote: >>>>> Hello, >>>>> I'm a test engineer at Seagate and we're using FIO to gather some >>>>> performance data. This email has two parts: a bug report and a feature >>>>> request. >>>>> The bug that I'm seeing is that when using the >>>>> random_distribution=zoned argument, the zone order is not honored. So >>>>> using zoned:18/90:7/5:75/5 will not weight IO towards the end of the >>>>> disk but rather towards the beginning. Using zoned:75/5:7/5:18/90 >>>>> apparently gives the same distribution, but also not the correct >>>>> distribution. I've attached a python3.6 script that shows this >>>>> behaviour. Here is the histogram information from running the two >>>>> zoned arguments as described above: >>>>> >>>>> Using fio --name=rand_reads --ioengine=libaio --direct=1 --exitall >>>>> --thread --filename=/dev/sdc --runtime=30 --readwrite=randread >>>>> --iodepth=1 --random_distribution=zoned:18/90:7/5:75/5 --norandommap >>>>> --output-format=terse >>>>> histogram bins = [2302, 25, 25, 30, 33, 36, 26, 32, 29, 37, 21, 21, >>>>> 32, 27, 49, 34, 24, 36, 26, 184] >>>>> histogram percents = [75.99867943215582, 0.8253549026081215, >>>>> 0.8253549026081215, 0.9904258831297458, 1.0894684714427203, >>>>> 1.188511059755695, 0.8583690987124464, 1.0564542753383954, >>>>> 0.9574116870254209, 1.2215252558600198, 0.6932981181908221, >>>>> 0.6932981181908221, 1.0564542753383954, 0.8913832948167713, >>>>> 1.6176956091119181, 1.1224826675470452, 0.7923407065037966, >>>>> 1.188511059755695, 0.8583690987124464, 6.074612083195774] >>>>> >>>>> Using fio --name=rand_reads --ioengine=libaio --direct=1 --exitall >>>>> --thread --filename=/dev/sdc --runtime=30 --readwrite=randread >>>>> --iodepth=1 --random_distribu tion=zoned:75/5:7/5:18/90 >>>>> --norandommap --output-format=terse >>>>> histogram bins = [2306, 25, 25, 30, 33, 36, 26, 32, 29, 37, 21, 21, >>>>> 32, 27, 49, 34, 24, 36, 26, 184] >>>>> histogram percents = [76.03033300362677, 0.8242664029014177, >>>>> 0.8242664029014177, 0.9891196834817013, 1.0880316518298714, >>>>> 1.1869436201780414, 0.8572370590174745, 1.0550609957138146, >>>>> 0.9561490273656446, 1.2199142762940982, 0.6923837784371909, >>>>> 0.6923837784371909, 1.0550609957138146, 0.8902077151335311, >>>>> 1.6155621496867787, 1.1210023079459281, 0.7912957467853611, >>>>> 1.1869436201780414, 0.8572370590174745, 6.0666007253544345] >>>>> >>>>> To run the script, use the -h flag to see usage, but at a minimum >>>>> you'll need to give the device handle to run on as the first argument >>>>> (the workload only does reads). The random_distribution argument is >>>>> set at the top of the file. >>>> >>>> I'll take a look at this tomorrow, that does seem very fishy. >>>> >>>>> Here is my environment information: >>>>> # cat /etc/centos-release >>>>> CentOS Linux release 7.3.1611 (Core) >>>>> # uname -r >>>>> 3.10.0-514.21.1.el7.x86_64 >>>>> I used fio-3.2-13-g40e5f which was the newest version I could see as of today. >>>>> >>>>> As for the feature request: >>>>> I am trying to adapt our current FIO job files for FLEX testing which >>>>> is a new protocol we announced recently >>>>> (http://blog.seagate.com/intelligent/new-flex-dynamic-recording-method-redefines-data-center-hard-drive/) >>>>> that has some requirements on where writes/reads are allowed. I would >>>>> like to have better control where random reads and writes are going >>>>> using the zoned random_distribution setting using sector numbers >>>>> rather than capacity percentages. Would that be a possible feature to >>>>> add? Or is there an existing way to randomly read/write to >>>>> non-contiguous zones on the disk with varying sizes? >>>> >>>> The best way to request something like that is to come up with a logical >>>> way to describe it. That's usually the hardest part, implementing it is >>>> usually not that hard. This is especially important since it has to be >>>> intuitively easy to use for the user, not requiring them to pour too >>>> much over man pages. >>>> >>>> For yours, the zoned setup already supports split ranges for >>>> reads/writes/trims. The change seems to be that you want to give the >>>> zones in absolute sizes instead. It'd be the easiest to extend the >>>> zoning to allow sizes instead. >>>> >>>> -- >>>> Jens Axboe >>>> >> >> > > > -- > Jens Axboe > -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
