On 23 January 2017 at 19:40, Tobias Oberstein <tobias.oberstein@xxxxxxxxx> wrote: > Am 23.01.2017 um 20:13 schrieb Sitsofe Wheeler: >> >> On 23 January 2017 at 18:33, Tobias Oberstein >> <tobias.oberstein@xxxxxxxxx> wrote: >>> >>> libaio is nowhere near what I get with engine=sync and high job counts. >>> Mmh. >>> Plus the strange behavior. >> >> Have you tried batching the IOs and controlling how much are you >> reaping at any one time? See >> >> http://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-iodepth_batch_submit >> for some of the options for controlling this... > > Thanks! Nice. > > For libaio, and with all the hints applied (no 4k sectors yet), I get (4k > randread) > > Individual NVMes: iops=7350.4K > MD (RAID-0) over NVMes: iops=4112.8K > > The going up and down of IOPS is gone. > > It's becoming more apparent I'd say, that tthere is a MD bottleneck though. If you're "just" trying for higher IOPS you can also try gtod_reduce (see http://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-gtod_reduce ). This subsumes things like disable_lat but you'll get fewer and less accurate measurement stats back. With libaio userspace reap (http://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-userspace_reap ) can sometimes nudge numbers up but at the cost of CPU. -- Sitsofe | http://sucs.org/~sits/ -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
