On Sunday, February 20, 2011, Jens Axboe <jaxboe@xxxxxxxxxxxx> wrote: > On 2011-02-17 03:14, john smith wrote: >> I have a couple of questions: >> >> 1) if it is normal (and if so, why) that while running a number of identical jobs (e.g 4), each bound to a different CPU (cpu_allowed=i) and each performing sequential (bs=512) reads on (e.g. 4) different HD, fio reports IOPS numbers that are highly unbalanced among HD with one HD reporting more than 7x of the IOPS of the minimum IOPS HD. >> Different runs may show most of the IOPS on a different HD, so it's not that one HD is faster. >> >> Here are some possibly interesting parameters: >> >> ioengine=libaio >> iodepth=32 >> direct=1 > > There should be very little variance between runs, and between jobs. > Each are separate threads/processes, so most of the variance you will > see will be due to system "artifacts". So I'd suggest you look there. If > you don't pin to specific CPUs, do you see smaller or larger > differences? > > Are you doing random or sequential IO? Could also be timing differences > in submission and dispatch from the IO scheduler. Is hyper threading enabled? Could be running some of the jobs on the same CPU core. Does it track to different sockets? > >> 2) why increasing "iodepth' (eg to 512) decreses the IOPS results significantly (by 10x)? > > You probably don't want a larger depth in fio than the system supports. > > -- > Jens Axboe > > -- > To unsubscribe from this list: send the line "unsubscribe fio" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
