> Thanks for the info. I don't know of anything block or nvme specific that might > explain an order of magnitude perf difference. > > Could you try the same test without the filesytems? You mentioned using mdraid, > so try '--filename=/dev/mdX'. If that also shows similiar performance > difference, try using one of your nvme member drives directly, like > '--filename=/dev/nvme1n1'. That should isolate which subsystem is contributing > to the difference. > With Keith's suggestions you can always take the perf numbers and compare before and after mainly for nvme_queue_rq() and nvme_irq(). -ck