On 16/6/20 1:21 a. m., Sitsofe Wheeler wrote:
Hi,
On Tue, 16 Jun 2020 at 04:57, Hamilton Tobon Mosquera
<htobonm@xxxxxxxxxxxx> wrote:
I've been using and analyzing io_uring's performance with fio. I'm
trying to replicate the results that Jens Axboe shows when he presents
io uring in Kernel Recipes 2019. The problem is that I'm not even closer
to that, nor that behavior. I have a server with two Intel Optanes (he
mentions that he had to use two devices), but I think the problem is
that I'm not using the proper fio parameter combination. Can you please
tell me what fio configuration is needed to get 1.6 million iops with
only one thread?, or at least that same behavior, not necesarily the 1.6
million iops. I'm comparing io_uring in polling mode vs libaio, but I
get the same iops with both of them, about 400K iops, issuing requests
to both devices through separate files.
Have you seen https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fitpeernetwork.intel.com%2Ftuning-performance-intel-optane-ssds-linux-operating-systems%2F&data=01%7C01%7Chtobonm%40eafit.edu.co%7C9fc53beeaf8344cbee6a08d811b52051%7C99f7b55e9cbe467b8143919782918afb%7C0&sdata=ipmfFDLCOnP%2BCZ3hN3aiKlEgREFHxaq5rmCeI2buAL0%3D&reserved=0
by Intel's Frank Ober? For polling to actually do its work you need to
ensure your kernel has been configured to do polling too. It would
also be good to see what you were using as your fio command line (and
I guess which kernel you are using).
Thanks for your answer.
Yes I saw that post and followed it. The CPU is in performance mode. I
activated polling in the nvme driver by loading it with the
poll_queues=N parameter, where N is the number of physical cores of the
CPU. I'm issuing 4K random and sequential reads, at different queue
depths (1,2,4,8,128,256). The peak performance is reached at a queue
depth of 128, as shown by Jens Axboe, but I'm still far away to get
those behaviors where io_uring over performs libaio. This is the jobfile:
[global]
ioengine=io_uring
hipri
direct=1
thread=1
buffered=0
size=100%
randrepeat=0
time_based
ramp_time=0
norandommap
refill_buffers
log_max_value=1
log_avg_msec=1000
group_reporting
percentile_list=50:60:70:80:90:95:99
[optanes]
name=test-optanes
bs=4k
rw=randread
numjobs=1
filename=/mnt/optane0/fiotest:/mnt/optane1/fiotest
iodepth=32
runtime=240
Thanks again. I will try with the jobfile that Frank Ober sent.
Hamilton.
--
Sitsofe | https://nam01.safelinks.protection.outlook.com/?url=http:%2F%2Fsucs.org%2F~sits%2F&data=01%7C01%7Chtobonm%40eafit.edu.co%7C9fc53beeaf8344cbee6a08d811b52051%7C99f7b55e9cbe467b8143919782918afb%7C0&sdata=teb4flHuOeyB%2BW%2BSujPM9AV90xmry12yQ3ieYR6YwGA%3D&reserved=0
La información contenida en este correo electrónico está dirigida únicamente a su destinatario y puede contener información confidencial, material privilegiado o información protegida por derecho de autor. Está prohibida cualquier copia, utilización, indebida retención, modificación, difusión, distribución o reproducción total o parcial. Si usted recibe este mensaje por error, por favor contacte al remitente y elimínelo. La información aquí contenida es responsabilidad exclusiva de su remitente por lo tanto la Universidad EAFIT no se hace responsable de lo que el mensaje contenga. The information contained in this email is addressed to its recipient only and may contain confidential information, privileged material or information protected by copyright. Its prohibited any copy, use, improper retention, modification, dissemination, distribution or total or partial reproduction. If you receive this message by error, please contact the sender and delete it. The information contained herein is the sole responsibility of the sender therefore Universidad EAFIT is not responsible for what the message contains.
