Ok, thanks. Now I built latest fio available :-) dl560g10spmem01:~ # /usr/local/bin/fio --filename=/dev/dax0.0 --rw=randrw --norandommap --randrepeat=0 --ioengine=mmap --bssplit=4k/4:8k/7:16k/7:32k/15:64k/65:128k/1:256k/1 --rwmixread=5 --iodepth=1 --numjobs=16 --runtime=1800 --group_reporting --name=4-rand-rw-3xx --size=290g --numa_cpu_nodes=0 4-rand-rw-3xx: (g=0): rw=randrw, bs=(R) 4096B-256KiB, (W) 4096B-256KiB, (T) 4096B-256KiB, ioengine=mmap, iodepth=1 ... fio-3.2-64-gaf7fb Starting 16 processes Jobs: 16 (f=16): [m(16)][8.0%][r=2138MiB/s,w=39.7GiB/s][r=42.2k,w=799k IOPS][eta 01m:44s] From perf top, Samples: 1M of event 'cycles:ppp', Event count (approx.): 503978848613 Overhead Shared Object Symbol ◆ 93.34% libc-2.22.so [.] __memcpy_avx_unaligned Just can't drill it down, __memcpy_avx_unaligned /lib64/libc-2.22.so Huh? No selection. Report to linux-kernel@xxxxxxxxxxxxxxx While other functions can, get_io_u /usr/local/bin/fio 0.08 │ xor %eax,%esi 0.01 │ shl $0x6,%rsi 0.02 │ and $0x1c0,%esi 0.00 │ sub %rax,%rsi 0.12 │ add %rdx,%rsi 0.00 │ sub %rax,%rdx │ nop │ /* │ * Fill offset into start of cacheline, time into end │ * of cacheline │ */ │ iptr = (void *) p + (offset << 6); │ *iptr = boffset; 16.01 │ 230: mov %rax,(%rsi,%rax,1) │ │ iptr = (void *) p + 64 - 2 * sizeof(uint64_t); │ iptr[0] = io_u->start_time.tv_sec; 7.03 │ mov 0x0(%rbp),%rcx 13.52 │ mov %rcx,0x30(%rdx,%rax,1) -----Original Message----- From: Jens Axboe [mailto:axboe@xxxxxxxxx] Sent: Thursday, November 30, 2017 11:24 PM To: Gavriliuk, Anton (HPS Ukraine) <anton.gavriliuk@xxxxxxx>; Robert Elliott (Persistent Memory) <elliott@xxxxxxx>; Rebecca Cran <rebecca@xxxxxxxxxxxx>; Sitsofe Wheeler <sitsofe@xxxxxxxxx> Cc: fio@xxxxxxxxxxxxxxx; Kani, Toshimitsu <toshi.kani@xxxxxxx> Subject: Re: fio 3.2 On 11/30/2017 07:17 AM, Gavriliuk, Anton (HPS Ukraine) wrote: > It's there any chance to fix it ? > > dl560g10spmem01:/var/work # /usr/local/bin/fio --filename=/dev/dax0.0 > --rw=randrw --refill_buffers --randrepeat=0 --ioengine=mmap > --bssplit=4k/4:8k/7:16k/7:32k/15:64k/65:128k/1:256k/1 --rwmixread=5 > --iodepth=1 --numjobs=16 --runtime=1800 --group_reporting > --name=4-rand-rw-3xx --size=290g > 4-rand-rw-3xx: (g=0): rw=randrw, bs=(R) 4096B-256KiB, (W) > 4096B-256KiB, (T) 4096B-256KiB, ioengine=mmap, iodepth=1 ... > fio-2.99 > Starting 16 processes > 4-rand-rw-3xx: failed to get file size of /dev/dax0.0 > > 4-rand-rw-3xx: failed to get file size of /dev/dax0.0 > > 4-rand-rw-3xx: failed to get file size of /dev/dax0.0 > > 4-rand-rw-3xx: failed to get file size of /dev/dax0.0 > > 4-rand-rw-3xx: failed to get file size of /dev/dax0.0 > > 4-rand-rw-3xx: failed to get file size of /dev/dax0.0 > > 4-rand-rw-3xx: failed to get file size of /dev/dax0.0 Should already be fixed as of yesterday morning. -- Jens Axboe ��.n��������+%������w��{.n�������^n�r������&��z�ޗ�zf���h���~����������_��+v���)ߣ�