From: Robert Elliott <elliott@xxxxxxx> In ioengines using mmap() and munmap(), don't call munmap() when doing an IO that accesses the last byte of a file. This affects the dev-dax, libpmem, and mmap ioengines. Example: io 12987 DEBUG fio_libpmem_prep io 12987 io_u->offset 7516061696 : fdd->libpmem_off 0 : io_u->buflen 131072 : fdd->libpmem_sz 7516192768 io 12987 munmap addr=0x7f5600000000 size=0x1c0000000 That accesses the last 128 KiB of a 7 GiB file, triggering an munmap because the comparison to 7 GiB fails: 7516061696 + 131072 < 7516192768 Change < to <= so that doesn't occur unless an overflow really happens. In one test in linux, this increases performance from 5 GB/s to 20 GB/s as fio stops spending time in mmap() page table mapping spinlocks and instead spends its time in memmove(). The perf top results improve from: 36.81% [kernel] [k] queued_spin_lock_slowpath 24.26% libc-2.26.so [.] __memmove_avx_unaligned_erms 4.09% [kernel] [k] __radix_tree_lookup 3.85% [kernel] [k] find_next_iomem_res 3.70% [kernel] [k] _raw_spin_lock_irq 3.48% [kernel] [k] down_read to: 79.76% libc-2.26.so [.] __memmove_avx_unaligned_erms 3.23% libc-2.26.so [.] vfprintf --- Changes in v2: applied same change to dev-dax.c which has the same pattern engines/dev-dax.c | 2 +- engines/libpmem.c | 2 +- engines/mmap.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/engines/dev-dax.c b/engines/dev-dax.c index b1f91a40..caae1e09 100644 --- a/engines/dev-dax.c +++ b/engines/dev-dax.c @@ -157,7 +157,7 @@ static int fio_devdax_prep(struct thread_data *td, struct io_u *io_u) * It fits within existing mapping, use it */ if (io_u->offset >= fdd->devdax_off && - io_u->offset + io_u->buflen < fdd->devdax_off + fdd->devdax_sz) + io_u->offset + io_u->buflen <= fdd->devdax_off + fdd->devdax_sz) goto done; /* diff --git a/engines/libpmem.c b/engines/libpmem.c index 3f4e44fb..30387845 100644 --- a/engines/libpmem.c +++ b/engines/libpmem.c @@ -430,7 +430,7 @@ static int fio_libpmem_prep(struct thread_data *td, struct io_u *io_u) io_u->buflen, fdd->libpmem_sz); if (io_u->offset >= fdd->libpmem_off && - (io_u->offset + io_u->buflen < + (io_u->offset + io_u->buflen <= fdd->libpmem_off + fdd->libpmem_sz)) goto done; diff --git a/engines/mmap.c b/engines/mmap.c index 51606e12..77556588 100644 --- a/engines/mmap.c +++ b/engines/mmap.c @@ -137,7 +137,7 @@ static int fio_mmapio_prep(struct thread_data *td, struct io_u *io_u) * It fits within existing mapping, use it */ if (io_u->offset >= fmd->mmap_off && - io_u->offset + io_u->buflen < fmd->mmap_off + fmd->mmap_sz) + io_u->offset + io_u->buflen <= fmd->mmap_off + fmd->mmap_sz) goto done; /* -- 2.14.3 -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
