On 07/02/2019 19:47, Marc Roos wrote:
Is this difference not related to chaching? And you filling up some
cache/queue at some point? If you do a sync after each write, do you
have still the same results?
No, the slow operations are slow from the very beginning. It's not about
filling a buffer/cache somewhere. I'm guessing the slow operations
trigger several synchronous writes to the underlying OSDs, while the
fast ones don't. But I'd like to know more about why exactly there is
this significant performance hit to truncation operations vs. normal writes.
To give some more numbers:
echo test | dd of=b conv=notrunc
This completes extremely quickly (microseconds). The data obviously
remains in the client cache at this point. This is what I want.
echo test | dd of=b conv=notrunc,fdatasync
This runs quickly until the fdatasync(), then that takes ~12ms, which is
about what I'd expect for a synchronous write to the underlying HDDs. Or
maybe that's two writes?
echo test | dd of=b
This takes ~10ms in the best case for the open() call (sometimes 30-40
or even more), and 6-8ms for the write() call.
echo test | dd of=b conv=fdatasync
This takes ~10ms for the open() call, ~8ms for the write() call, and
~18ms for the fdatasync() call.
So it seems like truncating/recreating an existing file introduces
several disk I/Os worth of latency and forces synchronous behavior
somewhere down the stack, while merely creating a new file or writing to
an existing one without truncation does not.
--
Hector Martin (hector@xxxxxxxxxxxxxx)
Public Key: https://mrcn.st/pub
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
