Hi Marcus,
Am 18.11.2013 23:51, schrieb mdw@xxxxxxxxxxxx:
On Mon, Nov 18, 2013 at 02:38:42PM +0100, Stefan Priebe - Profihost AG wrote:
You may actually be doing O_SYNC - recent kernels implement O_DSYNC,
but glibc maps O_DSYNC into O_SYNC. But since you're writing to the
block device this won't matter much.
No difference regarding O_DSYNC or O_SYNC the values are the same. Also
I'm using 3.10.19 as a kernel so it is recent enough.
I believe the effect of O_DIRECT by itself is just to bypass the buffer
cache, which is not going to make much difference for your dd case.
(It will mainly affect other applications that are also using the
buffer cache...)
> O_SYNC should be causing the writes to block until a response
> is received from the disk. Without O_SYNC, the writes will
> just queue operations and return - potentially very fast.
> Your dd is probably writing enough data that there is some
> throttling by the system as it runs out of disk buffers and
> has to wait for some previous data to be written to the drive,
> but the delay for any individual block is not likely to matter.
> With O_SYNC, you are measuring the delay for each block directly,
> and you have absolutely removed the ability for the disk to
> perform any sort of parallism.
That's correct but ceph uses O_DSYNC for his journal and may be other
stuff so it is important to have devices performing well with O_DSYNC.
Sounds like the intel 530 is has a much larger block write latency,
but can make up for it by performing more overlapped operations.
You might be able to vary this behavior by experimenting with sdparm,
smartctl or other tools, or possibly with different microcode in the drive.
Which values or which settings do you think of?
Greets
Stefan
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
