-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On 2012-04-04 00:39, Danny Kukawka wrote:
> Hi,
>
> we try to run fio against raw RBD devices on a ceph cluster. Here are
> the parameter we use to run fio:
>
> [test]
> filename=/dev/rbd0
> time_based
> runtime=3h
> rw=randrw
> rwmixread=80
> rwmixwrite=20
> blocksize_range=4k-1024k
> ioengine=libaio
> iodepth=64
> direct=1
> verify=crc32c-intel
> verify_fatal=1
> verify_async=16
> lockfile=exclusive
>
> We see some problems:
>
> 1) fio reports higher READ speeds than the used 1Gbit network interface
> could provide (the interface isn't even working to capacity in this
> situation). That happens if we use --rwmixread/rwmixwrite:
>
> READ: io=91560MB, aggrb=156206KB/s, minb=159955KB/s,
> maxb=159955KB/s, mint=600216msec, maxt=600216msec
> WRITE: io=22944MB, aggrb=39143KB/s, minb=40083KB/s, maxb=40083KB/s,
> mint=600216msec, maxt=600216msec
OK that's very odd. How are these numbers matching up with watching
vmstat 1 while the job runs? I've never seen fio over-report before.
> 2) fio 2.0.6 and also the current git tends to fail with this error:
>
> fio 2.0.6
> Starting 1 process
> fio: iolog.c:242: log_io_piece: Assertion `ipo->len == __ipo->len'
> failed.:08m:31s]
>
> It seems that we can prevent it when we use the --size parameter, but we
> would like to prevent the usage of the parameter, if possible
Strangely, I had two reports of this today. Did a bit of digging, and
it's due to this commit:
commit c04e4661e4da3b6079f415897e4507cf8e610c54
Author: Daniel Ehrenberg <dehrenberg@xxxxxxxxxx>
Date: Fri Mar 16 18:54:15 2012 +0100
time_based: Avoid restarting main I/O loop
I think what is going on here is that your job is time based, and when
we fail getting a new offset, we clear the offset bitmap. But we could
have verifies pending at this point, we'd need to prune this out as
well.
> 3) fio always reports minb == maxb and aggrb < minb as you can see
> above. From my understanding this would be false info.
Yeah, it's a cosmetic thing. I've never really taken the time to look at
that...
> 4) If we define rw=randrw, rwmixread=80, rwmixwrite=20 and the verify
> options, what does the READ process if there isn't enough data written
> out yet? Which data does READ read and verify?
If you have a split workload like that and are also doing verifies, the
read can be either a read verify or just an offset read. In the latter
case, it doesn't verify anything. It'll just read a block (or multiple
blocks) at some offset instead of writing it.
- --
Jens Axboe
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQIcBAEBCAAGBQJPfKXtAAoJEPfTWPspceCmqeMP/1p+btObGgcYDgCDa5ohSi/D
usBx/MYkxhgyeleC8ketcC0QVl9qU9dW4krUso1Fe7lyYvjtS+vyvu/ayxISY6hp
CJ0c6m/py+jNDZNANgQQNR0bC8pZ03GbcJnAg0Ari8hewRu8oF2s6razcpr9PstT
nhg0wSrrx+1LNalfYk0GmQe7VDX8QNgOvcBp+gN2v1R9sxYjvR86F6MGBib19LeR
g6fwnQ/zBQRHHVCY3e6AQrozLJk47QOfdwF8pIKTkxVcOCK51Qynbi5YK6zxArto
jFaCgdbQ+DxvUSH6korIaQv3PLmf2wmokQz7TmTjxYQyz8Q6RMW3M/ZdcZlxqHhe
VkBIxC/JYaU97wpLI+Y4XqMDJr+3vrp8JaS4DPrlb7+hNY4sDMFqKH9MCRy3Th1v
12/WZy244SVtqIF9iboS4Gx9QbxfQzZfL5tpQcJaormknGKaTpP3YHZ//egeI/pK
tZvZ34l8CRHS58wfWyh4qPaYyy/hjwoojNJ0/CZGFQnUa6tH6hnWKhRX++oVsEP7
24FbqI2xLG3tBTDMeQIYg0/4SZo65D10vvLF8zzvyqy9hhaZJTon2AmnxpOsm+EA
pUSDWmB7OdpbnIPwA8PpK0sATxBUA638S8UmGFYjqubjwCWpKr5x6HPR6aVj+XN+
O5FfQel0Tz8w4g+ARjju
=cM7O
-----END PGP SIGNATURE-----
--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
