Am 31.05.2012 15:21, schrieb Yann Dupont:
On 31/05/2012 09:30, Yehuda Sadeh wrote:
On Thu, May 31, 2012 at 12:10 AM, Stefan Priebe - Profihost AG
<s.priebe@xxxxxxxxxxxx> wrote:
But very strangely it's now rbd that isn't stable ?!
root@label5:~# rados -p rbd bench 20 write -t 16
Maintaining 16 concurrent writes of 4194304 bytes for at least 20 seconds.
sec Cur ops started finished avg MB/s cur MB/s last lat avg lat
0 0 0 0 0 0 - 0
1 16 155 139 555.87 556 0.046232 0.109021
2 16 250 234 467.923 380 0.046793 0.0985316
3 16 250 234 311.955 0 - 0.0985316
4 16 250 234 233.965 0 - 0.0985316
5 16 250 234 187.173 0 - 0.0985316
6 16 266 250 166.645 16 0.038083 0.175697
7 16 266 250 142.839 0 - 0.175697
8 16 441 425 212.475 350 0.05512 0.298391
9 16 476 460 204.422 140 0.04372 0.280483
10 16 531 515 205.976 220 0.125076 0.309449
11 16 734 718 261.06 812 0.127582 0.244134
12 16 795 779 259.637 244 0.065158 0.234156
13 16 818 802 246.742 92 0.054514 0.241704
14 16 830 814 232.546 48 0.044386 0.239006
15 16 837 821 218.909 28 3.41523 0.267521
16 16 1043 1027 256.721 824 0.04898 0.248212
17 16 1147 1131 266.088 416 0.048591 0.232725
18 16 1147 1131 251.305 0 - 0.232725
19 16 1202 1186 249.657 110 0.081777 0.25501
min lat: 0.033773 max lat: 5.92059 avg lat: 0.245711
sec Cur ops started finished avg MB/s cur MB/s last lat avg lat
20 16 1296 1280 255.97 376 0.053797 0.245711
21 9 1297 1288 245.305 32 0.708133 0.248248
22 9 1297 1288 234.155 0 - 0.248248
23 9 1297 1288 223.975 0 - 0.248248
24 9 1297 1288 214.643 0 - 0.248248
25 9 1297 1288 206.057 0 - 0.248248
26 9 1297 1288 198.131 0 - 0.248248
Total time run: 26.829870
Total writes made: 1297
Write size: 4194304
Bandwidth (MB/sec): 193.367
Average Latency: 0.295922
Max latency: 7.36701
Min latency: 0.033773
Strange. I'm wondering if this has something to do with cache (that is,
operation I could have done before on nodes, as all my nodes are just
freshly rebooted).
Please test setting these values on all OSDs and Clients:
sysctl -w net.ipv4.tcp_rmem="4096 87380 514873"
sysctl -w net.ipv4.tcp_wmem="4096 16384 514873"
Stefan
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
