25Gbe network. Servers have ConnectX-4 Pro, across a router, since L2 is terminated as the ToR:
100000 packets transmitted, 100000 received, 0% packet loss, time 1926ms
rtt min/avg/max/mdev = 0.013/0.013/0.205/0.004 ms, ipg/ewma 0.019/0.014 ms
Warren Wang
On 1/22/18, 4:06 PM, "ceph-users on behalf of Marc Roos" <ceph-users-bounces@xxxxxxxxxxxxxx on behalf of M.Roos@xxxxxxxxxxxxxxxxx> wrote:
ping -c 100000 -f
ping -M do -s 8972
10Gb ConnectX-3 Pro, DAC + Vlan
rtt min/avg/max/mdev = 0.010/0.013/0.200/0.003 ms, ipg/ewma 0.025/0.014
ms
8980 bytes from 10.0.0.11: icmp_seq=3 ttl=64 time=0.144 ms
8980 bytes from 10.0.0.11: icmp_seq=4 ttl=64 time=0.205 ms
8980 bytes from 10.0.0.11: icmp_seq=5 ttl=64 time=0.248 ms
8980 bytes from 10.0.0.11: icmp_seq=6 ttl=64 time=0.281 ms
8980 bytes from 10.0.0.11: icmp_seq=7 ttl=64 time=0.187 ms
8980 bytes from 10.0.0.11: icmp_seq=8 ttl=64 time=0.121 ms
I350 Gigabit + bond
rtt min/avg/max/mdev = 0.027/0.038/0.211/0.006 ms, ipg/ewma 0.050/0.041
ms
8980 bytes from 192.168.0.11: icmp_seq=1 ttl=64 time=0.555 ms
8980 bytes from 192.168.0.11: icmp_seq=2 ttl=64 time=0.508 ms
8980 bytes from 192.168.0.11: icmp_seq=3 ttl=64 time=0.514 ms
8980 bytes from 192.168.0.11: icmp_seq=4 ttl=64 time=0.555 ms
-----Original Message-----
From: Nick Fisk [mailto:nick@xxxxxxxxxx]
Sent: maandag 22 januari 2018 12:38
To: ceph-users@xxxxxxxxxxxxxx
Subject: Re: What is the should be the expected latency of
10Gbit network connections
Anyone with 25G ethernet willing to do the test? Would love to see what
the latency figures are for that.
From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of
Maged Mokhtar
Sent: 22 January 2018 11:28
To: ceph-users@xxxxxxxxxxxxxx
Subject: Re: What is the should be the expected latency of
10Gbit network connections
On 2018-01-22 08:39, Wido den Hollander wrote:
On 01/20/2018 02:02 PM, Marc Roos wrote:
If I test my connections with sockperf via a 1Gbit switch I
get around
25usec, when I test the 10Gbit connection via the switch I
have around
12usec is that normal? Or should there be a differnce of 10x.
No, that's normal.
Tests with 8k ping packets over different links I did:
1GbE: 0.800ms
10GbE: 0.200ms
40GbE: 0.150ms
Wido
sockperf ping-pong
sockperf: Warmup stage (sending a few dummy messages)...
sockperf: Starting test...
sockperf: Test end (interrupted by timer)
sockperf: Test ended
sockperf: [Total Run] RunTime=10.100 sec; SentMessages=432875;
ReceivedMessages=432874
sockperf: ========= Printing statistics for Server No: 0
sockperf: [Valid Duration] RunTime=10.000 sec;
SentMessages=428640;
ReceivedMessages=428640
sockperf: ====> avg-lat= 11.609 (std-dev=1.684)
sockperf: # dropped messages = 0; # duplicated messages = 0; #
out-of-order messages = 0
sockperf: Summary: Latency is 11.609 usec
sockperf: Total 428640 observations; each percentile contains
4286.40
observations
sockperf: ---> <MAX> observation = 856.944
sockperf: ---> percentile 99.99 = 39.789
sockperf: ---> percentile 99.90 = 20.550
sockperf: ---> percentile 99.50 = 17.094
sockperf: ---> percentile 99.00 = 15.578
sockperf: ---> percentile 95.00 = 12.838
sockperf: ---> percentile 90.00 = 12.299
sockperf: ---> percentile 75.00 = 11.844
sockperf: ---> percentile 50.00 = 11.409
sockperf: ---> percentile 25.00 = 11.124
sockperf: ---> <MIN> observation = 8.888
sockperf: Warmup stage (sending a few dummy messages)...
sockperf: Starting test...
sockperf: Test end (interrupted by timer)
sockperf: Test ended
sockperf: [Total Run] RunTime=1.100 sec; SentMessages=22065;
ReceivedMessages=22064
sockperf: ========= Printing statistics for Server No: 0
sockperf: [Valid Duration] RunTime=1.000 sec;
SentMessages=20056;
ReceivedMessages=20056
sockperf: ====> avg-lat= 24.861 (std-dev=1.774)
sockperf: # dropped messages = 0; # duplicated messages = 0; #
out-of-order messages = 0
sockperf: Summary: Latency is 24.861 usec
sockperf: Total 20056 observations; each percentile contains
200.56
observations
sockperf: ---> <MAX> observation = 77.158
sockperf: ---> percentile 99.99 = 54.285
sockperf: ---> percentile 99.90 = 37.864
sockperf: ---> percentile 99.50 = 34.406
sockperf: ---> percentile 99.00 = 33.337
sockperf: ---> percentile 95.00 = 27.497
sockperf: ---> percentile 90.00 = 26.072
sockperf: ---> percentile 75.00 = 24.618
sockperf: ---> percentile 50.00 = 24.443
sockperf: ---> percentile 25.00 = 24.361
sockperf: ---> <MIN> observation = 16.746
[root@c01 sbin]# sockperf ping-pong -i 192.168.0.12 -p 5001 -t
10
sockperf: == version #2.6 ==
sockperf[CLIENT] send on:sockperf: using recvfrom() to block
on
socket(s)
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
I find the ping command with flood option handy to measure latency,
gives stats min/max/average/std deviation
example:
ping -c 100000 -f 10.0.1.12
Maged
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
