ping -c 100000 -f ping -M do -s 8972 10Gb ConnectX-3 Pro, DAC + Vlan rtt min/avg/max/mdev = 0.010/0.013/0.200/0.003 ms, ipg/ewma 0.025/0.014 ms 8980 bytes from 10.0.0.11: icmp_seq=3 ttl=64 time=0.144 ms 8980 bytes from 10.0.0.11: icmp_seq=4 ttl=64 time=0.205 ms 8980 bytes from 10.0.0.11: icmp_seq=5 ttl=64 time=0.248 ms 8980 bytes from 10.0.0.11: icmp_seq=6 ttl=64 time=0.281 ms 8980 bytes from 10.0.0.11: icmp_seq=7 ttl=64 time=0.187 ms 8980 bytes from 10.0.0.11: icmp_seq=8 ttl=64 time=0.121 ms I350 Gigabit + bond rtt min/avg/max/mdev = 0.027/0.038/0.211/0.006 ms, ipg/ewma 0.050/0.041 ms 8980 bytes from 192.168.0.11: icmp_seq=1 ttl=64 time=0.555 ms 8980 bytes from 192.168.0.11: icmp_seq=2 ttl=64 time=0.508 ms 8980 bytes from 192.168.0.11: icmp_seq=3 ttl=64 time=0.514 ms 8980 bytes from 192.168.0.11: icmp_seq=4 ttl=64 time=0.555 ms -----Original Message----- From: Nick Fisk [mailto:nick@xxxxxxxxxx] Sent: maandag 22 januari 2018 12:38 To: ceph-users@xxxxxxxxxxxxxx Subject: Re: What is the should be the expected latency of 10Gbit network connections Anyone with 25G ethernet willing to do the test? Would love to see what the latency figures are for that. From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Maged Mokhtar Sent: 22 January 2018 11:28 To: ceph-users@xxxxxxxxxxxxxx Subject: Re: What is the should be the expected latency of 10Gbit network connections On 2018-01-22 08:39, Wido den Hollander wrote: On 01/20/2018 02:02 PM, Marc Roos wrote: If I test my connections with sockperf via a 1Gbit switch I get around 25usec, when I test the 10Gbit connection via the switch I have around 12usec is that normal? Or should there be a differnce of 10x. No, that's normal. Tests with 8k ping packets over different links I did: 1GbE: 0.800ms 10GbE: 0.200ms 40GbE: 0.150ms Wido sockperf ping-pong sockperf: Warmup stage (sending a few dummy messages)... sockperf: Starting test... sockperf: Test end (interrupted by timer) sockperf: Test ended sockperf: [Total Run] RunTime=10.100 sec; SentMessages=432875; ReceivedMessages=432874 sockperf: ========= Printing statistics for Server No: 0 sockperf: [Valid Duration] RunTime=10.000 sec; SentMessages=428640; ReceivedMessages=428640 sockperf: ====> avg-lat= 11.609 (std-dev=1.684) sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0 sockperf: Summary: Latency is 11.609 usec sockperf: Total 428640 observations; each percentile contains 4286.40 observations sockperf: ---> <MAX> observation = 856.944 sockperf: ---> percentile 99.99 = 39.789 sockperf: ---> percentile 99.90 = 20.550 sockperf: ---> percentile 99.50 = 17.094 sockperf: ---> percentile 99.00 = 15.578 sockperf: ---> percentile 95.00 = 12.838 sockperf: ---> percentile 90.00 = 12.299 sockperf: ---> percentile 75.00 = 11.844 sockperf: ---> percentile 50.00 = 11.409 sockperf: ---> percentile 25.00 = 11.124 sockperf: ---> <MIN> observation = 8.888 sockperf: Warmup stage (sending a few dummy messages)... sockperf: Starting test... sockperf: Test end (interrupted by timer) sockperf: Test ended sockperf: [Total Run] RunTime=1.100 sec; SentMessages=22065; ReceivedMessages=22064 sockperf: ========= Printing statistics for Server No: 0 sockperf: [Valid Duration] RunTime=1.000 sec; SentMessages=20056; ReceivedMessages=20056 sockperf: ====> avg-lat= 24.861 (std-dev=1.774) sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0 sockperf: Summary: Latency is 24.861 usec sockperf: Total 20056 observations; each percentile contains 200.56 observations sockperf: ---> <MAX> observation = 77.158 sockperf: ---> percentile 99.99 = 54.285 sockperf: ---> percentile 99.90 = 37.864 sockperf: ---> percentile 99.50 = 34.406 sockperf: ---> percentile 99.00 = 33.337 sockperf: ---> percentile 95.00 = 27.497 sockperf: ---> percentile 90.00 = 26.072 sockperf: ---> percentile 75.00 = 24.618 sockperf: ---> percentile 50.00 = 24.443 sockperf: ---> percentile 25.00 = 24.361 sockperf: ---> <MIN> observation = 16.746 [root@c01 sbin]# sockperf ping-pong -i 192.168.0.12 -p 5001 -t 10 sockperf: == version #2.6 == sockperf[CLIENT] send on:sockperf: using recvfrom() to block on socket(s) _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com I find the ping command with flood option handy to measure latency, gives stats min/max/average/std deviation example: ping -c 100000 -f 10.0.1.12 Maged _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |