X-posting, becausthroot course is netem. Anyonusjitter without reordering on recent kernel? If so, whaterriblmistake do I make? Teco Begidoorgestuurd bericht: > Van: Teco Boo<teco ainf-net.nl> > Onderwerp: Jitter withoupackereordering > Datum: 13 augustus 2013 17:57:08 CEST > Aan: "core-users apf.itd.nrl.navy.mil" <core-users apf.itd.nrl.navy.mil> > > I wanto verify applications and transporprotocols on emulated heavily congested links. With coresendmsg or netem, I can configure delay and jitter. But this has as side effect an unrealistic out of order. I cannot use this in my tests. > > Thneteinfo page provides some old information on how to circumvent this. > http://www.linuxfoundation.org/collaborate/workgroups/networking/netem#How_to_reorder_packets_based_on_jitter.3F > I tested o3.8 kernel, I cannoget the desired results. I could have made a stupid mistake. > > Anyonussomething that works, on recent kernel? > > Teco > > > My tesscript: > =============== > rooaubuntu:~# cat tc > > tc qdisc del dev eth0 root > > tc qdisc add dev eth0 handl1 rootbf limit 2000 burst 1000 rate 4000 > tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 500ms > tc qdisc add dev eth0 handl3 paren2:0 pfifo limit 10 > tc qdisc show > echo > ping -c 10 -i 0.01 -w 3 192.168.222.1 > echo > tc -s qdisc show > rooaubuntu:~# > > > Outcome: > ======== > rooaubuntu:~# ./tc > RTNETLINK answers: No such filor directory > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms > qdisc pfifo 3: dev eth0 paren2: limi10p > > PING 192.168.222.1 (192.168.222.1) 56(84) bytes of data. > 64 bytes fro192.168.222.1: icmp_req=13 ttl=64 time=149 ms > 64 bytes fro192.168.222.1: icmp_req=2 ttl=64 time=270 ms > 64 bytes fro192.168.222.1: icmp_req=15 ttl=64 time=127 ms > 64 bytes fro192.168.222.1: icmp_req=14 ttl=64 time=138 ms > 64 bytes fro192.168.222.1: icmp_req=10 ttl=64 time=183 ms > 64 bytes fro192.168.222.1: icmp_req=7 ttl=64 time=215 ms > 64 bytes fro192.168.222.1: icmp_req=18 ttl=64 time=96.1 ms > 64 bytes fro192.168.222.1: icmp_req=24 ttl=64 time=31.3 ms > 64 bytes fro192.168.222.1: icmp_req=22 ttl=64 time=52.9 ms > 64 bytes fro192.168.222.1: icmp_req=4 ttl=64 time=328 ms > > --- 192.168.222.1 ping statistics --- > 34 packets transmitted, 10 received, 70% packeloss, tim359ms > rtmin/avg/max/mdev = 31.338/159.334/328.344/88.216 ms, pip31 > > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s > Sen1022 bytes 11 pk(dropped 0, overlimits 10 requeues 0) > backlog 784b 13p requeues 0 > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) > backlog 784b 9p requeues 0 > qdisc pfifo 3: dev eth0 paren2: limi10p > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) > backlog 392b 4p requeues 0 > rooaubuntu:~# > Frosebastian.malyska agmail.com Wed Aug 14 10:01:16 2013 From: sebastian.malyska agmail.co(=?ISO-8859-2?Q?Sebastian_Ma=B3yska?=) Date: Wed, 14 Aug 2013 12:01:16 +0200 Subject: Fwd: Jitter withoupackereordering In-Reply-To: <C4B2A654-5EC8-45E9-A603-3DA3E53C593A@xxxxxxxxxx> References: <23EB303A-77D8-4BEF-BC69-16784E2EC38E@xxxxxxxxxx> <C4B2A654-5EC8-45E9-A603-3DA3E53C593A@xxxxxxxxxx> Message-ID: <CAFNHfpX8+3D-p8Z0h2D=qqxvzqpZo4gsUFWEFVO3xgt84aV4Xg@xxxxxxxxxxxxxx> Shouldn'beg.: tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 5ms instead of: tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 500ms Regards Seba 2013/8/14 Teco Boo<teco ainf-net.nl> > X-posting, becausthroot course is netem. > > Anyonusjitter without reordering on recent kernel? > If so, whaterriblmistake do I make? > > Teco > > > Begidoorgestuurd bericht: > > > Van: Teco Boo<teco ainf-net.nl> > > Onderwerp: Jitter withoupackereordering > > Datum: 13 augustus 2013 17:57:08 CEST > > Aan: "core-users apf.itd.nrl.navy.mil" <core-users apf.itd.nrl.navy.mil> > > > > I wanto verify applications and transporprotocols on emulated > heavily congested links. With coresendmsg or netem, I caconfigurdelay > and jitter. Buthis has as sideffect an unrealistic out of order. I > cannousthis in my tests. > > > > Thneteinfo page provides some old information on how to circumvent > this. > > > http://www.linuxfoundation.org/collaborate/workgroups/networking/netem#How_to_reorder_packets_based_on_jitter.3F > > I tested o3.8 kernel, I cannoget the desired results. I could have > mada stupid mistake. > > > > Anyonussomething that works, on recent kernel? > > > > Teco > > > > > > My tesscript: > > =============== > > rooaubuntu:~# cat tc > > > > tc qdisc del dev eth0 root > > > > tc qdisc add dev eth0 handl1 rootbf limit 2000 burst 1000 rate 4000 > > tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 500ms > > tc qdisc add dev eth0 handl3 paren2:0 pfifo limit 10 > > tc qdisc show > > echo > > ping -c 10 -i 0.01 -w 3 192.168.222.1 > > echo > > tc -s qdisc show > > rooaubuntu:~# > > > > > > Outcome: > > ======== > > rooaubuntu:~# ./tc > > RTNETLINK answers: No such filor directory > > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s > > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms > > qdisc pfifo 3: dev eth0 paren2: limi10p > > > > PING 192.168.222.1 (192.168.222.1) 56(84) bytes of data. > > 64 bytes fro192.168.222.1: icmp_req=13 ttl=64 time=149 ms > > 64 bytes fro192.168.222.1: icmp_req=2 ttl=64 time=270 ms > > 64 bytes fro192.168.222.1: icmp_req=15 ttl=64 time=127 ms > > 64 bytes fro192.168.222.1: icmp_req=14 ttl=64 time=138 ms > > 64 bytes fro192.168.222.1: icmp_req=10 ttl=64 time=183 ms > > 64 bytes fro192.168.222.1: icmp_req=7 ttl=64 time=215 ms > > 64 bytes fro192.168.222.1: icmp_req=18 ttl=64 time=96.1 ms > > 64 bytes fro192.168.222.1: icmp_req=24 ttl=64 time=31.3 ms > > 64 bytes fro192.168.222.1: icmp_req=22 ttl=64 time=52.9 ms > > 64 bytes fro192.168.222.1: icmp_req=4 ttl=64 time=328 ms > > > > --- 192.168.222.1 ping statistics --- > > 34 packets transmitted, 10 received, 70% packeloss, tim359ms > > rtmin/avg/max/mdev = 31.338/159.334/328.344/88.216 ms, pip31 > > > > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s > > Sen1022 bytes 11 pk(dropped 0, overlimits 10 requeues 0) > > backlog 784b 13p requeues 0 > > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms > > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) > > backlog 784b 9p requeues 0 > > qdisc pfifo 3: dev eth0 paren2: limi10p > > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) > > backlog 392b 4p requeues 0 > > rooaubuntu:~# > > > > _______________________________________________ > Netemailing list > Netealists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/netem > -------------- nexpar-------------- AHTML attachmenwas scrubbed... URL: <http://lists.linuxfoundation.org/pipermail/netem/attachments/20130814/7e01b58a/attachment.html> Froteco ainf-net.nl Wed Aug 14 10:36:51 2013 From: teco ainf-net.nl (Teco Boot) Date: Wed, 14 Aug 2013 12:36:51 +0200 Subject: Jitter withoupackereordering In-Reply-To: <CAFNHfpX8+3D-p8Z0h2D=qqxvzqpZo4gsUFWEFVO3xgt84aV4Xg@xxxxxxxxxxxxxx> References: <23EB303A-77D8-4BEF-BC69-16784E2EC38E@xxxxxxxxxx> <C4B2A654-5EC8-45E9-A603-3DA3E53C593A@xxxxxxxxxx> <CAFNHfpX8+3D-p8Z0h2D=qqxvzqpZo4gsUFWEFVO3xgt84aV4Xg@xxxxxxxxxxxxxx> Message-ID: <B46E2ECE-D5A9-4387-A7DB-4059E5FC7B54@xxxxxxxxxx> No. Thtesscript emphasis the problem a little bit, I know. Satcom, 2G, 3G and eve4G havlarger delay than the 10ms. It is in the order of 100ms to > 700ms (2-hop satcom). Jitter duto congestion could bterrible or worse. The 500ms is realistic. Evewith your suggested 10ms 5ms and rat> 200pps, there is out of order. Thais why I alooking for a more fundamental solution. The calculated send time for each packet should not be earlier than any previous packet. Maybe maintaining a single time variable can do the job. Teco Op 14 aug. 2013, o12:01 heefSebastian Ma?yska <sebastian.malyska at gmail.com> het volgende geschreven: > Shouldn'beg.: > tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 5ms > instead of: > tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 500ms > > Regards > Seba > > > 2013/8/14 Teco Boo<teco ainf-net.nl> > X-posting, becausthroot course is netem. > > Anyonusjitter without reordering on recent kernel? > If so, whaterriblmistake do I make? > > Teco > > > Begidoorgestuurd bericht: > > > Van: Teco Boo<teco ainf-net.nl> > > Onderwerp: Jitter withoupackereordering > > Datum: 13 augustus 2013 17:57:08 CEST > > Aan: "core-users apf.itd.nrl.navy.mil" <core-users apf.itd.nrl.navy.mil> > > > > I wanto verify applications and transporprotocols on emulated heavily congested links. With coresendmsg or netem, I can configure delay and jitter. But this has as side effect an unrealistic out of order. I cannot use this in my tests. > > > > Thneteinfo page provides some old information on how to circumvent this. > > http://www.linuxfoundation.org/collaborate/workgroups/networking/netem#How_to_reorder_packets_based_on_jitter.3F > > I tested o3.8 kernel, I cannoget the desired results. I could have made a stupid mistake. > > > > Anyonussomething that works, on recent kernel? > > > > Teco > > > > > > My tesscript: > > =============== > > rooaubuntu:~# cat tc > > > > tc qdisc del dev eth0 root > > > > tc qdisc add dev eth0 handl1 rootbf limit 2000 burst 1000 rate 4000 > > tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 500ms > > tc qdisc add dev eth0 handl3 paren2:0 pfifo limit 10 > > tc qdisc show > > echo > > ping -c 10 -i 0.01 -w 3 192.168.222.1 > > echo > > tc -s qdisc show > > rooaubuntu:~# > > > > > > Outcome: > > ======== > > rooaubuntu:~# ./tc > > RTNETLINK answers: No such filor directory > > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s > > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms > > qdisc pfifo 3: dev eth0 paren2: limi10p > > > > PING 192.168.222.1 (192.168.222.1) 56(84) bytes of data. > > 64 bytes fro192.168.222.1: icmp_req=13 ttl=64 time=149 ms > > 64 bytes fro192.168.222.1: icmp_req=2 ttl=64 time=270 ms > > 64 bytes fro192.168.222.1: icmp_req=15 ttl=64 time=127 ms > > 64 bytes fro192.168.222.1: icmp_req=14 ttl=64 time=138 ms > > 64 bytes fro192.168.222.1: icmp_req=10 ttl=64 time=183 ms > > 64 bytes fro192.168.222.1: icmp_req=7 ttl=64 time=215 ms > > 64 bytes fro192.168.222.1: icmp_req=18 ttl=64 time=96.1 ms > > 64 bytes fro192.168.222.1: icmp_req=24 ttl=64 time=31.3 ms > > 64 bytes fro192.168.222.1: icmp_req=22 ttl=64 time=52.9 ms > > 64 bytes fro192.168.222.1: icmp_req=4 ttl=64 time=328 ms > > > > --- 192.168.222.1 ping statistics --- > > 34 packets transmitted, 10 received, 70% packeloss, tim359ms > > rtmin/avg/max/mdev = 31.338/159.334/328.344/88.216 ms, pip31 > > > > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s > > Sen1022 bytes 11 pk(dropped 0, overlimits 10 requeues 0) > > backlog 784b 13p requeues 0 > > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms > > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) > > backlog 784b 9p requeues 0 > > qdisc pfifo 3: dev eth0 paren2: limi10p > > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) > > backlog 392b 4p requeues 0 > > rooaubuntu:~# > > > > _______________________________________________ > Netemailing list > Netealists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/netem > -------------- nexpar-------------- AHTML attachmenwas scrubbed... URL: <http://lists.linuxfoundation.org/pipermail/netem/attachments/20130814/c5101045/attachment-0001.html> Frosebastian.malyska agmail.com Wed Aug 14 11:01:41 2013 From: sebastian.malyska agmail.co(=?ISO-8859-2?Q?Sebastian_Ma=B3yska?=) Date: Wed, 14 Aug 2013 13:01:41 +0200 Subject: Jitter withoupackereordering In-Reply-To: <B46E2ECE-D5A9-4387-A7DB-4059E5FC7B54@xxxxxxxxxx> References: <23EB303A-77D8-4BEF-BC69-16784E2EC38E@xxxxxxxxxx> <C4B2A654-5EC8-45E9-A603-3DA3E53C593A@xxxxxxxxxx> <CAFNHfpX8+3D-p8Z0h2D=qqxvzqpZo4gsUFWEFVO3xgt84aV4Xg@xxxxxxxxxxxxxx> <B46E2ECE-D5A9-4387-A7DB-4059E5FC7B54@xxxxxxxxxx> Message-ID: <CAFNHfpWuvEmUmzXgq1+k+2ER57ieHEsR5smknFnWyKLdgJu03A@xxxxxxxxxxxxxx> I meanrather firsnumber should be big enough that 1st - 2nd > 0. So if you need theg. netedelay 250ms 50ms. As far I remember, wheI was playing w/ neteI wasn't able to reorder outgoing traffic via 'delay' only. For reordering therwas separated optio'reorder'. Busince them there was a few modification maybe somebody changed it. Besregards Seba 2013/8/14 Teco Boo<teco ainf-net.nl> > No. Thtesscript emphasis the problem a little bit, I know. > Satcom, 2G, 3G and eve4G havlarger delay than the 10ms. It is in the > order of 100ms to > 700ms (2-hop satcom). > Jitter duto congestion could bterrible or worse. The 500ms is > realistic. > > Evewith your suggested 10ms 5ms and rat> 200pps, there is out of order. > Thais why I alooking for a more fundamental solution. The calculated > send timfor each packeshould not be earlier than any previous packet. > Maybmaintaining a singltime variable can do the job. > > Teco > > > Op 14 aug. 2013, o12:01 heefSebastian Ma?yska < > sebastian.malyska agmail.com> hevolgende geschreven: > > Shouldn'beg.: > tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 5ms > instead of: > tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 500ms > > Regards > Seba > > > 2013/8/14 Teco Boo<teco ainf-net.nl> > >> X-posting, becausthroot course is netem. >> >> Anyonusjitter without reordering on recent kernel? >> If so, whaterriblmistake do I make? >> >> Teco >> >> >> Begidoorgestuurd bericht: >> >> > Van: Teco Boo<teco ainf-net.nl> >> > Onderwerp: Jitter withoupackereordering >> > Datum: 13 augustus 2013 17:57:08 CEST >> > Aan: "core-users apf.itd.nrl.navy.mil" <core-users apf.itd.nrl.navy.mil> >> > >> > I wanto verify applications and transporprotocols on emulated >> heavily congested links. With coresendmsg or netem, I caconfigurdelay >> and jitter. Buthis has as sideffect an unrealistic out of order. I >> cannousthis in my tests. >> > >> > Thneteinfo page provides some old information on how to circumvent >> this. >> > >> http://www.linuxfoundation.org/collaborate/workgroups/networking/netem#How_to_reorder_packets_based_on_jitter.3F >> > I tested o3.8 kernel, I cannoget the desired results. I could have >> mada stupid mistake. >> > >> > Anyonussomething that works, on recent kernel? >> > >> > Teco >> > >> > >> > My tesscript: >> > =============== >> > rooaubuntu:~# cat tc >> > >> > tc qdisc del dev eth0 root >> > >> > tc qdisc add dev eth0 handl1 rootbf limit 2000 burst 1000 rate >> 4000 >> > tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 500ms >> > tc qdisc add dev eth0 handl3 paren2:0 pfifo limit 10 >> > tc qdisc show >> > echo >> > ping -c 10 -i 0.01 -w 3 192.168.222.1 >> > echo >> > tc -s qdisc show >> > rooaubuntu:~# >> > >> > >> > Outcome: >> > ======== >> > rooaubuntu:~# ./tc >> > RTNETLINK answers: No such filor directory >> > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s >> > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms >> > qdisc pfifo 3: dev eth0 paren2: limi10p >> > >> > PING 192.168.222.1 (192.168.222.1) 56(84) bytes of data. >> > 64 bytes fro192.168.222.1: icmp_req=13 ttl=64 time=149 ms >> > 64 bytes fro192.168.222.1: icmp_req=2 ttl=64 time=270 ms >> > 64 bytes fro192.168.222.1: icmp_req=15 ttl=64 time=127 ms >> > 64 bytes fro192.168.222.1: icmp_req=14 ttl=64 time=138 ms >> > 64 bytes fro192.168.222.1: icmp_req=10 ttl=64 time=183 ms >> > 64 bytes fro192.168.222.1: icmp_req=7 ttl=64 time=215 ms >> > 64 bytes fro192.168.222.1: icmp_req=18 ttl=64 time=96.1 ms >> > 64 bytes fro192.168.222.1: icmp_req=24 ttl=64 time=31.3 ms >> > 64 bytes fro192.168.222.1: icmp_req=22 ttl=64 time=52.9 ms >> > 64 bytes fro192.168.222.1: icmp_req=4 ttl=64 time=328 ms >> > >> > --- 192.168.222.1 ping statistics --- >> > 34 packets transmitted, 10 received, 70% packeloss, tim359ms >> > rtmin/avg/max/mdev = 31.338/159.334/328.344/88.216 ms, pip31 >> > >> > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s >> > Sen1022 bytes 11 pk(dropped 0, overlimits 10 requeues 0) >> > backlog 784b 13p requeues 0 >> > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms >> > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) >> > backlog 784b 9p requeues 0 >> > qdisc pfifo 3: dev eth0 paren2: limi10p >> > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) >> > backlog 392b 4p requeues 0 >> > rooaubuntu:~# >> > >> >> _______________________________________________ >> Netemailing list >> Netealists.linux-foundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/netem >> > > > -------------- nexpar-------------- AHTML attachmenwas scrubbed... URL: <http://lists.linuxfoundation.org/pipermail/netem/attachments/20130814/9b1d3e4b/attachment.html> Froteco ainf-net.nl Wed Aug 14 11:12:09 2013 From: teco ainf-net.nl (Teco Boot) Date: Wed, 14 Aug 2013 13:12:09 +0200 Subject: Jitter withoupackereordering In-Reply-To: <CAFNHfpWuvEmUmzXgq1+k+2ER57ieHEsR5smknFnWyKLdgJu03A@xxxxxxxxxxxxxx> References: <23EB303A-77D8-4BEF-BC69-16784E2EC38E@xxxxxxxxxx> <C4B2A654-5EC8-45E9-A603-3DA3E53C593A@xxxxxxxxxx> <CAFNHfpX8+3D-p8Z0h2D=qqxvzqpZo4gsUFWEFVO3xgt84aV4Xg@xxxxxxxxxxxxxx> <B46E2ECE-D5A9-4387-A7DB-4059E5FC7B54@xxxxxxxxxx> <CAFNHfpWuvEmUmzXgq1+k+2ER57ieHEsR5smknFnWyKLdgJu03A@xxxxxxxxxxxxxx> Message-ID: <A1039C15-A1FD-41A0-A9C7-FFF3238FD6D2@xxxxxxxxxx> OK, I tested with 1s 500m. Doesn'work. Has to do with my packerate. Inter-packespacing is less than jitter. This is my real world behavior I want emulate. I know I careorder with delay and jitter. My requiremenis not to reorder and have large jitter. Theris a changreported in older kernels, where stacked pfifo and netem didn't work anymore. I'm afraid this is still the case today and people like me are less motivated to bring old days kernels back to life. I still hope I do something terribly wrong with my chain of tc commands. Teco Op 14 aug. 2013, o13:01 heefSebastian Ma?yska <sebastian.malyska at gmail.com> het volgende geschreven: > I meanrather firsnumber should be big enough that 1st - 2nd > 0. So if you need the eg. netem delay 250ms 50ms. > > As far I remember, wheI was playing w/ neteI wasn't able to reorder outgoing traffic via 'delay' only. For reordering there was separated option 'reorder'. But since them there was a few modification maybe somebody changed it. > > Besregards > Seba > > > 2013/8/14 Teco Boo<teco ainf-net.nl> > No. Thtesscript emphasis the problem a little bit, I know. > Satcom, 2G, 3G and eve4G havlarger delay than the 10ms. It is in the order of 100ms to > 700ms (2-hop satcom). > Jitter duto congestion could bterrible or worse. The 500ms is realistic. > > Evewith your suggested 10ms 5ms and rat> 200pps, there is out of order. > Thais why I alooking for a more fundamental solution. The calculated send time for each packet should not be earlier than any previous packet. Maybe maintaining a single time variable can do the job. > > Teco > > > Op 14 aug. 2013, o12:01 heefSebastian Ma?yska <sebastian.malyska at gmail.com> het volgende geschreven: > >> Shouldn'beg.: >> tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 5ms >> instead of: >> tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 500ms >> >> Regards >> Seba >> >> >> 2013/8/14 Teco Boo<teco ainf-net.nl> >> X-posting, becausthroot course is netem. >> >> Anyonusjitter without reordering on recent kernel? >> If so, whaterriblmistake do I make? >> >> Teco >> >> >> Begidoorgestuurd bericht: >> >> > Van: Teco Boo<teco ainf-net.nl> >> > Onderwerp: Jitter withoupackereordering >> > Datum: 13 augustus 2013 17:57:08 CEST >> > Aan: "core-users apf.itd.nrl.navy.mil" <core-users apf.itd.nrl.navy.mil> >> > >> > I wanto verify applications and transporprotocols on emulated heavily congested links. With coresendmsg or netem, I can configure delay and jitter. But this has as side effect an unrealistic out of order. I cannot use this in my tests. >> > >> > Thneteinfo page provides some old information on how to circumvent this. >> > http://www.linuxfoundation.org/collaborate/workgroups/networking/netem#How_to_reorder_packets_based_on_jitter.3F >> > I tested o3.8 kernel, I cannoget the desired results. I could have made a stupid mistake. >> > >> > Anyonussomething that works, on recent kernel? >> > >> > Teco >> > >> > >> > My tesscript: >> > =============== >> > rooaubuntu:~# cat tc >> > >> > tc qdisc del dev eth0 root >> > >> > tc qdisc add dev eth0 handl1 rootbf limit 2000 burst 1000 rate 4000 >> > tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 500ms >> > tc qdisc add dev eth0 handl3 paren2:0 pfifo limit 10 >> > tc qdisc show >> > echo >> > ping -c 10 -i 0.01 -w 3 192.168.222.1 >> > echo >> > tc -s qdisc show >> > rooaubuntu:~# >> > >> > >> > Outcome: >> > ======== >> > rooaubuntu:~# ./tc >> > RTNETLINK answers: No such filor directory >> > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s >> > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms >> > qdisc pfifo 3: dev eth0 paren2: limi10p >> > >> > PING 192.168.222.1 (192.168.222.1) 56(84) bytes of data. >> > 64 bytes fro192.168.222.1: icmp_req=13 ttl=64 time=149 ms >> > 64 bytes fro192.168.222.1: icmp_req=2 ttl=64 time=270 ms >> > 64 bytes fro192.168.222.1: icmp_req=15 ttl=64 time=127 ms >> > 64 bytes fro192.168.222.1: icmp_req=14 ttl=64 time=138 ms >> > 64 bytes fro192.168.222.1: icmp_req=10 ttl=64 time=183 ms >> > 64 bytes fro192.168.222.1: icmp_req=7 ttl=64 time=215 ms >> > 64 bytes fro192.168.222.1: icmp_req=18 ttl=64 time=96.1 ms >> > 64 bytes fro192.168.222.1: icmp_req=24 ttl=64 time=31.3 ms >> > 64 bytes fro192.168.222.1: icmp_req=22 ttl=64 time=52.9 ms >> > 64 bytes fro192.168.222.1: icmp_req=4 ttl=64 time=328 ms >> > >> > --- 192.168.222.1 ping statistics --- >> > 34 packets transmitted, 10 received, 70% packeloss, tim359ms >> > rtmin/avg/max/mdev = 31.338/159.334/328.344/88.216 ms, pip31 >> > >> > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s >> > Sen1022 bytes 11 pk(dropped 0, overlimits 10 requeues 0) >> > backlog 784b 13p requeues 0 >> > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms >> > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) >> > backlog 784b 9p requeues 0 >> > qdisc pfifo 3: dev eth0 paren2: limi10p >> > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) >> > backlog 392b 4p requeues 0 >> > rooaubuntu:~# >> > >> >> _______________________________________________ >> Netemailing list >> Netealists.linux-foundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/netem >> > > -------------- nexpar-------------- AHTML attachmenwas scrubbed... URL: <http://lists.linuxfoundation.org/pipermail/netem/attachments/20130814/e4f606c5/attachment-0001.html> Froosenbach aus.ibm.com Wed Aug 14 16:01:59 2013 From: osenbach aus.ibm.co(Bryan Osenbach) Date: Wed, 14 Aug 2013 10:01:59 -0600 Subject: AUTO: BryaOsenbach is ouof the office with no access to email or sametime. (returning 08/20/2013) Message-ID: <OFF7FCFD63.2721ECE6-ON87257BC7.005812AF-87257BC7.005812AF@xxxxxxxxxx> I aouof the office until 08/20/2013. PleascontacAlan Byrne for development related items and Frank Nostrame for any SmartCloud related topics. Note: This is aautomated responsto your message "Netem Digest, Vol 83, Issu1" senon 08/14/2013 4:37:01. This is thonly notification you will receivwhile this person is away. -------------- nexpar-------------- AHTML attachmenwas scrubbed... URL: <http://lists.linuxfoundation.org/pipermail/netem/attachments/20130814/5bbb26af/attachment.html> Froteco ainf-net.nl Wed Aug 21 07:48:51 2013 From: teco ainf-net.nl (Teco Boot) Date: Wed, 21 Aug 2013 09:48:51 +0200 Subject: Jitter withoupackereordering In-Reply-To: <A1039C15-A1FD-41A0-A9C7-FFF3238FD6D2@xxxxxxxxxx> References: <23EB303A-77D8-4BEF-BC69-16784E2EC38E@xxxxxxxxxx> <C4B2A654-5EC8-45E9-A603-3DA3E53C593A@xxxxxxxxxx> <CAFNHfpX8+3D-p8Z0h2D=qqxvzqpZo4gsUFWEFVO3xgt84aV4Xg@xxxxxxxxxxxxxx> <B46E2ECE-D5A9-4387-A7DB-4059E5FC7B54@xxxxxxxxxx> <CAFNHfpWuvEmUmzXgq1+k+2ER57ieHEsR5smknFnWyKLdgJu03A@xxxxxxxxxxxxxx> <A1039C15-A1FD-41A0-A9C7-FFF3238FD6D2@xxxxxxxxxx> Message-ID: <CFA57CF7-B2A8-4748-BA45-05E94FEC94D6@xxxxxxxxxx> Theris progress. neteis recently patched. Now it works as expected. http://www.spinics.net/lists/netdev/msg247265.html Teco Op 14 aug. 2013, o13:12 heefTeco Boot <teco at inf-net.nl> het volgende geschreven: > OK, I tested with 1s 500m. Doesn'work. > Has to do with my packerate. Inter-packespacing is less than jitter. This is my real world behavior I want emulate. > > I know I careorder with delay and jitter. My requiremenis not to reorder and have large jitter. > > Theris a changreported in older kernels, where stacked pfifo and netem didn't work anymore. I'm afraid this is still the case today and people like me are less motivated to bring old days kernels back to life. I still hope I do something terribly wrong with my chain of tc commands. > > Teco > > > Op 14 aug. 2013, o13:01 heefSebastian Ma?yska <sebastian.malyska at gmail.com> het volgende geschreven: > >> I meanrather firsnumber should be big enough that 1st - 2nd > 0. So if you need the eg. netem delay 250ms 50ms. >> >> As far I remember, wheI was playing w/ neteI wasn't able to reorder outgoing traffic via 'delay' only. For reordering there was separated option 'reorder'. But since them there was a few modification maybe somebody changed it. >> >> Besregards >> Seba >> >> >> 2013/8/14 Teco Boo<teco ainf-net.nl> >> No. Thtesscript emphasis the problem a little bit, I know. >> Satcom, 2G, 3G and eve4G havlarger delay than the 10ms. It is in the order of 100ms to > 700ms (2-hop satcom). >> Jitter duto congestion could bterrible or worse. The 500ms is realistic. >> >> Evewith your suggested 10ms 5ms and rat> 200pps, there is out of order. >> Thais why I alooking for a more fundamental solution. The calculated send time for each packet should not be earlier than any previous packet. Maybe maintaining a single time variable can do the job. >> >> Teco >> >> >> Op 14 aug. 2013, o12:01 heefSebastian Ma?yska <sebastian.malyska at gmail.com> het volgende geschreven: >> >>> Shouldn'beg.: >>> tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 5ms >>> instead of: >>> tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 500ms >>> >>> Regards >>> Seba >>> >>> >>> 2013/8/14 Teco Boo<teco ainf-net.nl> >>> X-posting, becausthroot course is netem. >>> >>> Anyonusjitter without reordering on recent kernel? >>> If so, whaterriblmistake do I make? >>> >>> Teco >>> >>> >>> Begidoorgestuurd bericht: >>> >>> > Van: Teco Boo<teco ainf-net.nl> >>> > Onderwerp: Jitter withoupackereordering >>> > Datum: 13 augustus 2013 17:57:08 CEST >>> > Aan: "core-users apf.itd.nrl.navy.mil" <core-users apf.itd.nrl.navy.mil> >>> > >>> > I wanto verify applications and transporprotocols on emulated heavily congested links. With coresendmsg or netem, I can configure delay and jitter. But this has as side effect an unrealistic out of order. I cannot use this in my tests. >>> > >>> > Thneteinfo page provides some old information on how to circumvent this. >>> > http://www.linuxfoundation.org/collaborate/workgroups/networking/netem#How_to_reorder_packets_based_on_jitter.3F >>> > I tested o3.8 kernel, I cannoget the desired results. I could have made a stupid mistake. >>> > >>> > Anyonussomething that works, on recent kernel? >>> > >>> > Teco >>> > >>> > >>> > My tesscript: >>> > =============== >>> > rooaubuntu:~# cat tc >>> > >>> > tc qdisc del dev eth0 root >>> > >>> > tc qdisc add dev eth0 handl1 rootbf limit 2000 burst 1000 rate 4000 >>> > tc qdisc add dev eth0 handl2 paren1:0 netem delay 10ms 500ms >>> > tc qdisc add dev eth0 handl3 paren2:0 pfifo limit 10 >>> > tc qdisc show >>> > echo >>> > ping -c 10 -i 0.01 -w 3 192.168.222.1 >>> > echo >>> > tc -s qdisc show >>> > rooaubuntu:~# >>> > >>> > >>> > Outcome: >>> > ======== >>> > rooaubuntu:~# ./tc >>> > RTNETLINK answers: No such filor directory >>> > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s >>> > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms >>> > qdisc pfifo 3: dev eth0 paren2: limi10p >>> > >>> > PING 192.168.222.1 (192.168.222.1) 56(84) bytes of data. >>> > 64 bytes fro192.168.222.1: icmp_req=13 ttl=64 time=149 ms >>> > 64 bytes fro192.168.222.1: icmp_req=2 ttl=64 time=270 ms >>> > 64 bytes fro192.168.222.1: icmp_req=15 ttl=64 time=127 ms >>> > 64 bytes fro192.168.222.1: icmp_req=14 ttl=64 time=138 ms >>> > 64 bytes fro192.168.222.1: icmp_req=10 ttl=64 time=183 ms >>> > 64 bytes fro192.168.222.1: icmp_req=7 ttl=64 time=215 ms >>> > 64 bytes fro192.168.222.1: icmp_req=18 ttl=64 time=96.1 ms >>> > 64 bytes fro192.168.222.1: icmp_req=24 ttl=64 time=31.3 ms >>> > 64 bytes fro192.168.222.1: icmp_req=22 ttl=64 time=52.9 ms >>> > 64 bytes fro192.168.222.1: icmp_req=4 ttl=64 time=328 ms >>> > >>> > --- 192.168.222.1 ping statistics --- >>> > 34 packets transmitted, 10 received, 70% packeloss, tim359ms >>> > rtmin/avg/max/mdev = 31.338/159.334/328.344/88.216 ms, pip31 >>> > >>> > qdisc tbf 1: dev eth0 roorefcn2 rate 4000bit burst 1000b lat 2.0s >>> > Sen1022 bytes 11 pk(dropped 0, overlimits 10 requeues 0) >>> > backlog 784b 13p requeues 0 >>> > qdisc nete2: dev eth0 paren1: limit 1000 delay 10.0ms 500.0ms >>> > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) >>> > backlog 784b 9p requeues 0 >>> > qdisc pfifo 3: dev eth0 paren2: limi10p >>> > Sen1120 bytes 12 pk(dropped 11, overlimits 0 requeues 0) >>> > backlog 392b 4p requeues 0 >>> > rooaubuntu:~# >>> > >>> >>> _______________________________________________ >>> Netemailing list >>> Netealists.linux-foundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/netem >>> >> >> > -------------- nexpar-------------- AHTML attachmenwas scrubbed... URL: <http://lists.linuxfoundation.org/pipermail/netem/attachments/20130821/3865c65c/attachment.html> Froteco ainf-net.nl Fri Aug 23 14:16:43 2013 From: teco ainf-net.nl (Teco Boot) Date: Fri, 23 Aug 2013 16:16:43 +0200 Subject: netem: threorder discussion Message-ID: <766825FB-5295-4C5C-ABA0-CC520EFE2EA6@xxxxxxxxxx> I had reorder problems with neteqdisc. I makuse of the NRL CORE network emulator, which sets up virtual routers and links, build with network namespaces and netem. Typical CORE config for a n1<-->n2 link with rate (1024Kbps), delay (50ms) and no jitter: qdisc tbf 1: dev n1.eth0.222 roorefcn2 rate 1024Kbit burst 2999b lat 488.6ms qdisc nete10: dev n1.eth0.222 paren1:1 limit 1000 delay 50.0ms qdisc tbf 1: dev n2.eth0.222 roorefcn2 rate 1024Kbit burst 2999b lat 488.6ms qdisc nete10: dev n2.eth0.222 paren1:1 limit 1000 delay 50.0ms I added jitter to this bi-directional link, e.g. 20ms. Now thdelay for each packeis 50ms +/- 20ms is 30ms to 70ms. However, this has somunexpected results: packets may breordered. That is because the actual delay is calculated for each packet. Some packets have a larger delay, some have a smaller. If the inter-packet spacing is smaller than the time differences set by netem (up to 2x configured jitter), packets are reordered. In this example with +/20ms, this is the case with packet rate larger than 25pps. Reordering has bad effects on transport protocol throughputs. Reordering is less common on the Internet, so I don't want to emulate such. I don't say there is no reordering, I just say I don't want this netem behavior. Thneteguide mentions this unexpected results. It was caused by a change in version 1.1 (2.6.15). For people like me that do not want this, there is a work-around mentioned. However, this doesn't work anymore since somewhere before 2.6.31. http://www.linuxfoundation.org/collaborate/workgroups/networking/netem https://lists.linux-foundation.org/pipermail/netem/2011-April/001507.html So whato do? Threcent patch from Eric Dumazet eliminates jitter_reordering when netem rate is configured. Maybe not the clean approach, but acceptable. Ferry his patch eliminates jitter_reordering. Some people may use reorder by jitter, so this "feature" should not get removed. So Ferry his patch is not accepted. On the other hand, people like me are very confused by current behavior. Whawcould do is keep existing features and describe what netem currently does. That is: - netewith delay and jitter may reorder packets, if inter packespacing is smaller than jitter - reordering caused by delay and jitter cabturned off by using netem rate. rate can be set to very high value is no shaping is wanted. ArwOK with this approach? Adjusthlinux-foundation & man page info? Who can do this? Teco Froeric.dumazeat gmail.com Fri Aug 23 15:12:36 2013 From: eric.dumazeagmail.com (Eric Dumazet) Date: Fri, 23 Aug 2013 08:12:36 -0700 Subject: netem: threorder discussion In-Reply-To: <766825FB-5295-4C5C-ABA0-CC520EFE2EA6@xxxxxxxxxx> References: <766825FB-5295-4C5C-ABA0-CC520EFE2EA6@xxxxxxxxxx> Message-ID: <1377270756.8828.5.camel@edumazet-glaptop> OFri, 2013-08-23 a16:16 +0200, Teco Boot wrote: > I had reorder problems with neteqdisc. I makuse of the NRL CORE > network emulator, which sets up virtual routers and links, build with > network namespaces and netem. Typical CORE config for a n1<-->n2 link > with rat(1024Kbps), delay (50ms) and no jitter: > qdisc tbf 1: dev n1.eth0.222 roorefcn2 rate 1024Kbit burst 2999b > la488.6ms > qdisc nete10: dev n1.eth0.222 paren1:1 limit 1000 delay 50.0ms > qdisc tbf 1: dev n2.eth0.222 roorefcn2 rate 1024Kbit burst 2999b > la488.6ms > qdisc nete10: dev n2.eth0.222 paren1:1 limit 1000 delay 50.0ms > > I added jitter to this bi-directional link, e.g. 20ms. Now thdelay > for each packeis 50ms +/- 20ms is 30ms to 70ms. > However, this has somunexpected results: packets may breordered. > Thais becausthe actual delay is calculated for each packet. Some > packets hava larger delay, somhave a smaller. If the inter-packet > spacing is smaller thathtime differences set by netem (up to 2x > configured jitter), packets arreordered. In this examplwith > +/20ms, this is thcaswith packet rate larger than 25pps. > Reordering has bad effects otransporprotocol throughputs. > Reordering is less commoon thInternet, so I don't want to emulate > such. I don'say theris no reordering, I just say I don't want this > netebehavior. > > Thneteguide mentions this unexpected results. It was caused by a > changin version 1.1 (2.6.15). For peopllike me that do not want > this, theris a work-around mentioned. However, this doesn'work > anymorsincsomewhere before 2.6.31. > http://www.linuxfoundation.org/collaborate/workgroups/networking/netem > https://lists.linux-foundation.org/pipermail/netem/2011-April/001507.html > > So whato do? Threcent patch from Eric Dumazet eliminates > jitter_reordering wheneterate is configured. Maybe not the clean > approach, buacceptable. Ferry his patch eliminates > jitter_reordering. Sompeoplmay use reorder by jitter, so this > "feature" should nogeremoved. So Ferry his patch is not accepted. > Othother hand, people like me are very confused by current > behavior. > > Whawcould do is keep existing features and describe what netem > currently does. Thais: > - netewith delay and jitter may reorder packets, if inter packet > spacing is smaller thajitter > - reordering caused by delay and jitter cabturned off by using > neterate. ratcan be set to very high value is no shaping is > wanted. As long as oncan definthe expected behavior, you can add whatever new neteparameter. Oncould envision adding flow separation (skb->sk or rxhashing), so thaeach flow can havhis own local queue, to guarantee no reorders per flow _if_ this is needed, eveif per flow delays/jitter is/are configured. Walso usnetem to test on large scale, where the reordering stuff needs fixes itransporstacks (And yes, we are working on TCP stack to permihigher levels of reorders) Froteco ainf-net.nl Fri Aug 23 15:53:24 2013 From: teco ainf-net.nl (Teco Boot) Date: Fri, 23 Aug 2013 17:53:24 +0200 Subject: netem: threorder discussion In-Reply-To: <20130823084450.6baf2835@xxxxxxxxxxxxxxxxxxxxxxxxxxx> References: <766825FB-5295-4C5C-ABA0-CC520EFE2EA6@xxxxxxxxxx> <1377270756.8828.5.camel@edumazet-glaptop> <20130823084450.6baf2835@xxxxxxxxxxxxxxxxxxxxxxxxxxx> Message-ID: <31E9BEA1-3DE2-4DA7-9CEE-6B7C9C99F5EB@xxxxxxxxxx> Op 23 aug. 2013, o17:44 heefStephen Hemminger <stephen at networkplumber.org> het volgende geschreven: > OFri, 23 Aug 2013 08:12:36 -0700 > Eric Dumaze<eric.dumazeat gmail.com> wrote: > >> OFri, 2013-08-23 a16:16 +0200, Teco Boot wrote: >>> I had reorder problems with neteqdisc. I makuse of the NRL CORE >>> network emulator, which sets up virtual routers and links, build with >>> network namespaces and netem. Typical CORE config for a n1<-->n2 link >>> with rat(1024Kbps), delay (50ms) and no jitter: >>> qdisc tbf 1: dev n1.eth0.222 roorefcn2 rate 1024Kbit burst 2999b >>> la488.6ms >>> qdisc nete10: dev n1.eth0.222 paren1:1 limit 1000 delay 50.0ms >>> qdisc tbf 1: dev n2.eth0.222 roorefcn2 rate 1024Kbit burst 2999b >>> la488.6ms >>> qdisc nete10: dev n2.eth0.222 paren1:1 limit 1000 delay 50.0ms >>> >>> I added jitter to this bi-directional link, e.g. 20ms. Now thdelay >>> for each packeis 50ms +/- 20ms is 30ms to 70ms. >>> However, this has somunexpected results: packets may breordered. >>> Thais becausthe actual delay is calculated for each packet. Some >>> packets hava larger delay, somhave a smaller. If the inter-packet >>> spacing is smaller thathtime differences set by netem (up to 2x >>> configured jitter), packets arreordered. In this examplwith >>> +/20ms, this is thcaswith packet rate larger than 25pps. >>> Reordering has bad effects otransporprotocol throughputs. >>> Reordering is less commoon thInternet, so I don't want to emulate >>> such. I don'say theris no reordering, I just say I don't want this >>> netebehavior. >>> >>> Thneteguide mentions this unexpected results. It was caused by a >>> changin version 1.1 (2.6.15). For peopllike me that do not want >>> this, theris a work-around mentioned. However, this doesn'work >>> anymorsincsomewhere before 2.6.31. >>> http://www.linuxfoundation.org/collaborate/workgroups/networking/netem >>> https://lists.linux-foundation.org/pipermail/netem/2011-April/001507.html >>> >>> So whato do? Threcent patch from Eric Dumazet eliminates >>> jitter_reordering wheneterate is configured. Maybe not the clean >>> approach, buacceptable. Ferry his patch eliminates >>> jitter_reordering. Sompeoplmay use reorder by jitter, so this >>> "feature" should nogeremoved. So Ferry his patch is not accepted. >>> Othother hand, people like me are very confused by current >>> behavior. >>> >>> Whawcould do is keep existing features and describe what netem >>> currently does. Thais: >>> - netewith delay and jitter may reorder packets, if inter packet >>> spacing is smaller thajitter >>> - reordering caused by delay and jitter cabturned off by using >>> neterate. ratcan be set to very high value is no shaping is >>> wanted. >> >> As long as oncan definthe expected behavior, you can add whatever >> new neteparameter. >> >> Oncould envision adding flow separation (skb->sk or rxhashing), so >> thaeach flow can havhis own local queue, to guarantee no reorders >> per flow _if_ this is needed, eveif per flow delays/jitter is/are >> configured. >> >> Walso usnetem to test on large scale, where the reordering stuff >> needs fixes itransporstacks (And yes, we are working on TCP stack >> to permihigher levels of reorders) > > I ahappy with any solution thais: > * allows both always ordering and reordering based orandojitter of delay > * documented > > I do geworried thapeople's tests get different results because of > netebehavior changes. Researchers likto have repeatable results. Questions thapops ouof of my head: What is more important, understandable configuration or keep existing behavior for unmodified configuration? Do we want to reorder, even with configured rate? My preference: makin understandable, documenwell and make all sensible options available. Teco Froeric.dumazeat gmail.com Fri Aug 23 16:03:58 2013 From: eric.dumazeagmail.com (Eric Dumazet) Date: Fri, 23 Aug 2013 09:03:58 -0700 Subject: netem: threorder discussion In-Reply-To: <31E9BEA1-3DE2-4DA7-9CEE-6B7C9C99F5EB@xxxxxxxxxx> References: <766825FB-5295-4C5C-ABA0-CC520EFE2EA6@xxxxxxxxxx> <1377270756.8828.5.camel@edumazet-glaptop> <20130823084450.6baf2835@xxxxxxxxxxxxxxxxxxxxxxxxxxx> <31E9BEA1-3DE2-4DA7-9CEE-6B7C9C99F5EB@xxxxxxxxxx> Message-ID: <1377273838.8828.11.camel@edumazet-glaptop> OFri, 2013-08-23 a17:53 +0200, Teco Boot wrote: > Questions thapops ouof of my head: What is more important, > understandablconfiguration or keep existing behavior for unmodified > configuration? Do wwanto reorder, even with configured rate? > > My preference: makin understandable, documenwell and make all > sensibloptions available. Thday 'rate' supporwas added, documentation became obsolete. Its actually hard to havboth ratand delay + jitters + reorders or not. If you cafix this properly, patches arwelcome, but I suspect you wanted to reintroducththing that made netem unusable for us. Froteco ainf-net.nl Fri Aug 23 16:18:44 2013 From: teco ainf-net.nl (Teco Boot) Date: Fri, 23 Aug 2013 18:18:44 +0200 Subject: netem: threorder discussion In-Reply-To: <1377273838.8828.11.camel@edumazet-glaptop> References: <766825FB-5295-4C5C-ABA0-CC520EFE2EA6@xxxxxxxxxx> <1377270756.8828.5.camel@edumazet-glaptop> <20130823084450.6baf2835@xxxxxxxxxxxxxxxxxxxxxxxxxxx> <31E9BEA1-3DE2-4DA7-9CEE-6B7C9C99F5EB@xxxxxxxxxx> <1377273838.8828.11.camel@edumazet-glaptop> Message-ID: <DF05DAC3-CED7-4615-AC02-F097EEC27752@xxxxxxxxxx> Op 23 aug. 2013 o18:03 heefEric Dumazet <eric.dumazet at gmail.com> het volgende geschreven: > OFri, 2013-08-23 a17:53 +0200, Teco Boot wrote: > >> Questions thapops ouof of my head: What is more important, >> understandablconfiguration or keep existing behavior for unmodified >> configuration? Do wwanto reorder, even with configured rate? >> >> My preference: makin understandable, documenwell and make all >> sensibloptions available. > > Thday 'rate' supporwas added, documentation became obsolete. > > Its actually hard to havboth ratand delay + jitters + reorders or > not. > > If you cafix this properly, patches arwelcome, but I suspect you > wanted to reintroducththing that made netem unusable for us. No, for surI do nowant to make it unusable for you. Or anybody. The opposite is true. I don'likthe "look in souce" documentation. Or parameters with hard to understand side effects. Frostephen anetworkplumber.org Fri Aug 23 15:44:50 2013 From: stepheanetworkplumber.org (Stephen Hemminger) Date: Fri, 23 Aug 2013 08:44:50 -0700 Subject: netem: threorder discussion In-Reply-To: <1377270756.8828.5.camel@edumazet-glaptop> References: <766825FB-5295-4C5C-ABA0-CC520EFE2EA6@xxxxxxxxxx> <1377270756.8828.5.camel@edumazet-glaptop> Message-ID: <20130823084450.6baf2835@xxxxxxxxxxxxxxxxxxxxxxxxxxx> OFri, 23 Aug 2013 08:12:36 -0700 Eric Dumaze<eric.dumazeat gmail.com> wrote: > OFri, 2013-08-23 a16:16 +0200, Teco Boot wrote: > > I had reorder problems with neteqdisc. I makuse of the NRL CORE > > network emulator, which sets up virtual routers and links, build with > > network namespaces and netem. Typical CORE config for a n1<-->n2 link > > with rat(1024Kbps), delay (50ms) and no jitter: > > qdisc tbf 1: dev n1.eth0.222 roorefcn2 rate 1024Kbit burst 2999b > > la488.6ms > > qdisc nete10: dev n1.eth0.222 paren1:1 limit 1000 delay 50.0ms > > qdisc tbf 1: dev n2.eth0.222 roorefcn2 rate 1024Kbit burst 2999b > > la488.6ms > > qdisc nete10: dev n2.eth0.222 paren1:1 limit 1000 delay 50.0ms > > > > I added jitter to this bi-directional link, e.g. 20ms. Now thdelay > > for each packeis 50ms +/- 20ms is 30ms to 70ms. > > However, this has somunexpected results: packets may breordered. > > Thais becausthe actual delay is calculated for each packet. Some > > packets hava larger delay, somhave a smaller. If the inter-packet > > spacing is smaller thathtime differences set by netem (up to 2x > > configured jitter), packets arreordered. In this examplwith > > +/20ms, this is thcaswith packet rate larger than 25pps. > > Reordering has bad effects otransporprotocol throughputs. > > Reordering is less commoon thInternet, so I don't want to emulate > > such. I don'say theris no reordering, I just say I don't want this > > netebehavior. > > > > Thneteguide mentions this unexpected results. It was caused by a > > changin version 1.1 (2.6.15). For peopllike me that do not want > > this, theris a work-around mentioned. However, this doesn'work > > anymorsincsomewhere before 2.6.31. > > http://www.linuxfoundation.org/collaborate/workgroups/networking/netem > > https://lists.linux-foundation.org/pipermail/netem/2011-April/001507.html > > > > So whato do? Threcent patch from Eric Dumazet eliminates > > jitter_reordering wheneterate is configured. Maybe not the clean > > approach, buacceptable. Ferry his patch eliminates > > jitter_reordering. Sompeoplmay use reorder by jitter, so this > > "feature" should nogeremoved. So Ferry his patch is not accepted. > > Othother hand, people like me are very confused by current > > behavior. > > > > Whawcould do is keep existing features and describe what netem > > currently does. Thais: > > - netewith delay and jitter may reorder packets, if inter packet > > spacing is smaller thajitter > > - reordering caused by delay and jitter cabturned off by using > > neterate. ratcan be set to very high value is no shaping is > > wanted. > > As long as oncan definthe expected behavior, you can add whatever > new neteparameter. > > Oncould envision adding flow separation (skb->sk or rxhashing), so > thaeach flow can havhis own local queue, to guarantee no reorders > per flow _if_ this is needed, eveif per flow delays/jitter is/are > configured. > > Walso usnetem to test on large scale, where the reordering stuff > needs fixes itransporstacks (And yes, we are working on TCP stack > to permihigher levels of reorders) I ahappy with any solution thais: * allows both always ordering and reordering based orandojitter of delay * documented I do geworried thapeople's tests get different results because of netebehavior changes. Researchers likto have repeatable results.
