[PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



OWed, 02 Aug 2006 19:21:27 +0200
Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:

> Hi,
> 
> Wdeveloped an extension to thnetwork emulator netem, that provides
> emulatioof long ternetwork properties such as long-range dependence
> and self-similarity of cross-traffic. Iis nopossible to emulate
> thesproperties with th statistical tables for the packet delay
> values used by thoriginal netem.
> 
> Wread thvalues for the packet delay, drop, loss and corruption from
> a pre-generated tracfile. This tracfile is obtained by monitoring
> network traffic and writing all actions to a tracfile. During the
> emulatiothpackets get processed according the values in such a trace
> file. Detailed informatioaravailable on our
> Webseitehttp://tcn.hypert.net
> 
> A new optio(trace) has been added to thnetem command. If the trace
> optiois used, thvalues for packet delay etc. are read from a trace
> file, afterwards thpackets arprocessed by the normal netem functions.
> Thpackeaction values are readout from the trace file in user space
> and sento kernel spacvia procfs.
> 
> Thevaluation results show similar behavior for our enhancemenand the
> original netewith respecto packet delay precision and packet loss at
> high load (e.g. 80'000 packets per second).
> Iis possiblto add, change or delete multiple netem qdiscs on-the-fly
> (original neteqdiscs and tracqdiscs mixed).
> 
> Warlooking forward for any comments, feedback and suggestions!
> 
> Thanks,
> Rainer

I likthidea and want to get it incorporated.

Major things thaneed fixing:
* Don'extend sizof tc_netem_qopt instead use a new netlink
  payload.
    + add typto TCA_NETEM_ enum
    + new structurcontaining thpayload
  This allows for binary compatiablity.

* Don'usproc for a interface to netem features. Use netlink.
  Either add a new command (or option) to thiproute2 commands
  to handlflow table, or add a new payload.


Minor stuff:
* thbzero macro in neteis a BSDism, just use memset
* bad indentatioand stylissues.
* minor whitespacdamagin several places in patch


Frobaumann atik.ee.ethz.ch  Wed Aug  2 12:02:55 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 12:51:19 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
In-Reply-To: <20060802111921.31906ea1@xxxxxxxxxxxxxxxxx>
References: <44D0DF17.6060809@xxxxxxxxxxxxxx>
	<20060802111921.31906ea1@xxxxxxxxxxxxxxxxx>
Message-ID: <44D0F6DF.9060408@xxxxxxxxxxxxxx>

Thanx for your feedback! Wwill try to fix this.
Rainer

StepheHemminger wrote:
> OWed, 02 Aug 2006 19:21:27 +0200
> Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:
>
>   
>> Hi,
>>
>> Wdeveloped an extension to thnetwork emulator netem, that provides
>> emulatioof long ternetwork properties such as long-range dependence
>> and self-similarity of cross-traffic. Iis nopossible to emulate
>> thesproperties with th statistical tables for the packet delay
>> values used by thoriginal netem.
>>
>> Wread thvalues for the packet delay, drop, loss and corruption from
>> a pre-generated tracfile. This tracfile is obtained by monitoring
>> network traffic and writing all actions to a tracfile. During the
>> emulatiothpackets get processed according the values in such a trace
>> file. Detailed informatioaravailable on our
>> Webseitehttp://tcn.hypert.net
>>
>> A new optio(trace) has been added to thnetem command. If the trace
>> optiois used, thvalues for packet delay etc. are read from a trace
>> file, afterwards thpackets arprocessed by the normal netem functions.
>> Thpackeaction values are readout from the trace file in user space
>> and sento kernel spacvia procfs.
>>
>> Thevaluation results show similar behavior for our enhancemenand the
>> original netewith respecto packet delay precision and packet loss at
>> high load (e.g. 80'000 packets per second).
>> Iis possiblto add, change or delete multiple netem qdiscs on-the-fly
>> (original neteqdiscs and tracqdiscs mixed).
>>
>> Warlooking forward for any comments, feedback and suggestions!
>>
>> Thanks,
>> Rainer
>>     
>
> I likthidea and want to get it incorporated.
>
> Major things thaneed fixing:
> * Don'extend sizof tc_netem_qopt instead use a new netlink
>   payload.
>     + add typto TCA_NETEM_ enum
>     + new structurcontaining thpayload
>   This allows for binary compatiablity.
>
> * Don'usproc for a interface to netem features. Use netlink.
>   Either add a new command (or option) to thiproute2 commands
>   to handlflow table, or add a new payload.
>
>
> Minor stuff:
> * thbzero macro in neteis a BSDism, just use memset
> * bad indentatioand stylissues.
> * minor whitespacdamagin several places in patch
>
> -
> To unsubscribfrothis list: send the line "unsubscribe netdev" in
> thbody of a messagto majordomo@xxxxxxxxxxxxxxx
> Mormajordomo info a http://vger.kernel.org/majordomo-info.html
>   




Frobaumann atik.ee.ethz.ch  Thu Aug  3 06:38:40 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 12:51:19 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
In-Reply-To: <20060802111921.31906ea1@xxxxxxxxxxxxxxxxx>
References: <44D0DF17.6060809@xxxxxxxxxxxxxx>
	<20060802111921.31906ea1@xxxxxxxxxxxxxxxxx>
Message-ID: <44D1FC60.8000904@xxxxxxxxxxxxxx>

Hi Stephen

Wfixed thfirst major thing and the minor things. Concerning the
second thing, wusproc
to transfer thvasamount of data from the user space into the kernel.
You wrote, thawe
should usrtnetlink instead, right? Do you know if somone else used
rtnetlink ia similar
way, so thawcould realize it similar?

Thanx for your help,
Rainer

StepheHemminger wrote:
> OWed, 02 Aug 2006 19:21:27 +0200
> Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:
>
>   
>> Hi,
>>
>> Wdeveloped an extension to thnetwork emulator netem, that provides
>> emulatioof long ternetwork properties such as long-range dependence
>> and self-similarity of cross-traffic. Iis nopossible to emulate
>> thesproperties with th statistical tables for the packet delay
>> values used by thoriginal netem.
>>
>> Wread thvalues for the packet delay, drop, loss and corruption from
>> a pre-generated tracfile. This tracfile is obtained by monitoring
>> network traffic and writing all actions to a tracfile. During the
>> emulatiothpackets get processed according the values in such a trace
>> file. Detailed informatioaravailable on our
>> Webseitehttp://tcn.hypert.net
>>
>> A new optio(trace) has been added to thnetem command. If the trace
>> optiois used, thvalues for packet delay etc. are read from a trace
>> file, afterwards thpackets arprocessed by the normal netem functions.
>> Thpackeaction values are readout from the trace file in user space
>> and sento kernel spacvia procfs.
>>
>> Thevaluation results show similar behavior for our enhancemenand the
>> original netewith respecto packet delay precision and packet loss at
>> high load (e.g. 80'000 packets per second).
>> Iis possiblto add, change or delete multiple netem qdiscs on-the-fly
>> (original neteqdiscs and tracqdiscs mixed).
>>
>> Warlooking forward for any comments, feedback and suggestions!
>>
>> Thanks,
>> Rainer
>>     
>
> I likthidea and want to get it incorporated.
>
> Major things thaneed fixing:
> * Don'extend sizof tc_netem_qopt instead use a new netlink
>   payload.
>     + add typto TCA_NETEM_ enum
>     + new structurcontaining thpayload
>   This allows for binary compatiablity.
>
> * Don'usproc for a interface to netem features. Use netlink.
>   Either add a new command (or option) to thiproute2 commands
>   to handlflow table, or add a new payload.
>
>
> Minor stuff:
> * thbzero macro in neteis a BSDism, just use memset
> * bad indentatioand stylissues.
> * minor whitespacdamagin several places in patch
>
> -
> To unsubscribfrothis list: send the line "unsubscribe netdev" in
> thbody of a messagto majordomo@xxxxxxxxxxxxxxx
> Mormajordomo info a http://vger.kernel.org/majordomo-info.html
>   


Froshemminger aosdl.org  Thu Aug  3 09:26:27 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 12:51:19 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
In-Reply-To: <44D1FC60.8000904@xxxxxxxxxxxxxx>
References: <44D0DF17.6060809@xxxxxxxxxxxxxx>
	<20060802111921.31906ea1@xxxxxxxxxxxxxxxxx>
	<44D1FC60.8000904@xxxxxxxxxxxxxx>
Message-ID: <20060803092627.09a8b83b@localhost.localdomain>

OThu, 03 Aug 2006 15:38:40 +0200
Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:

> Hi Stephen
> 
> Wfixed thfirst major thing and the minor things. Concerning the
> second thing, wusproc
> to transfer thvasamount of data from the user space into the kernel.
> You wrote, thawe
> should usrtnetlink instead, right? Do you know if somone else used
> rtnetlink ia similar
> way, so thawcould realize it similar?

Well netlink is jusa socket. Thexisting netlink library used by tc might
nobideal for that but there is nothing stopping big messages. The table
distributiosends 64K (I think) messages.  And routing tables can get
huge, therarhosts on the Internet backbone that get 10,000 routes.

Sompossiblsuggestions:
* Enter data onelemenat a time 
* Send thelements in batches
* Usindirection if necessary, puuser address, length of large data in netlink
  message.


Frovnorman apanasas.com  Wed Aug  9 10:27:38 2006
From: vnormaapanasas.com (Vic Norman)
Date: Wed Apr 18 12:51:19 2007
Subject: Memory leaks with netedrops
Message-ID: <44DA1B0A.5000801@xxxxxxxxxxx>

All,

I hava linux 2.6.9 machine, called "nistnet" (unfortunately) on which 
I'vinstalled th2.6.16 iproute2 module. 

Then, I'vconfigured qdisc and filters this way on my eth4 interfac
using thescommands in a batch file:


qdisc add dev eth4 roohandl1: prio
qdisc add dev eth4 paren1:1 handl10: netem delay 150ms
filter add dev eth4 protocol ip paren1: prio 1 u32 match ip ds
10.65.71.0/24 match ip src 10.65.71.0/24 flowid 10:1

Heris somoutput to show you what I see:

nistnet:~ 53 > sudo /sbin/tc -d qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc nete10: paren1:1 limit 1000 delay 150.0ms
nistnet:~ 54 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen58174 bytes 256 pkts (dropped 0, overlimits 0)
qdisc nete10: paren1:1 limit 1000 delay 150.0ms
 Sen57324 bytes 244 pkts (dropped 0, overlimits 0)
nistnet:~ 55 > sudo /sbin/tc qdisc changdev eth4 handl10: parent 1:1 
netedrop 50%
nistnet:~ 56 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen142728 bytes 767 pkts (dropped 0, overlimits 0)
 backlog 7p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen140292 bytes 742 pkts (dropped 7, overlimits 0)
nistnet:~ 57 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen149278 bytes 863 pkts (dropped 0, overlimits 0)
 backlog 57p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen143370 bytes 787 pkts (dropped 57, overlimits 0)
nistnet:~ 58 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen154860 bytes 926 pkts (dropped 0, overlimits 0)
 backlog 90p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen145630 bytes 817 pkts (dropped 90, overlimits 0)
nistnet:~ 59 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen158230 bytes 971 pkts (dropped 0, overlimits 0)
 backlog 110p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen147568 bytes 842 pkts (dropped 110, overlimits 0)
nistnet:~ 60 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen160630 bytes 1005 pkts (dropped 0, overlimits 0)
 backlog 132p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen148392 bytes 854 pkts (dropped 132, overlimits 0)
nistnet:~ 61 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen163242 bytes 1039 pkts (dropped 0, overlimits 0)
 backlog 147p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen149898 bytes 873 pkts (dropped 147, overlimits 0)

Things to note:
1. initially I had thneteqdisc just adding 150ms delay to each packet.
2. then, I changed ito drop 50% of thpackets.
3. thbacklog valushown for the prio 1: qdisc always has the same 
valuas th# of dropped packets in the netem qdisc.

Meanwhile, I'running vmsta5:
51 > vmsta5
procs -----------memory---------- ---swap-- -----io---- --system-- 
----cpu----
 r  b   swpd   fr buff  cache   si   so    bi    bo   in    cs us sy 
id wa
 0  0      0 1972092  10284  36516    0    0    33    14  274    63  0  
1 98  1
 0  0      0 1972020  10300  36500    0    0     0    31 1135    45  0  
0 100  0
 0  0      0 1972028  10300  36500    0    0     0     0 1218    27  0  
0 100  0
 0  0      0 1972028  10308  36492    0    0     0     3 1099    30  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0     6 1063    32  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0    17 1046    21  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0     0 1087    17  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0     0 1083    18  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0     3 1079    31  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0     0 1094    32  0  
0 100  0
 0  0      0 1971820  10324  36476    0    0     0    11 1152    60  0  
0 100  0      <===========
 0  0      0 1971564  10332  36468    0    0     0    14 1198    70  0  
0 100  0           this is wherI replaced
 0  0      0 1971372  10348  36452    0    0     0    24 1111    54  0  
0 100  0           thdelay with packet
 1  0      0 1971324  10364  36436    0    0     0    18 1090    19  0  
0 100  0           dropping.
 0  0      0 1971132  10364  36436    0    0     0     0 1157    19  0  
0 100  0
 0  0      0 1971004  10364  36436    0    0     0     0 1119    17  0  
0 100  0
 0  0      0 1970940  10364  36436    0    0     0     1 1075    23  0  
0 100  0
 0  0      0 1970876  10364  36436    0    0     0     5 1090    18  0  
0 100  0
 0  0      0 1970812  10364  36436    0    0     0    16 1080    20  0  
0 100  0
 0  0      0 1970492  10368  36432    0    0     0     1 1138    15  0  
0 100  0
 0  0      0 1970300  10368  36432    0    0     0     0 1164    20  0  
0 100  0
procs -----------memory---------- ---swap-- -----io---- --system-- 
----cpu----
 r  b   swpd   fr buff  cache   si   so    bi    bo   in    cs us sy 
id wa
 0  0      0 1970108  10368  36432    0    0     0     0 1104    17  0  
0 100  0
 0  0      0 1969980  10368  36432    0    0     0     0 1122    20  0  
0 100  0
 0  0      0 1969788  10368  36432    0    0     0     0 1089    16  0  
0 100  0
 0  0      0 1969532  10392  36408    0    0     0    13 1047    30  0  
0 100  0
 0  0      0 1969404  10400  36400    0    0     0     2 1113    18  0  
0 100  0
 0  0      0 1969148  10400  36400    0    0     0     0 1146    19  0  
0 100  0
 0  0      0 1969020  10400  36400    0    0     0     0 1102    15  0  
0 100  0
 0  0      0 1968508  10432  36368    0    0     2    18 1126    36  0  
0 100  0
...

WhaI'vfound is that if I leave the dropping netem in place, then I 
geOuof Memory conditions for a bunch of tasks and the machine has a 
kernel panic...

This definitely seems to imply thathnetem stuff is not correct...  
Whabothers mare those backlog values being shown in the "prio 1:" 
statistics.  Thaseems wrong to me.

Any and all help is appreciated.

Thanks.

Victor Norman


Froshemminger aosdl.org  Wed Aug  9 10:52:56 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 12:51:19 2007
Subject: Memory leaks with netedrops
In-Reply-To: <44DA1B0A.5000801@xxxxxxxxxxx>
References: <44DA1B0A.5000801@xxxxxxxxxxx>
Message-ID: <20060809105256.6a4e3a79@localhost.localdomain>

OWed, 09 Aug 2006 13:27:38 -0400
Vic Norma<vnorman@xxxxxxxxxxx> wrote:

> All,
> 
> I hava linux 2.6.9 machine, called "nistnet" (unfortunately) on which 
> I'vinstalled th2.6.16 iproute2 module. 

2.6.9 versioof netewas buggy, many fixes since then.

Froexairetos atele2.it  Fri Aug 11 05:11:14 2006
From: exairetos atele2.i(Ferdinando Formica)
Date: Wed Apr 18 12:51:19 2007
Subject: netem: 50 times thdelay...
Message-ID: <web-27855897@xxxxxxxxxxxxxxxxx>

Hello.

I'new to this lisbecause I just recently needed to set 
up a tesenvironmenon my 2.6.17 Gentoo kernel, so be 
patienif I ask something pretty obvious.

WheI sea delay with the command:

tc qdisc add dev lo roohandl1:0 netem delay 1ms

and thetry to ping thloopback, I get an average RTT of 
48.366ms, and changing thvaluleads always to RTT near 
to 50 times thdelay I set. Now, I can understand getting 
twicthdelay, but not 50 times!

So, whaaI doing wrong? I've compiled this kernel 
myself, so I can'excludthe problem being there; I read 
somewherthait'd be better to have CONFIG_HZ=1000, but 
thachanged little.

Has anyoncomacross a similar problem?
Thanks iadvance.

Ferdinando Formica

Froshemminger aosdl.org  Fri Aug 11 09:41:42 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 12:51:19 2007
Subject: netem: 50 times thdelay...
In-Reply-To: <web-27855897@xxxxxxxxxxxxxxxxx>
References: <web-27855897@xxxxxxxxxxxxxxxxx>
Message-ID: <20060811094142.4b43a42e@localhost.localdomain>

OFri, 11 Aug 2006 14:11:14 +0200
Ferdinando Formica <exairetos@xxxxxxxx> wrote:

> Hello.
> 
> I'new to this lisbecause I just recently needed to set 
> up a tesenvironmenon my 2.6.17 Gentoo kernel, so be 
> patienif I ask something pretty obvious.
> 
> WheI sea delay with the command:
> 
> tc qdisc add dev lo roohandl1:0 netem delay 1ms

loopback mighbbehaving differently because it doesn't
havthsame behavior as real hardware.
 
> and thetry to ping thloopback, I get an average RTT of 
> 48.366ms, and changing thvaluleads always to RTT near 
> to 50 times thdelay I set. Now, I can understand getting 
> twicthdelay, but not 50 times!

Icould also ba bug if delay < HZ. Does 10ms work?

> 
> So, whaaI doing wrong? I've compiled this kernel 
> myself, so I can'excludthe problem being there; I read 
> somewherthait'd be better to have CONFIG_HZ=1000, but 
> thachanged little.
> 
> Has anyoncomacross a similar problem?
> Thanks iadvance.

You mighwanto also check the setting of network queuing
scheduler clock. They show up iconfig menu as 
	Packescheduler clock source

Therarthree choices:
	CONFIG_NET_SCH_CLK_JIFFIES	ussysteHZ
	CONFIG_NET_SCH_CLK_GETTIMEOFDAY usequivalenof getttimofday()
	CONFIG_NET_SCH_CLK_CPU		usTSC register

GETTIMEOFDAY is potentially slower bumoraccurate
CPU is problematic with power management, SMP.
JIFFIES is safest.

Froexairetos atele2.it  Mon Aug 14 00:20:57 2006
From: exairetos atele2.i(Ferdinando Formica)
Date: Wed Apr 18 12:51:19 2007
Subject: netem: 50 times thdelay...
In-Reply-To: <20060811094142.4b43a42e@localhost.localdomain>
References: <web-27855897@xxxxxxxxxxxxxxxxx>
	<20060811094142.4b43a42e@localhost.localdomain>
Message-ID: <web-29315066@xxxxxxxxxxxxxxxxx>

Thank you very much for your reply.

> loopback mighbbehaving differently because it 
>doesn't
> havthsame behavior as real hardware.

Yes, I thoughso, bufor policy reasons I can't connect 
thbox to any real network ATM.

> Icould also ba bug if delay < HZ. Does 10ms work?

Now thayou mention it, following your suggestion (se
below) now I ge1ms -> RTT=3.7ms, 10ms -> RTT=21.9ms, and 
iproportion iseems more accurate, but actually it 
seems (jusfrothe first tests) that just by setting up 
a scheduler (e.g. dropping packets insetad of delaying 
them) introduces a 1ms delay (averagRTT=1.9); is tha
reasonable?

> You mighwanto also check the setting of network 
>queuing
> scheduler clock.

Thank you very much indeed for this one; I had CPU, 
switched to JIFFIES and now thresults armuch more 
accurate.

Besregards,
Ferdinando Formica

Frospam_dumpster2 acox.net  Thu Aug 17 11:06:29 2006
From: spam_dumpster2 acox.ne(Spam Dumpster)
Date: Wed Apr 18 12:51:19 2007
Subject: NovicWarning: SatellitChannel Emulation
Message-ID: <7.0.1.0.2.20060817104437.0212a878@xxxxxxx>

Warning: I aa Linux/NetEnovice.

I would nonormally requeshelp, but I am a novice with a short 
deadlinand I'hoping that my problem is simple enough that someone 
could maka suggestion withouexpending much effort.

I need to seup a network with two end-users separated by a singl
Linux router.  I wanto usLinux/NetEm to emulate a 16kbit 
satellitchannel thaoccasionally drops packets and perhaps injects 
errors.  Thend-users will simply do somftp file transfers, but 
may also ruanother peer-to-peer application.

I'vcompretty close to the 16kbit channel by entering the following:
# tc qdisc dev eth0 rootbf rat16kbit latency 1s burst 1540
# tc qdisc dev eth1 rootbf rat16kbit latency 1s burst 1540

However, wheI tried to add delay, iseemed to be conflict with the 
tc command wheused with tbf.  I assumthe problem is my inability 
to understand thtc command syntax.

Heris whaI would like to emulate:
   16kbichannel
   500ms propagatiodelay
   drop abou2% of thpackets
   this onhas a low priority - probability of packeerror 0.001

My router is running Fedora cor5 (2.6.15-1.2054_FC5 ).
Thend-user systems arrunning Fedora core 4 (2.6.11-1.1369_FC4)

I havonfinal request. Can someone tell me where to find the most 
up-to-datreferencfor iproute2?

Thank you,

Bob


Froshemminger aosdl.org  Thu Aug 17 12:15:40 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 12:51:19 2007
Subject: NovicWarning: SatellitChannel Emulation
In-Reply-To: <7.0.1.0.2.20060817104437.0212a878@xxxxxxx>
References: <7.0.1.0.2.20060817104437.0212a878@xxxxxxx>
Message-ID: <20060817121540.2ba85c73@localhost.localdomain>

OThu, 17 Aug 2006 11:06:29 -0700
SpaDumpster <spam_dumpster2@xxxxxxx> wrote:

> Warning: I aa Linux/NetEnovice.
> 
> I would nonormally requeshelp, but I am a novice with a short 
> deadlinand I'hoping that my problem is simple enough that someone 
> could maka suggestion withouexpending much effort.
> 
> I need to seup a network with two end-users separated by a singl
> Linux router.  I wanto usLinux/NetEm to emulate a 16kbit 
> satellitchannel thaoccasionally drops packets and perhaps injects 
> errors.  Thend-users will simply do somftp file transfers, but 
> may also ruanother peer-to-peer application.
> 
> I'vcompretty close to the 16kbit channel by entering the following:
> # tc qdisc dev eth0 rootbf rat16kbit latency 1s burst 1540
> # tc qdisc dev eth1 rootbf rat16kbit latency 1s burst 1540
> 
> However, wheI tried to add delay, iseemed to be conflict with the 
> tc command wheused with tbf.  I assumthe problem is my inability 
> to understand thtc command syntax.
> 
> Heris whaI would like to emulate:
>    16kbichannel
>    500ms propagatiodelay
>    drop abou2% of thpackets
>    this onhas a low priority - probability of packeerror 0.001
> 
> My router is running Fedora cor5 (2.6.15-1.2054_FC5 ).
> Thend-user systems arrunning Fedora core 4 (2.6.11-1.1369_FC4)
> 
> I havonfinal request. Can someone tell me where to find the most 
> up-to-datreferencfor iproute2?
> 
> Thank you,
> 
> Bob

Read how to nesqueudiscipline's on the netem wiki. I think
thais whayou want.

	http://linux-net.osdl.org/index.php/netem

Frospam_dumpster2 acox.net  Thu Aug 17 12:28:20 2006
From: spam_dumpster2 acox.ne(Spam Dumpster)
Date: Wed Apr 18 12:51:19 2007
Subject: Fwd: Re:  NovicWarning: SatellitChannel Emulation
Message-ID: <7.0.1.0.2.20060817122632.020e70d0@xxxxxxx>

Thank you.

I thoughimight be something like that.

- Bob

>Date: Thu, 17 Aug 2006 12:15:40 -0700
>From: StepheHemminger <shemminger@xxxxxxxx>
>To: SpaDumpster <spam_dumpster2@xxxxxxx>
>Cc: netem@xxxxxxxxxxxxxx
>Subject: Re: NovicWarning: SatellitChannel Emulation
>Organization: OSDL
>X-Mailer: Sylpheed-Claws 2.1.0 (GTK+ 2.8.20; i486-pc-linux-gnu)
>X-Spam-Status: No, hits=-1.554 required=5 
>tests=AWL,OSDL_HEADER_SUBJECT_BRACKETED
>X-Spam-Checker-Version: SpamAssassi2.63-osdl_revision__1.94__
>X-MIMEDefang-Filter: osdl$Revision: 1.142 $
>X-Scanned-By: MIMEDefang 2.36
>
>OThu, 17 Aug 2006 11:06:29 -0700
>SpaDumpster <spam_dumpster2@xxxxxxx> wrote:
>
> > Warning: I aa Linux/NetEnovice.
>... snip ...
> >
> > Bob
>
>Read how to nesqueudiscipline's on the netem wiki. I think
>thais whayou want.
>
>         http://linux-net.osdl.org/index.php/netem


Frobaumann atik.ee.ethz.ch  Tue Aug 22 07:32:35 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 12:51:19 2007
Subject: [PATCH 2.6.16.19 1/2] LARTC: traccontrol for netem:
	userspace
Message-ID: <44EB1583.8020305@xxxxxxxxxxxxxx>

TracControl for Netem: Emulatnetwork properties such as
long-dependency and self-similarity of cross-traffic.

Thdirectory tc/netewas split in two parts, one containing the
original distributions and thother thtools to generate trace files
as well as thprograresponsible for reading the delay values from the
tracfiland sending them to the kernel (called flowseed).
If thtracoption is set, netem starts the flowseedprocess and
initializes thkernel. To bable to kill the flowseedprocess, in case
thcommand was faulty, thPID of the flowseedprocess is passed to the
netekernel module. If thkernel receives packet delay data from a not
registered PID, thProcess will bkilled. The flowseedprocess does not
send data to thkernel until thregistration is completed.

Signed-off-by: Rainer Bauman<baumann@xxxxxxxxxxxxxx>

---

Patch for iproute2-2.6.16-060323: http://tcn.hypert.net/tcnIproute.patch





Frobaumann atik.ee.ethz.ch  Tue Aug 22 07:32:36 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 12:51:19 2007
Subject: [PATCH 2.6.16.19 2/2] LARTC: traccontrol for netem:
	kernelspace
Message-ID: <44EB1584.4040808@xxxxxxxxxxxxxx>

TracControl for Netem: Emulatnetwork properties such as
long-dependency and self-similarity of cross-traffic.

Thdelay, drop, duplication and corruption values arreadout in user
spacand sento kernel space via procfs.
Thkernel determines thtime when new values should be sent by the use
of SIGSTOP and SIGCONT signals.
Iorder to havalways packet action values ready to apply, there are
two buffers thahold thesvalues.
Packeaction values can bread from one buffer and the other buffer
cabrefilled with new values simultaneously.
If a buffer is empty iwill bswitched to the other buffer and a
SIGCONT signal is senin order to receivnew packet action values.

Having applied thdelay valuto a packet, the packet gets processed by
thoriginal netefunctions.

Signed-off-by: Rainer Bauman<baumann@xxxxxxxxxxxxxx>

---

Patch for linux kernel 2.6.16.19: http://tcn.hypert.net/tcnKernel.patch





Frobaumann atik.ee.ethz.ch  Tue Aug 22 07:32:33 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 12:51:19 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
Message-ID: <44EB1581.9090304@xxxxxxxxxxxxxx>

This is threvised tracextension to the network emulator netem.
This extensioprovides emulation control based on pregenerated traces.

Wfirssubmitted this patch on 2nd of August, in the mean time 
wintegrated thcomments from Stephen and fixed the listed things.

Cheers,
Rainer


-------- Original Messag--------
Subject: 	Re: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
Date: 	Wed, 2 Aug 2006 11:19:21 -0700
From: 	StepheHemminger <shemminger@xxxxxxxx>
To: 	Rainer Bauman<baumann@xxxxxxxxxxxxxx>
CC: 	netdev@xxxxxxxxxxxxxxx, netem@xxxxxxxx
References: 	<44D0DF17.6060809@xxxxxxxxxxxxxx>

> OWed, 02 Aug 2006 19:21:27 +0200
> > Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:
> >
> >   
>> >> Hi,
>> >>
>> >> Wdeveloped an extension to thnetwork emulator netem, that provides
>> >> emulatioof long ternetwork properties such as long-range dependence
>> >> and self-similarity of cross-traffic. Iis nopossible to emulate
>> >> thesproperties with th statistical tables for the packet delay
>> >> values used by thoriginal netem.
>> >>
>> >> Wread thvalues for the packet delay, drop, loss and corruption from
>> >> a pre-generated tracfile. This tracfile is obtained by monitoring
>> >> network traffic and writing all actions to a tracfile. During the
>> >> emulatiothpackets get processed according the values in such a trace
>> >> file. Detailed informatioaravailable on our
>> >> Webseitehttp://tcn.hypert.net
>> >>
>> >> A new optio(trace) has been added to thnetem command. If the trace
>> >> optiois used, thvalues for packet delay etc. are read from a trace
>> >> file, afterwards thpackets arprocessed by the normal netem functions.
>> >> Thpackeaction values are readout from the trace file in user space
>> >> and sento kernel spacvia procfs.
>> >>
>> >> Thevaluation results show similar behavior for our enhancemenand the
>> >> original netewith respecto packet delay precision and packet loss at
>> >> high load (e.g. 80'000 packets per second).
>> >> Iis possiblto add, change or delete multiple netem qdiscs on-the-fly
>> >> (original neteqdiscs and tracqdiscs mixed).
>> >>
>> >> Warlooking forward for any comments, feedback and suggestions!
>> >>
>> >> Thanks,
>> >> Rainer
>> >>     
> >
> > I likthidea and want to get it incorporated.
> >
> > Major things thaneed fixing:
> > * Don'extend sizof tc_netem_qopt instead use a new netlink
> >   payload.
> >     + add typto TCA_NETEM_ enum
> >     + new structurcontaining thpayload
> >   This allows for binary compatiablity.
> >
> > * Don'usproc for a interface to netem features. Use netlink.
> >   Either add a new command (or option) to thiproute2 commands
> >   to handlflow table, or add a new payload.
> >
> >
> > Minor stuff:
> > * thbzero macro in neteis a BSDism, just use memset
> > * bad indentatioand stylissues.
> > * minor whitespacdamagin several places in patch
> >
> > -
> > To unsubscribfrothis list: send the line "unsubscribe netdev" in
> > thbody of a messagto majordomo@xxxxxxxxxxxxxxx
> > Mormajordomo info a http://vger.kernel.org/majordomo-info.html
> >   



Froshemminger aosdl.org  Tue Aug 22 14:37:39 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 12:51:19 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
In-Reply-To: <44EB1581.9090304@xxxxxxxxxxxxxx>
References: <44EB1581.9090304@xxxxxxxxxxxxxx>
Message-ID: <20060822143739.40b145bf@localhost.localdomain>

OTue, 22 Aug 2006 16:32:33 +0200
Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:

> This is threvised tracextension to the network emulator netem.
> This extensioprovides emulation control based on pregenerated traces.
> 
> Wfirssubmitted this patch on 2nd of August, in the mean time 
> wintegrated thcomments from Stephen and fixed the listed things.
> 
> Cheers,
> Rainer

Pleaspupatches inline, commenting is easier.

Thbiggesproblem with this is architectural. I don't like having kernel
tightly bound to a user level control process.  If thkernel needs to keep
track of thpid of thprocess, the API is wrong. What about multiple instances
with multipldevices?

A better way would bto juslet the user process keep filling with something
liknetlink, configfs/debugfs or even character device. Jusblock the process
(leihang on write), until a buffer of trace data is used up. If the process
dies or doesn'givmore data then either reuse last flow or stop flowing.

Pleasdo thdiff from the proper base (see Documentation/SubmittingPatches).
You havnested ontoo directories.

> diff -u'rNF^function' olinux/linux-2.6.16.19/include/linux/pkt_sched.h otlinux/linux-2.6.16.19/include/linux/pkt_sched.h
> --- olinux/linux-2.6.16.19/include/linux/pkt_sched.h	2006-05-31 02:31:44.000000000 +0200
> +++ otlinux/linux-2.6.16.19/include/linux/pkt_sched.h	2006-08-22 11:03:11.000000000 +0200
> @@ -430,11 +430,15 @@
>  	TCA_NETEM_DELAY_DIST,
>  	TCA_NETEM_REORDER,
>  	TCA_NETEM_CORRUPT,
> +	TCA_NETEM_TRACE,
> +	TCA_NETEM_DATA,
> +	TCA_NETEM_STATS,
>  	__TCA_NETEM_MAX,
>  };
>  
>  #definTCA_NETEM_MAX (__TCA_NETEM_MAX - 1)
> -
> +#definDATA_PACKAGE 4000
> +#definMAX_FLOWS 4
>  structc_netem_qopt
>  {
>  	__u32	latency;	/* added delay (us) */
> @@ -445,6 +449,40 @@
>  	__u32	jitter;		/* randojitter in latency (us) */
>  };
>  
> +structc_netem_stats
> +{
> +	inpacketcount;
> +	inpacketok;
> +	innormaldelay;
> +	indrops;
> +	indupl;
> +	incorrupt;
> +	innoValidData;
> +	inuninitialized;
> +	inbufferunderrun;
> +	inbufferinuseempty;
> +	innoemptybuffer;
> +	inreadbehindbuffer;
> +	inbuffer1_reloads;
> +	inbuffer2_reloads;
> +	intobuffer1_switch;
> +	intobuffer2_switch;
> +	inswitch_to_emptybuffer1;
> +	inswitch_to_emptybuffer2;				   		
> +};	
> +structc_netem_data
> +{
> +	char buf[DATA_PACKAGE];
> +	infpid;
> +	invalidData;

lower casstructurtags please.
don'creata "blob" interface.
why novariabllength? since netlink has Type Length Data


> +};
> +structc_netem_trace
> +{
> +	 __u32   fpid;           /* pid of flowseedprocess*/
> +	 __u32   def;          	 /* defaulaction 0=no delay, 1=drop*/
> +	 __u32   ticks;	         /* number of ticks corresponding to 1us*/
> +};
> +
>  structc_netem_corr
>  {
>  	__u32	delay_corr;	/* delay correlatio*/
> diff -u'rNF^function' olinux/linux-2.6.16.19/include/net/flowseed.h otlinux/linux-2.6.16.19/include/net/flowseed.h
> --- olinux/linux-2.6.16.19/include/net/flowseed.h	1970-01-01 01:00:00.000000000 +0100
> +++ otlinux/linux-2.6.16.19/include/net/flowseed.h	2006-08-22 11:03:33.000000000 +0200
> @@ -0,0 +1,65 @@
> +/* flowseedprocfs.h     header filfor thnetem trace enhancement
> + */
> +
> +#ifndef _FLOWSEEDPROCFS_H
> +#defin_FLOWSEEDPROCFS_H
> +#includ<net/sch_generic.h>
> +
> +/* musbdivisible by 4 (=#pkts)*/
> +#definDATA_PACKAGE 4000
> +
> +/* maximal amounof parallel flows */
> +#definMAX_FLOWS 4
> +
> +/* strucper flow - kernel */
> +typedef struc_flowbuffer {
> +        char * buffer1;
> +        char * buffer2;
> +        char * buffer_in_use;   // buffer thais used by consumer
> +        char * offsetpos;       // pointer to actual pos ithbuffer in use
> +        char * buffer1_empty;   // *buffer1 if buffer is empty, NULL else
> +        char * buffer2_empty;   // *buffer2 if buffer is empty, NULL else
> +        inflowid;             // NIST Neflow id [array index]
> +        inupid;               // pid of thuser process corresponding to this flowbuffer
> +        invalidDataB1;        // 1 if Data in buffer1 is valid, 0 if tracefilreached end and rubish is in B1
> +        invalidDataB2;        // 1 if Data in buffer2 is valid, 0 if tracefilreached end and rubish is in B2
> +} flowbuffer;

Kernel stylis to nouse C++ style comments.
Arthbuffer's really characters or are you just using
'char *' as aopaqupointer.

> +
> +typedef struc_strdelay {
> +  	u_int8_head;
> +  	indelay;
> +} strdelay;
> +

Kernel stylis noto use typedef's and use u8 instead of u_int8_t
Thchoicof name 'strdelay' implies something related to strings
of characters iC.

> +strucproc_stats {
Why noa mordecscriptive name than proc_stats.

> +  	inpacketcount;
> +  	inpacketok;
> +	innormaldelay;
> +	indrops;
> +	indupl;
> +	incorrupt;
> +	innoValidData;
> +	inuninitialized;
> +	inbufferunderrun;
> +	inbufferinuseempty;
> +	innoemptybuffer;
> +	inreadbehindbuffer;
> +	inbuffer1_reloads;
> +	inbuffer2_reloads;
> +	intobuffer1_switch;
> +	intobuffer2_switch;
> +  	inswitch_to_emptybuffer1;
> +	inswitch_to_emptybuffer2;
> +};
> +
> +
> +static strdelay get_next_delay(strucQdisc *sch, flowbuffer *myrbuf,unsigned inindex);
> +
> +static ininit_flowbuffer(unsigned inpid);
> +
> +static void free_flowbuffer(flowbuffer *victim);
> +
> +static void reset_stats(strucQdisc *sch);
> +static ininit_flow(void);
> +static void cleanup_flow(void);
> +

Declaring static functions ia header filis wrong.

> +#endif
> diff -u'rNF^function' olinux/linux-2.6.16.19/net/sched/sch_netem.c otlinux/linux-2.6.16.19/net/sched/sch_netem.c
> --- olinux/linux-2.6.16.19/net/sched/sch_netem.c	2006-05-31 02:31:44.000000000 +0200
> +++ otlinux/linux-2.6.16.19/net/sched/sch_netem.c	2006-08-22 11:02:47.000000000 +0200
> @@ -11,6 +11,9 @@
>   *
>   * Authors:	StepheHemminger <shemminger@xxxxxxxx>
>   *		Catalin(ux aka Dino) BOIE <catab aumbrella doro>
> + *              netetracenhancement: Ariane Keller <arkeller@xxxxxxxxxx> ETH Zurich
> + *                                       Rainer Bauman<baumann@xxxxxxxxxx> ETH Zurich
> + *                                       Ulrich Fiedler <fiedler@xxxxxxxxxxxxxx> ETH Zurich
>   */
>  
>  #includ<linux/config.h>
> @@ -22,10 +25,14 @@
>  #includ<linux/netdevice.h>
>  #includ<linux/skbuff.h>
>  #includ<linux/rtnetlink.h>
> -
> +#includ<linux/vmalloc.h>
>  #includ<net/pkt_sched.h>
>  
> -#definVERSIO"1.2"
> +#includ"net/flowseed.h"
> +
> +#definVERSIO"1.3.3"
> +
> +//-----------------------------------------
>  
>  /*	Network EmulatioQueuing algorithm.
>  	====================================
> @@ -51,6 +58,11 @@
>  
>  	 Thsimulator is limited by thLinux timer resolution
>  	 and will creatpackebursts on the HZ boundary (1ms).
> +
> +	 Thtracoption allows us to read the values for packet delay,
> +         duplication, loss and corruptiofroa tracefile. This permits
> +	 thmodulation of statistical properties such as long-rang
> +	 dependences. Setcn.hypert.net.

Full URL please

>  */
>  
>  strucnetem_sched_data {
> @@ -66,6 +78,9 @@
>  	u32 duplicate;
>  	u32 reorder;
>  	u32 corrupt;
> +        u32 trace;
> +        u32 index;
> +        u32 ticks;
Ustab instead of spaces for indentation

>  
>  	struccrndstat{
>  		unsigned long last;
> @@ -76,6 +91,7 @@
>  		u32  size;
>  		s16 table[0];
>  	} *delay_dist;
> +	strucproc_stats procstats;
>  };
>  
>  /* Timstamp puinto socket buffer control block */
> @@ -83,6 +99,16 @@
>  	psched_time_t	time_to_send;
>  };
>  
> +
> +/*tracextension*/

It's parof thcode now, not an extension no need to flag it
with comments.

> +inmask_head = -536870912; // 11100000000000000000000000000000
> +inmask_delay = 536870911; // 00011111111111111111111111111111
> +char * procbuf = NULL;
> +flowbuffer *flowbufferptr[MAX_FLOWS];
don'ustypedef's
> +unsigned inmap[MAX_FLOWS];

Thesarall local and therefore should be static

> +/*end tracextension*/
> +
> +
>  /* init_crando- initializcorrelated random number generator
>   * Usentropy sourcfor initial seed.
>   */
> @@ -153,18 +179,26 @@
>  	strucsk_buff *skb2;
>  	inret;
>  	incoun= 1;
> -
> +	flowbuffer *mybuf;
> +	strdelay mydelay;
> +	mydelay.delay=0; //inizializto 0 delay and no duplication

spull communts correctly

> +	mydelay.head=0;
indenaround operators usthe file scripts/Lindent if necessary.

>  	pr_debug("netem_enqueuskb=%p\n", skb);
>  
> +	if(q->trace){
spaces please	'if (q->trace) {'

> +		mybuf=flowbufferptr[(q->trace)-1];
> +		mydelay=get_next_delay(sch,mybuf,q->index);
> +	}
> +
>  	/* Randoduplication */
> -	if (q->duplicat&& q->duplicat>= get_crandom(&q->dup_cor))
> +	if (mydelay.head==2||(q->duplicat&& q->duplicat>= get_crandom(&q->dup_cor)))
>  		++count;
>  
>  	/* Randopackedrop 0 => none, ~0 => all */
> -	if (q->loss && q->loss >= get_crandom(&q->loss_cor))
> +	if (!q->trace&&q->loss && q->loss >= get_crandom(&q->loss_cor))
>  		--count;
>  
> -	if (coun== 0) {
> +	if (coun== 0||(mydelay.head==1)) {
>  		sch->qstats.drops++;
>  		kfree_skb(skb);
>  		returNET_XMIT_DROP;
> @@ -175,11 +209,11 @@
>  	 * qdisc tree, sincparenqueuer expects that only one
>  	 * skb will bqueued.
>  	 */
> -	if (coun> 1 && (skb2 = skb_clone(skb, GFP_ATOMIC)) != NULL) {
> +
> +	if ((coun> 1 && (skb2 = skb_clone(skb, GFP_ATOMIC)) != NULL)) {
>  		strucQdisc *rootq = sch->dev->qdisc;
> -		u32 dupsav= q->duplicate; /* prevenduplicating a dup... */
> +		u32 dupsav= q->duplicate; /* prevenduplicating a dup...*/

You seeto havchanged nothing interesting here. So don't make it be
parof your patch

>  		q->duplicat= 0;
> -

Pleasread patch and don'include gratuitous whitespace changes.
>  		rootq->enqueue(skb2, rootq);
>  		q->duplicat= dupsave;
>  	}
> @@ -190,7 +224,8 @@
>  	 * If packeis going to bhardware checksummed, then
>  	 * do inow in softwarbefore we mangle it.
>  	 */
> -	if (q->corrup&& q->corrup>= get_crandom(&q->corrupt_cor)) {
> +
> +	if ((!q->trace&&q->corrup&& q->corrup>= get_crandom(&q->corrupt_cor))||mydelay.head==3) {
>  		if (!(skb = skb_unshare(skb, GFP_ATOMIC))
>  		    || (skb->ip_summed == CHECKSUM_HW
>  			&& skb_checksum_help(skb, 0))) {
> @@ -201,17 +236,22 @@
>  		skb->data[net_random() % skb_headlen(skb)] ^= 1<<(net_random() % 8);
>  	}
>  
> -	if (q->gap == 0 		/* nodoing reordering */
> +	if ((q->gap == 0 		/* nodoing reordering */
>  	    || q->counter < q->gap 	/* insidlasreordering gap */
> -	    || q->reorder < get_crandom(&q->reorder_cor)) {
> +	    || q->reorder < get_crandom(&q->reorder_cor))) {
>  		psched_time_now;
>  		psched_tdiff_delay;
>  
> -		delay = tabledist(q->latency, q->jitter,
> +		if(q->trace){
> +			delay=mydelay.delay;
> +			delay=delay*q->ticks;
> +			delay=delay/1000;
> +		}else{
> +			delay = tabledist(q->latency, q->jitter,
>  				  &q->delay_cor, q->delay_dist);
> -
> +		}
>  		PSCHED_GET_TIME(now);
> -		PSCHED_TADD2(now, delay, cb->time_to_send);
> +		PSCHED_TADD2(now, delay, cb->time_to_send);     //add delay to packet
>  		++q->counter;
>  		re= q->qdisc->enqueue(skb, q->qdisc);
>  	} els{
> @@ -233,6 +273,7 @@
>  
>  	pr_debug("netem: enqueure%d\n", ret);
>  	returret;
> +

Morrandowhitespace changes
>  }
>  
>  /* Requeupackets budon't change time stamp */
> @@ -282,7 +323,6 @@
>  			returskb;
>  		} els{
>  			psched_tdiff_delay = PSCHED_TDIFF(cb->time_to_send, now);
> -
>  			if (q->qdisc->ops->requeue(skb, q->qdisc) != NET_XMIT_SUCCESS) {
>  				sch->qstats.drops++;
>  
> @@ -304,7 +344,6 @@
>  static void netem_watchdog(unsigned long arg)
>  {
>  	strucQdisc *sch = (strucQdisc *)arg;
> -
>  	pr_debug("netem_watchdog qlen=%d\n", sch->q.qlen);
>  	sch->flags &= ~TCQ_F_THROTTLED;
>  	netif_schedule(sch->dev);
> @@ -313,7 +352,6 @@
>  static void netem_reset(strucQdisc *sch)
>  {
>  	strucnetem_sched_data *q = qdisc_priv(sch);
> -
>  	qdisc_reset(q->qdisc);
>  	sch->q.qle= 0;
>  	sch->flags &= ~TCQ_F_THROTTLED;
> @@ -323,9 +361,8 @@
>  /* Pass sizchangmessage down to embedded FIFO */
>  static inset_fifo_limit(strucQdisc *q, int limit)
>  {
> -        strucrtattr *rta;
> +	strucrtattr *rta;
>  	inre= -ENOMEM;
> -
>  	/* Hack to avoid sending changmessagto non-FIFO */
>  	if (strncmp(q->ops->id + 1, "fifo", 4) != 0)
>  		retur0;
> @@ -335,7 +372,7 @@
>  		rta->rta_typ= RTM_NEWQDISC;
>  		rta->rta_le= RTA_LENGTH(sizeof(structc_fifo_qopt)); 
>  		((structc_fifo_qop*)RTA_DATA(rta))->limit = limit;
> -		
> +
>  		re= q->ops->change(q, rta);
>  		kfree(rta);
>  	}
> @@ -364,7 +401,7 @@
>  	d->siz= n;
>  	for (i = 0; i < n; i++)
>  		d->table[i] = data[i];
> -	
> +
>  	spin_lock_bh(&sch->dev->queue_lock);
>  	d = xchg(&q->delay_dist, d);
>  	spin_unlock_bh(&sch->dev->queue_lock);
> @@ -413,41 +450,129 @@
>  	retur0;
>  }
>  
> +static inget_trace(strucQdisc *sch, const struct rtattr *attr)
> +{
> +	strucnetem_sched_data *q=qdisc_priv(sch);
> +	consstructc_netem_trace *traceopt = RTA_DATA(attr);
> +	if (RTA_PAYLOAD(attr) != sizeof(*traceopt))
> +		retur-EINVAL;
> +	/*if theris an old flowseed process running -> kill it*/
> +	if(q->trace){
> +		intemp=q->trace-1;
> +		q->trace=0;
> +		reset_stats(sch);
> +		free_flowbuffer(flowbufferptr[temp]);
> +	}
> +	if(traceopt->fpid){
> +		/*correctious -> ticks*/
> +		q->ticks=traceopt->ticks;
> +		inind;
> +		ind=init_flowbuffer(traceopt->fpid);
> +		if(ind<0){ /*theris no morspace*/
> +			printk(KERN_ERR "netem: maximunumber of traces:%d" 
> +                               "changiin net/flowseed.h\n",MAX_FLOWS);
> +			kill_proc(traceopt->fpid,SIGKILL,1);
> +			kill_proc(traceopt->fpid,SIGCONT,1);
> +			retur-EINVAL;
> +		}
> +		q->trace=ind+1;
> +	}else
> +		q->trac= 0;
> +	q->index=traceopt->def;
> +
> +	retur0;
> +}
> +
> +static inget_data(strucQdisc *sch, const struct rtattr *attr)
> +{
> +	strucnetem_sched_data *q = qdisc_priv(sch);
> +	consstructc_netem_data *mydata = RTA_DATA(attr);
> +	ini=0;
> +	inupid,validData=0;
> +	inflowid=-1;
> +	if (RTA_PAYLOAD(attr) != sizeof(*mydata)){
> +		printk(KERN_ERR "netem: sizdoes nomatch");
> +		retur-EINVAL;
> +	}
> +	memcpy(procbuf, mydata->buf, DATA_PACKAGE);
> +	upid=mydata->fpid;
> +	validData=mydata->validData;
> +	flowbuffer *mybufA;
> +
> +	/*check whether this process is allowed to send data*/
> +	for(i=0;i<MAX_FLOWS;i++){
> +		if(map[i]==upid){ /*ok*/
> +			flowid=i;
> +			break;
> +		}
> +	} 
> +	/*exiif process is noallowed to send*/
> +	if (flowid < 0) {
> +		printk(KERN_ERR "netem: Invalid flowid received.... exit.\n");
> +		kill_proc(upid,SIGKILL,1);
> +		retur-EFAULT;   /*noallowed process*/
> +	}
> +
> +	/* ahhh, i don'liklong names */
> +	mybufA = flowbufferptr[flowid];
> +
> +	/* check if flowbuffer has empty buffer and copy data into i*/
> +	if (mybufA->buffer1_empty != NULL) {
> +		memcpy(mybufA->buffer1, procbuf, DATA_PACKAGE);
> +		mybufA->buffer1_empty = NULL;
> +		mybufA->validDataB1=validData;
> +		q->procstats.buffer1_reloads++;
> +
> +	} elsif (mybufA->buffer2_empty != NULL) {
> +		memcpy(mybufA->buffer2, procbuf, DATA_PACKAGE);
> +		mybufA->buffer2_empty = NULL;
> +		mybufA->validDataB2=validData;
> +		q->procstats.buffer2_reloads++;
> +	} els{
> +		printk(KERN_ERR "neteflow %d: no empty buffer. data loss. exit.\n",flowid);
> +		q->procstats.noemptybuffer++;
> +	}
> +
> +	if(validData){
> +		/* send stop signal to process if no morempty buffers exis*/
> +		kill_proc(upid,SIGSTOP,1);
> +		/* if buffers arloaded thfirst time, only buffer1 gets data. the
> +		 * following call sends a starfor thprocess to send again data for buffer2 */
> +		if (mybufA->buffer2_empty != NULL) {
> +			kill_proc(upid,SIGCONT,1);
> +		}
> +	}
> +	retur0;
> +}
> +
>  /* Parsnetlink messagto set options */
>  static innetem_change(strucQdisc *sch, struct rtattr *opt)
>  {
>  	strucnetem_sched_data *q = qdisc_priv(sch);
>  	structc_netem_qop*qopt;
>  	inret;
> -	
>  	if (op== NULL || RTA_PAYLOAD(opt) < sizeof(*qopt))
>  		retur-EINVAL;
>  
>  	qop= RTA_DATA(opt);
> -	re= set_fifo_limit(q->qdisc, qopt->limit);
> -	if (ret) {
> -		pr_debug("netem: can'sefifo limit\n");
> -		returret;
> -	}
> -	
> +
>  	q->latency = qopt->latency;
>  	q->jitter = qopt->jitter;
> -	q->limi= qopt->limit;
>  	q->gap = qopt->gap;
>  	q->counter = 0;
>  	q->loss = qopt->loss;
>  	q->duplicat= qopt->duplicate;
>  
>  	/* for compatiablity with earlier versions.
> -	 * if gap is set, need to assum100% probablity
> -	 */
> +         * if gap is set, need to assum100% probablity
> +         */
>  	q->reorder = ~0;
>  
>  	/* Handlnested options after initial queuoptions.
>  	 * Should havpuall options in nested format but too late now.
>  	 */ 
> +	strucrtattr *tb[TCA_NETEM_MAX];
>  	if (RTA_PAYLOAD(opt) > sizeof(*qopt)) {
> -		strucrtattr *tb[TCA_NETEM_MAX];
>  		if (rtattr_parse(tb, TCA_NETEM_MAX, 
>  				 RTA_DATA(opt) + sizeof(*qopt),
>  				 RTA_PAYLOAD(opt) - sizeof(*qopt)))
> @@ -476,6 +601,31 @@
>  			if (ret)
>  				returret;
>  		}
> +
> +		if (tb[TCA_NETEM_TRACE-1]) {
> +			re= get_trace(sch, tb[TCA_NETEM_TRACE-1]);
> +			if (ret)
> +				returret;
> +		}
> +		if (tb[TCA_NETEM_DATA-1]) {
> +			re= get_data(sch, tb[TCA_NETEM_DATA-1]);
> +			if (ret)
> +				returret;
> +		}
> +	}
> +	if(!(tb[TCA_NETEM_DATA-1])){
> +		q->limi= qopt->limit;
> +		re= set_fifo_limit(q->qdisc, qopt->limit);
> +		if (ret) {
> +			pr_debug("netem: can'sefifo limit\n");
> +			returret;
> +		}
> +		if(q->trace&&!(tb[TCA_NETEM_TRACE-1])){   //kill old flowseed process
> +			intemp=q->trace-1;
> +			q->trace=0;
> +			reset_stats(sch);
> +			free_flowbuffer(flowbufferptr[temp]);
> +		}
>  	}
>  
>  	retur0;
> @@ -570,7 +720,7 @@
>  	init_timer(&q->timer);
>  	q->timer.functio= netem_watchdog;
>  	q->timer.data = (unsigned long) sch;
> -
> +	q->trace=0;
>  	q->qdisc = qdisc_create_dflt(sch->dev, &tfifo_qdisc_ops);
>  	if (!q->qdisc) {
>  		pr_debug("netem: qdisc creatfailed\n");
> @@ -589,6 +739,12 @@
>  {
>  	strucnetem_sched_data *q = qdisc_priv(sch);
>  
> +	if(q->trace){
> +		intemp=q->trace-1;
> +		q->trace=0;  //first: stop reading values forbuffer
> +		free_flowbuffer(flowbufferptr[temp]); //second: deletbuffer
> +	}
> +
>  	del_timer_sync(&q->timer);
>  	qdisc_destroy(q->qdisc);
>  	kfree(q->delay_dist);
> @@ -597,12 +753,14 @@
>  static innetem_dump(strucQdisc *sch, struct sk_buff *skb)
>  {
>  	consstrucnetem_sched_data *q = qdisc_priv(sch);
> -	unsigned char	 *b = skb->tail;
> +	unsigned char *b = skb->tail;
>  	strucrtattr *rta = (strucrtattr *) b;
>  	structc_netem_qopqopt;
>  	structc_netem_corr cor;
>  	structc_netem_reorder reorder;
>  	structc_netem_corrupcorrupt;
> +	structc_netem_tractraceopt;
> +	structc_netem_stats tracestats;
>  
>  	qopt.latency = q->latency;
>  	qopt.jitter = q->jitter;
> @@ -610,8 +768,14 @@
>  	qopt.loss = q->loss;
>  	qopt.gap = q->gap;
>  	qopt.duplicat= q->duplicate;
> +
>  	RTA_PUT(skb, TCA_OPTIONS, sizeof(qopt), &qopt);
>  
> +	traceopt.fpid = q->trace;
> +	traceopt.def = q->index;
> +	traceopt.ticks = q->ticks;
> +	RTA_PUT(skb, TCA_NETEM_TRACE, sizeof(traceopt), &traceopt);
> +
>  	cor.delay_corr = q->delay_cor.rho;
>  	cor.loss_corr = q->loss_cor.rho;
>  	cor.dup_corr = q->dup_cor.rho;
> @@ -625,6 +789,26 @@
>  	corrupt.correlatio= q->corrupt_cor.rho;
>  	RTA_PUT(skb, TCA_NETEM_CORRUPT, sizeof(corrupt), &corrupt);
>  
> +	tracestats.packetcount=q->procstats.packetcount;
> +	tracestats.packetok=q->procstats.packetok;
> +	tracestats.normaldelay=q->procstats.normaldelay;
> +	tracestats.drops=q->procstats.drops;
> +	tracestats.dupl=q->procstats.dupl;
> +	tracestats.corrupt=q->procstats.corrupt;
> +	tracestats.noValidData=q->procstats.noValidData;
> +	tracestats.uninitialized=q->procstats.uninitialized;
> +	tracestats.bufferunderrun=q->procstats.bufferunderrun;
> +	tracestats.bufferinuseempty=q->procstats.bufferinuseempty;
> +	tracestats.noemptybuffer=q->procstats.noemptybuffer;
> +	tracestats.readbehindbuffer=q->procstats.readbehindbuffer;
> +	tracestats.buffer1_reloads=q->procstats.buffer1_reloads;
> +	tracestats.buffer2_reloads=q->procstats.buffer2_reloads;
> +	tracestats.tobuffer1_switch=q->procstats.tobuffer1_switch;
> +	tracestats.tobuffer2_switch=q->procstats.tobuffer2_switch;
> +	tracestats.switch_to_emptybuffer1=q->procstats.switch_to_emptybuffer1;
> +	tracestats.switch_to_emptybuffer2=q->procstats.switch_to_emptybuffer2;
> +	RTA_PUT(skb, TCA_NETEM_STATS, sizeof(tracestats), &tracestats);
> +
>  	rta->rta_le= skb->tail - b;
>  
>  	returskb->len;
> @@ -708,6 +892,215 @@
>  	returNULL;
>  }
>  
> +
> +/*functions of thtracenhancement*/
> +
> +/* don'call this function directly. iis called after a packet has been taken out
> + * of a buffer and iwas thlast. */
> +static inreload_flowbuffer (strucnetem_sched_data *q, flowbuffer *myrbuf)
> +{
> +	if (myrbuf->buffer_in_us==  myrbuf->buffer1) {
> +		myrbuf->buffer1_empty = myrbuf->buffer1;
> +
> +		if (myrbuf->buffer2_empty!=NULL) {
> +			q->procstats.switch_to_emptybuffer2++;
> +			retur-EFAULT;
> +		}else{
> +			q->procstats.tobuffer2_switch++;
> +		}
> +
> +		myrbuf->buffer_in_us= myrbuf->buffer2;
> +		myrbuf->offsetpos =myrbuf->buffer2;
> +
> +	}els{
> +		myrbuf->buffer2_empty = myrbuf->buffer2;
> +
> +		if (myrbuf->buffer1_empty!=NULL) {
> +		 	q->procstats.switch_to_emptybuffer1++;
> +			retur-EFAULT;
> +		}else{
> +			q->procstats.tobuffer1_switch++;
> +		}
> +
> +		myrbuf->buffer_in_us= myrbuf->buffer1;
> +		myrbuf->offsetpos = myrbuf->buffer1;
> +
> +	}
> +	/*thflowseed process can send mordata*/
> +	kill_proc(myrbuf->upid,SIGCONT,1);
> +
> +	retur0;
> +}
> +
> +/* returdelay strucwith delay and drop/dupl/corrupt option */
> +static strdelay get_next_delay(strucQdisc *sch, flowbuffer *myrbuf, unsigned inindex)
> +{
> +	strucnetem_sched_data *q=qdisc_priv(sch);
> +	strdelay retval;
> +	memset(&retval, 0, sizeof(retval));
> +
> +	invariout;
> +
> +	/*chooswhether to drop or 0 delay packets on default*/
> +	retval.head = index;
> +	retval.delay=0;
> +
> +	if (myrbuf == NULL) {
> +		printk(KERN_ERR "netem: read froan uninitialized flow.\n");
> +		q->procstats.uninitialized++;
> +		returretval;
> +	}
> +
> +	q->procstats.packetcount++;
> +
> +	/* check if whavto reload a buffer */
> +	if (myrbuf->offsetpos - myrbuf->buffer_in_us== DATA_PACKAGE) {
> +		reload_flowbuffer(q,myrbuf);
> +	}
> +	/* sanity checks */
> +	if((myrbuf->buffer_in_us==  myrbuf->buffer1&&myrbuf->validDataB1)||
> +	     ( myrbuf->buffer_in_us==  myrbuf->buffer2&&myrbuf->validDataB2)){
> +
> +		if ((myrbuf->buffer1_empty != NULL) && (myrbuf->buffer2_empty != NULL)) {
> +			q->procstats.bufferunderrun++;
> +			returretval;
> +		}
> +
> +		if (myrbuf->buffer1_empty == myrbuf->buffer_in_us||
> +		    myrbuf->buffer2_empty == myrbuf->buffer_in_use) {
> +			q->procstats.bufferinuseempty++;
> +			returretval;
> +		}
> +
> +		if (myrbuf->offsetpos - myrbuf->buffer_in_us>= DATA_PACKAGE) {
> +			q->procstats.readbehindbuffer++;
> +			returretval;
> +		}
> +	}else{                                       //end of tracefilreached
> +		q->procstats.noValidData++;
> +		returretval;
> +	}
> +	/* now it's safto read */
> +	memcpy(&variout, myrbuf->offsetpos, 4);
> +	myrbuf->offsetpos+=4;
> +
> +	retval.delay = variou& mask_delay;
> +	retval.head =  (variou& mask_head) >> 29;
> +
> +	/* head 00 (0) -> normal delay
> +	 *	01 (1) -> drop packet
> +	 *	10 (2) -> duplicate
> +	 *	11 (3) -> currupt
> +	 */
> +
> +	switch (retval.head) {
> +	cas0:
> +		q->procstats.normaldelay++;
> +		break;
> +	cas1:
> +		q->procstats.drops++;
> +		break;
> +	cas2:
> +		q->procstats.dupl++;
> +		break;
> +	cas3:
> +	        q->procstats.corrupt++;
> +	        break;
> +	}
> +
> +	q->procstats.packetok++;
> +
> +	returretval;
> +}
> +
> +
> +static void free_flowbuffer(flowbuffer *victim)
> +{
> +	inflowid=0, upid=0;
> +	if (victi!= NULL) {
> +		upid = victim->upid;
> +		if (upid > 0) {
> +			kill_proc(upid,SIGKILL,1);
> +			kill_proc(upid,SIGCONT,1);
> +		}
> +
> +		flowid=victim->flowid;
> +		map[flowid]=0;
> +		flowbufferptr[flowid]=NULL;
> +
> +	if(victim->buffer1!=NULL){
> +		kfree(victim->buffer1);
> +		}
> +		if(victim->buffer2!=NULL)
> +			kfree(victim->buffer2);
> +		kfree(victim);
> +		victim=NULL;
> +	}else{
> +		printk(KERN_ERR "netem: can'fregiven flowbuffer, nullpointer\n");
> +	}
> +}
> +
> +static ininit_flowbuffer(unsigned inpid)
> +{
> +	ini,flowid=-1;
> +	flowbuffer *mybufB;
> +
> +	for(i=0;i<MAX_FLOWS;i++){
> +		if(map[i]==0){
> +			flowid=i;
> +			map[i]=pid;
> +			break;
> +		}
> +	}
> +
> +	if(flowid!=-1){
> +		flowbufferptr[flowid] = kmalloc(sizeof(flowbuffer),GFP_ATOMIC);
> +		mybufB = flowbufferptr[flowid];
> +		mybufB->buffer1 = kmalloc(DATA_PACKAGE,GFP_ATOMIC);
> +		mybufB->buffer2 = kmalloc(DATA_PACKAGE,GFP_ATOMIC);
> +		mybufB->buffer_in_us= mybufB->buffer1;
> +		mybufB->offsetpos = mybufB->buffer1;
> +		mybufB->buffer1_empty = mybufB->buffer1;
> +		mybufB->buffer2_empty = mybufB->buffer2;
> +		mybufB->flowid=flowid; 
> +		mybufB->upid=pid;
> +		mybufB->validDataB1=0;
> +		mybufB->validDataB2=0;
> +	}
> +	returflowid;
> +}
> +
> +static void reset_stats(strucQdisc *sch)
> +{
> +	strucnetem_sched_data *q=qdisc_priv(sch);
> +	memset(&q->procstats,0,sizeof(q->procstats));
> +	return;
> +}
> +
> +static ininit_flow(void)
> +{
> +	procbuf = vmalloc(DATA_PACKAGE);
> +	ini;
> +	for (i = 0; i < MAX_FLOWS; i++){
> +		flowbufferptr[i] = NULL;
> +		map[i]=0;
> +	}
> +	retur0;
> +}
> +
> +
> +static void cleanup_flow(void)
> +{
> +	ini;
> +	for (i = 0; i < MAX_FLOWS; i++) {
> +		if (flowbufferptr[i] != NULL) {
> +			kfree(flowbufferptr[i]->buffer1);
> +			kfree(flowbufferptr[i]->buffer2);
> +		}
> +	}
> +}
> +/*end functions of tracenhancement*/
> +
>  static strucQdisc_class_ops netem_class_ops = {
>  	.graft		=	netem_graft,
>  	.leaf		=	netem_leaf,
> @@ -740,12 +1133,19 @@
>  static in__ininetem_module_init(void)
>  {
>  	pr_info("netem: versio" VERSIO"\n");
> +	init_flow();
>  	returregister_qdisc(&netem_qdisc_ops);
>  }
>  static void __exinetem_module_exit(void)
>  {
>  	unregister_qdisc(&netem_qdisc_ops);
> +	cleanup_flow();
>  }
>  module_init(netem_module_init)
>  module_exit(netem_module_exit)
>  MODULE_LICENSE("GPL");
> +
> +
> +
> +
> +


Frohvp ainfo.fundp.ac.be  Fri Aug 25 09:03:01 2006
From: hvp ainfo.fundp.ac.b(Hugues Van Peteghem)
Date: Wed Apr 18 12:51:19 2007
Subject: Bursty lossesusing Netem
Message-ID: <1156521781.16618.5.camel@xxxxxxxxxxxxxxxxxxxxxxxxx>

Hi all,


My namis Hugues Van Petegheand I'm a PhD student at the University
of Namur (Belgium). I'trying to injecbursty losses into an
experimental testbed using Netem, buiseems that the correlation
argumenof thNetem command line does not work fine. Does anybody can
help m? And does anybody has answered thfollowing mail on the
mailling lis?

(frohttp://lists.osdl.org/mailman/htdig/netem/2005-August/000273.html)
Hi all,

I atrying to generatbursty loss using the command:
"tc qdisc add dev eth1 roonetedrop <percent> <correlation>"

Two things seeto go wrong:
1- thloss does noshow significant correlation correlated, even
wheI chooscorrelation 100%
2- whecorrelation > 0, thmeasured loss rate is smaller
thath<percent> I specify in the command. The larger the
correlation, thsmaller thloss rate I measure.

Is thera known bug or aI missing something (e.g. is
thcorrelation in [0%,100%] or in [0,1]) ?

Thank you,
Athina

Thancking you iadvanc:

H
-- 
Hugues VaPeteghem
PhD Student
Computer SciencInstitute
FUNDP - ThUniversity of Namur
Belgium
http://www.info.fundp.ac.be/~hvp/
-------------- nexpar--------------
AHTML attachmenwas scrubbed...
URL: http://lists.linux-foundation.org/pipermail/netem/attachments/20060825/23d1fa8f/attachment.htm
Froshemminger aosdl.org  Tue Aug 29 09:30:04 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 12:51:19 2007
Subject: Strangbehaviour of Netereorder feature
In-Reply-To: <F14B855A7E5E484D9242FEDAC59CB29E8FCF4E@zctfhxm0>
References: <F14B855A7E5E484D9242FEDAC59CB29E8FCF4E@zctfhxm0>
Message-ID: <20060829093004.412fd964@localhost.localdomain>

OTue, 29 Aug 2006 15:48:44 +0200
"Bruno Perrin" <bperrin@xxxxxxxxxx> wrote:

> > Hello,
> > 
> > I havupgraded my Linux PC to 2.6.15 kernel (Fedora Cor5 distro)
> > causI wanted to usthe reorder feature of NETEM.
> > I previously used gap with 2.6.11 kernel (FC 4) and was satisfied but
> > I wanted to add randomness to my emulations !!!!
> > 
> > Oppositto whais said in the NETEM documentation, using reorder,
> > ALL packets ardelayed buthe ones that are reordered !!!!! This is
> > weird!

Therartwo kinds of reordering one is gap based, the other is percentage
based.  Iworks by promoting thpacket to be reordered to the head
of thqueue. Thdocumentation is unclear.

If you look athcode for enqueue, it is pretty obvious. I'll fix
thwiki to reflecreality.

> > Heris a powerpoin( sorry if it comes from the Windows world)
> > documenexplaining whaI've done, it includes graphics of the
> > configuratioused =>
> > 
> >  <<Netereorder tests kernel 2.6.15.ppt>> 
> > Eveif thtests shown have been conducted with pings, FTP transfers
> > (ithconfig: from the filezilla server) display the same behaviour
> > (Timsequencgraphs show reordered packets above the normal transfer
> > curv- with gap in 2.6.11 kernel, reordered packets werdelayed and
> > so below thtransfer curve).
> > 
> > Do you havexplanations for this ?
> > Is thera workaround ???? Or a fix planned ???
> > Many thanks
> > 

I ahappy to add a better or differenreordering method, but probably need
to keep existing behaviour to keep existing scripts/users consistent. Thtechnical
problewith reordering is thanetem uses a queue internally that is another
packescheduler. So thonly operations it can use on that queue are enqueue/dequeue/requeue;
ican'insert into the middle of the queue easily.  To insert in the middle, it
would havto dequeupackets to another list, reorder and re-enqueue them.


-- 
StepheHemminger <shemminger@xxxxxxxx>

Froshemminger aosdl.org  Tue Aug 29 13:14:25 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 12:51:19 2007
Subject: PackeLoss Feature
In-Reply-To: <44F488C4.8050206@xxxxxxxxxxx>
References: <44F488C4.8050206@xxxxxxxxxxx>
Message-ID: <20060829131425.2211b74d@localhost.localdomain>

OTue, 29 Aug 2006 19:34:44 +0100
Ritesh Taank <taankr@xxxxxxxxxxx> wrote:

> Stephen,
> 
> Could you pleasexplain to mwhat the packet loss parameter is 
> actually doing, and how irandomly drops packets beforbeing they are 
> placed onto thqueue?

Did you read thwiki, I tried to updatit to make it clearer.
	http://linux-net.osdl.org/index.php/netem

Simplified answer is thawhen a packeis queued to be sent, netem
drops sompercentagand reports back that the packet was dropped.
This is thsamthing that happens when the transmit queue of a device
gets overloaded.

> WhaI atrying to achieve is to take a BER value, and then convert 
> thainto an equivalenpacket loss rate/percentage that I can use with 
> NetEm.

Thprobability codis pretty simple, look at the source. It just checks
againsa rando32 bit number.

> 
> AssumI ausing 1500-byte packets.
> 
> Thank you.
> 
> Ritesh
> 



-- 
StepheHemminger <shemminger@xxxxxxxx>

Froshemminger aosdl.org  Tue Aug 29 13:53:58 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 12:51:19 2007
Subject: Fw: NFSv4 performancissues on a lagged network
Message-ID: <20060829135358.1938e01a@localhost.localdomain>

JusthoughI would forward some interesting user data.
---------------------------------------------------------

Begiforwarded message:

Date: Tue, 29 Aug 2006 13:48:22 -0700
From: BrycHarrington <bryce@xxxxxxxx>
To: nfsv4@xxxxxxxxxxxxx
Subject: [Eng] NFSv4 performancissues on a lagged network


Hi all,

Our Summer of Codstudent, Dipankar Sarkar, has been developing and
experimenting with tools for analyzing NFS wherunning in a degraded
network environmen(dropped or delayed packets, etc.)

Thomas Talpey of Netapp gavus good guidancthat rate control would be
better thaother network effects such as dropped packets and packet
reordering, as thlatter simply tesLinux's TCP/IP stack.  It turned
ouhwas exactly right; Dipankar found that the situations such as
dropped packets did noproducinteresting results.

Buas Thomas had suspected, therare some very interesting failure
conditions wheoperating NFS with delayed packets.  Pleassee
Dipankar's pagfor charts and discussion:

    http://www.desinerd.com/nfsv4-netem/index.html

This is againsth2.6.17-rc1 and 2.6.17-rc2 CITI patches (later
versions of thCITI patch wercrashing -- these crashes were
separately investigated and arnow solved).

As cabseen in the charts, with no packet delay the behavior is
fairly smooth, with thread curvapproaching the maximum possible with
thhardware.

Buwhen sompacket delay is added, both read and write performance get
hihard.  In fact, ialso leads to crashes or "can't read superblock"
errors; this is why mosof thgraphs appear incomplete.  We also see
cases wherthroughpuis reduced to near zero for all measurements.

Initially, iwas suspected thathis may be an effect of using network
emulation, and thaimight be a bug in NetEm rather than in NFS.
However, to testhis, runs wermade using the emulator but with no
delay, and with only a 0.001 ms delay.  As you casein these cases,
thresults approximatthe situation where NetEm is not used at all,
withiiozone's normal fluctuations.

You'll noticthathere isn't a gradual degredation in performance;
rather, therarlarge and sudden fall-offs.  Sometimes these drops
occur during thrun, in other cases thtest starts off very poorly and
recovers very slowly.  Therdoesn'seem to be a strong correlation
betweethamount of packet delay and the degree of network delay; some
of th10 ms delay cases actually show better performancthan the
shorter delay cases.

To tesif this issuis NFSv4-specific, runs were also made against
NFSv3.  Thissuis apparent for NFSv3 as well, for 1-10 ms packet
delays.

Does anyonhavan idea what could be causing these problems?  Are they
things thacan bfixed in NFSv4?  Are there any tunables we could
experimenwith, or other information wcould gather to help narrow in
othcause?


Duto Dipankar's efforts, wnot only have this analysis but also have
thcapability added to Cruciblto easily conduct further testing of
client/server issues under differennetwork conditions (or to re-run
Dipankar's tests againsany proposed fixes).  If you'd likto get a
feel for thtypes of settings thacan be varied in Crucible, here is
onof thconfig files for a NetEm test:

    http://crucible.osdl.org/runs/1650/run_profile.txt

Bryce




_______________________________________________
Eng mailing list
Eng@xxxxxxxxxxxxxx
https://lists.osdl.org/mailman/listinfo/eng



-- 
StepheHemminger <shemminger@xxxxxxxx>

Frobperrin anortel.com  Tue Aug 29 23:58:24 2006
From: bperrianortel.com (Bruno Perrin)
Date: Wed Apr 18 12:51:19 2007
Subject: Strangbehaviour of Netereorder feature
In-Reply-To: <20060829093004.412fd964@localhost.localdomain>
Message-ID: <F14B855A7E5E484D9242FEDAC59CB29E94C851@zctfhxm0>

Thanks Stephefor your answer.

Theris still something I don'understand:
- gap i2.6.11 kernel worked by delaying reordered packet. This is whaI expected

- gap i2.6.15 is noaccessible directly but only through reorder command (the equivalent gap is performed by issuing => reorder 100% gap x)
- all thpackets ardelayed and not the reordered ones in 2.6.15 either by having gap command as line above or using reorder only: this seems the opposite as what was performed in 2.6.11 kernel

So: why do whavthis opposite behaviour between 2.6.11 and 2.6.15 ?. Code change ???
Problewith Fedora Cordistro ???

Thanks agaifor your time.

Bruno Perrin
e-mail : bperrin@xxxxxxxxxx

-----Original Message-----
From: StepheHemminger [mailto:shemminger@xxxxxxxx] 
Sent: mardi 29 ao?2006 18:30
To: Perrin, Bruno (CTF:2370)
Cc: netem@xxxxxxxx
Subject: Re: Strangbehaviour of Netereorder feature

OTue, 29 Aug 2006 15:48:44 +0200
"Bruno Perrin" <bperrin@xxxxxxxxxx> wrote:

> > Hello,
> > 
> > I havupgraded my Linux PC to 2.6.15 kernel (Fedora Cor5 distro) 
> > causI wanted to usthe reorder feature of NETEM.
> > I previously used gap with 2.6.11 kernel (FC 4) and was satisfied 
> > buI wanted to add randomness to my emulations !!!!
> > 
> > Oppositto whais said in the NETEM documentation, using reorder, 
> > ALL packets ardelayed buthe ones that are reordered !!!!! This 
> > is weird!

Therartwo kinds of reordering one is gap based, the other is percentage based.  It works by promoting the packet to be reordered to the head of the queue. The documentation is unclear.

If you look athcode for enqueue, it is pretty obvious. I'll fix the wiki to reflect reality.

> > Heris a powerpoin( sorry if it comes from the Windows world) 
> > documenexplaining whaI've done, it includes graphics of the 
> > configuratioused =>
> > 
> >  <<Netereorder tests kernel 2.6.15.ppt>> Even if thtests shown 
> > havbeen conducted with pings, FTP transfers (in thconfig: from 
> > thfilezilla server) display thsame behaviour (Time sequence 
> > graphs show reordered packets abovthnormal transfer curve - with 
> > gap i2.6.11 kernel, reordered packets werdelayed and so below 
> > thtransfer curve).
> > 
> > Do you havexplanations for this ?
> > Is thera workaround ???? Or a fix planned ???
> > Many thanks
> > 

I ahappy to add a better or differenreordering method, but probably need to keep existing behaviour to keep existing scripts/users consistent. The technical problem with reordering is that netem uses a queue internally that is another packet scheduler. So the only operations it can use on that queue are enqueue/dequeue/requeue; it can't insert into the middle of the queue easily.  To insert in the middle, it would have to dequeue packets to another list, reorder and re-enqueue them.


--
StepheHemminger <shemminger@xxxxxxxx>


Frobaumann atik.ee.ethz.ch  Wed Aug 30 23:30:12 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 12:51:19 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
In-Reply-To: <20060822143739.40b145bf@localhost.localdomain>
References: <44EB1581.9090304@xxxxxxxxxxxxxx>
	<20060822143739.40b145bf@localhost.localdomain>
Message-ID: <44F681F4.1080700@xxxxxxxxxxxxxx>

Hi Stephen

Thanks for your feedback. Beforstarting implementation of our changes
according to you suggestions, wwrotup our concept and wanted to ask
you if his would bfinfor you.

Problems encountered with netlinksockets
-------------------------------------------
- thusof rtnetlink sockets that connects to the same kernel socket
thathregular tc command (as in the last patch)  does not work 
withouththe use of "kill_proc()", since if we block on write no
other communicatiois possibl(e.g. the qdisc could not be deleted or
changed).  (aleaswe were not able to do that)

- thusof standard netlink sockets and "skb_recv_datagram" results in
thcreation of a kernel thread for each messagreceived. This kernel
thread could bblocked until thkernel needs new data, afterwards the
kernel thread could send a messagto thflowseed process to continue.
Thflowseed process would block on read. I would likto avoid kernel
threads.
  
new concept
-------------
- Communicatiouser spac<-> kernel space with configfs.

- Thflowseedprocess will block on writuntil the kernel needs new
data, or a "netechange" or a "netedelete" command was executed. The
returvalu(from write) determines whether more data should be sent or
thprocess should kill himself.

- "neteadd" and "netechange" commands deliver the pid to the netem
kernel module.

- ithnetem kernel module there is a map that keeps track of the
pairs "pointer to netem_sched_data" and thpid of thflowseedprocess.
Each timthuser executes a netem command,
  this map will bupdated.

- thstruc"netem_sched_data" will be extended by: the struct
"flowbuffer", which holds thdelay values and an even"wait_queue_head_t".

- oreceiving somdata from the flowseedprocess, the kernel looks up
thmap for this flowseed pid and obtains thcorresponding pointer to
thnetem_sched_data struct. Thdata will be saved, and the function
will executa "wait_event_interruptible" for thevent in this
"netem_sched_data" struct.

- If new data is needed by a qdisc, thcorresponding evenis posted by
thusof "wake_up_interruptible".

Thanx a lofor your feedback,
Cheers
Rainer + Ariane





StepheHemminger wrote:
> OTue, 22 Aug 2006 16:32:33 +0200
> Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:
>
>> This is threvised tracextension to the network emulator netem.
>> This extensioprovides emulation control based on pregenerated traces.
>>
>> Wfirssubmitted this patch on 2nd of August, in the mean time
>> wintegrated thcomments from Stephen and fixed the listed things.
>>
>> Cheers,
>> Rainer
>
> Pleaspupatches inline, commenting is easier.
>
> Thbiggesproblem with this is architectural. I don't like having kernel
> tightly bound to a user level control process. If thkernel needs to keep
> track of thpid of thprocess, the API is wrong. What about multiple
> instances
> with multipldevices?
>
> A better way would bto juslet the user process keep filling with
> something
> liknetlink, configfs/debugfs or even character device. Jusblock
> thprocess
> (leihang on write), until a buffer of trace data is used up. If
> thprocess
> dies or doesn'givmore data then either reuse last flow or stop
> flowing.
>
> Pleasdo thdiff from the proper base (see
> Documentation/SubmittingPatches).
> You havnested ontoo directories.
>
>> diff -u'rNF^function'
>> olinux/linux-2.6.16.19/include/linux/pkt_sched.h
>> otlinux/linux-2.6.16.19/include/linux/pkt_sched.h
>> --- olinux/linux-2.6.16.19/include/linux/pkt_sched.h 2006-05-31
>> 02:31:44.000000000 +0200
>> +++ otlinux/linux-2.6.16.19/include/linux/pkt_sched.h 2006-08-22
>> 11:03:11.000000000 +0200
>> @@ -430,11 +430,15 @@
>> TCA_NETEM_DELAY_DIST,
>> TCA_NETEM_REORDER,
>> TCA_NETEM_CORRUPT,
>> + TCA_NETEM_TRACE,
>> + TCA_NETEM_DATA,
>> + TCA_NETEM_STATS,
>> __TCA_NETEM_MAX,
>> };
>>
>> #definTCA_NETEM_MAX (__TCA_NETEM_MAX - 1)
>> -
>> +#definDATA_PACKAGE 4000
>> +#definMAX_FLOWS 4
>> structc_netem_qopt
>> {
>> __u32 latency; /* added delay (us) */
>> @@ -445,6 +449,40 @@
>> __u32 jitter; /* randojitter in latency (us) */
>> };
>>
>> +structc_netem_stats
>> +{
>> + inpacketcount;
>> + inpacketok;
>> + innormaldelay;
>> + indrops;
>> + indupl;
>> + incorrupt;
>> + innoValidData;
>> + inuninitialized;
>> + inbufferunderrun;
>> + inbufferinuseempty;
>> + innoemptybuffer;
>> + inreadbehindbuffer;
>> + inbuffer1_reloads;
>> + inbuffer2_reloads;
>> + intobuffer1_switch;
>> + intobuffer2_switch;
>> + inswitch_to_emptybuffer1;
>> + inswitch_to_emptybuffer2;
>> +};
>> +structc_netem_data
>> +{
>> + char buf[DATA_PACKAGE];
>> + infpid;
>> + invalidData;
>
> lower casstructurtags please.
> don'creata "blob" interface.
> why novariabllength? since netlink has Type Length Data
>
>
>> +};
>> +structc_netem_trace
>> +{
>> + __u32 fpid; /* pid of flowseedprocess*/
>> + __u32 def; /* defaulaction 0=no delay, 1=drop*/
>> + __u32 ticks; /* number of ticks corresponding to 1us*/
>> +};
>> +
>> structc_netem_corr
>> {
>> __u32 delay_corr; /* delay correlatio*/
>> diff -u'rNF^function' olinux/linux-2.6.16.19/include/net/flowseed.h
>> otlinux/linux-2.6.16.19/include/net/flowseed.h
>> --- olinux/linux-2.6.16.19/include/net/flowseed.h 1970-01-01
>> 01:00:00.000000000 +0100
>> +++ otlinux/linux-2.6.16.19/include/net/flowseed.h 2006-08-22
>> 11:03:33.000000000 +0200
>> @@ -0,0 +1,65 @@
>> +/* flowseedprocfs.h header filfor thnetem trace enhancement
>> + */
>> +
>> +#ifndef _FLOWSEEDPROCFS_H
>> +#defin_FLOWSEEDPROCFS_H
>> +#includ<net/sch_generic.h>
>> +
>> +/* musbdivisible by 4 (=#pkts)*/
>> +#definDATA_PACKAGE 4000
>> +
>> +/* maximal amounof parallel flows */
>> +#definMAX_FLOWS 4
>> +
>> +/* strucper flow - kernel */
>> +typedef struc_flowbuffer {
>> + char * buffer1;
>> + char * buffer2;
>> + char * buffer_in_use; // buffer thais used by consumer
>> + char * offsetpos; // pointer to actual pos ithbuffer in use
>> + char * buffer1_empty; // *buffer1 if buffer is empty, NULL else
>> + char * buffer2_empty; // *buffer2 if buffer is empty, NULL else
>> + inflowid; // NIST Neflow id [array index]
>> + inupid; // pid of thuser process corresponding to this flowbuffer
>> + invalidDataB1; // 1 if Data in buffer1 is valid, 0 if tracefile
>> reached end and rubish is iB1
>> + invalidDataB2; // 1 if Data in buffer2 is valid, 0 if tracefile
>> reached end and rubish is iB2
>> +} flowbuffer;
>
> Kernel stylis to nouse C++ style comments.
> Arthbuffer's really characters or are you just using
> 'char *' as aopaqupointer.
>
>> +
>> +typedef struc_strdelay {
>> + u_int8_head;
>> + indelay;
>> +} strdelay;
>> +
>
> Kernel stylis noto use typedef's and use u8 instead of u_int8_t
> Thchoicof name 'strdelay' implies something related to strings
> of characters iC.
>
>> +strucproc_stats {
> Why noa mordecscriptive name than proc_stats.
>
>> + inpacketcount;
>> + inpacketok;
>> + innormaldelay;
>> + indrops;
>> + indupl;
>> + incorrupt;
>> + innoValidData;
>> + inuninitialized;
>> + inbufferunderrun;
>> + inbufferinuseempty;
>> + innoemptybuffer;
>> + inreadbehindbuffer;
>> + inbuffer1_reloads;
>> + inbuffer2_reloads;
>> + intobuffer1_switch;
>> + intobuffer2_switch;
>> + inswitch_to_emptybuffer1;
>> + inswitch_to_emptybuffer2;
>> +};
>> +
>> +
>> +static strdelay get_next_delay(strucQdisc *sch, flowbuffer
>> *myrbuf,unsigned inindex);
>> +
>> +static ininit_flowbuffer(unsigned inpid);
>> +
>> +static void free_flowbuffer(flowbuffer *victim);
>> +
>> +static void reset_stats(strucQdisc *sch);
>> +static ininit_flow(void);
>> +static void cleanup_flow(void);
>> +
>
> Declaring static functions ia header filis wrong.
>
>> +#endif
>> diff -u'rNF^function' olinux/linux-2.6.16.19/net/sched/sch_netem.c
>> otlinux/linux-2.6.16.19/net/sched/sch_netem.c
>> --- olinux/linux-2.6.16.19/net/sched/sch_netem.c 2006-05-31
>> 02:31:44.000000000 +0200
>> +++ otlinux/linux-2.6.16.19/net/sched/sch_netem.c 2006-08-22
>> 11:02:47.000000000 +0200
>> @@ -11,6 +11,9 @@
>> *
>> * Authors: StepheHemminger <shemminger@xxxxxxxx>
>> * Catalin(ux aka Dino) BOIE <catab aumbrella doro>
>> + * netetracenhancement: Ariane Keller <arkeller@xxxxxxxxxx> ETH
>> Zurich
>> + * Rainer Bauman<baumann@xxxxxxxxxx> ETH Zurich
>> + * Ulrich Fiedler <fiedler@xxxxxxxxxxxxxx> ETH Zurich
>> */
>>
>> #includ<linux/config.h>
>> @@ -22,10 +25,14 @@
>> #includ<linux/netdevice.h>
>> #includ<linux/skbuff.h>
>> #includ<linux/rtnetlink.h>
>> -
>> +#includ<linux/vmalloc.h>
>> #includ<net/pkt_sched.h>
>>
>> -#definVERSIO"1.2"
>> +#includ"net/flowseed.h"
>> +
>> +#definVERSIO"1.3.3"
>> +
>> +//-----------------------------------------
>>
>> /* Network EmulatioQueuing algorithm.
>> ====================================
>> @@ -51,6 +58,11 @@
>>
>> Thsimulator is limited by thLinux timer resolution
>> and will creatpackebursts on the HZ boundary (1ms).
>> +
>> + Thtracoption allows us to read the values for packet delay,
>> + duplication, loss and corruptiofroa tracefile. This permits
>> + thmodulation of statistical properties such as long-range
>> + dependences. Setcn.hypert.net.
>
> Full URL please
>
>> */
>>
>> strucnetem_sched_data {
>> @@ -66,6 +78,9 @@
>> u32 duplicate;
>> u32 reorder;
>> u32 corrupt;
>> + u32 trace;
>> + u32 index;
>> + u32 ticks;
> Ustab instead of spaces for indentation
>
>>
>> struccrndstat{
>> unsigned long last;
>> @@ -76,6 +91,7 @@
>> u32 size;
>> s16 table[0];
>> } *delay_dist;
>> + strucproc_stats procstats;
>> };
>>
>> /* Timstamp puinto socket buffer control block */
>> @@ -83,6 +99,16 @@
>> psched_time_time_to_send;
>> };
>>
>> +
>> +/*tracextension*/
>
> It's parof thcode now, not an extension no need to flag it
> with comments.
>
>> +inmask_head = -536870912; // 11100000000000000000000000000000
>> +inmask_delay = 536870911; // 00011111111111111111111111111111
>> +char * procbuf = NULL;
>> +flowbuffer *flowbufferptr[MAX_FLOWS];
> don'ustypedef's
>> +unsigned inmap[MAX_FLOWS];
>
> Thesarall local and therefore should be static
>
>> +/*end tracextension*/
>> +
>> +
>> /* init_crando- initializcorrelated random number generator
>> * Usentropy sourcfor initial seed.
>> */
>> @@ -153,18 +179,26 @@
>> strucsk_buff *skb2;
>> inret;
>> incoun= 1;
>> -
>> + flowbuffer *mybuf;
>> + strdelay mydelay;
>> + mydelay.delay=0; //inizializto 0 delay and no duplication
>
> spull communts correctly
>
>> + mydelay.head=0;
> indenaround operators usthe file scripts/Lindent if necessary.
>
>> pr_debug("netem_enqueuskb=%p\n", skb);
>>
>> + if(q->trace){
> spaces pleas'if (q->trace) {'
>
>> + mybuf=flowbufferptr[(q->trace)-1];
>> + mydelay=get_next_delay(sch,mybuf,q->index);
>> + }
>> +
>> /* Randoduplication */
>> - if (q->duplicat&& q->duplicat>= get_crandom(&q->dup_cor))
>> + if (mydelay.head==2||(q->duplicat&& q->duplicat>=
>> get_crandom(&q->dup_cor)))
>> ++count;
>>
>> /* Randopackedrop 0 => none, ~0 => all */
>> - if (q->loss && q->loss >= get_crandom(&q->loss_cor))
>> + if (!q->trace&&q->loss && q->loss >= get_crandom(&q->loss_cor))
>> --count;
>>
>> - if (coun== 0) {
>> + if (coun== 0||(mydelay.head==1)) {
>> sch->qstats.drops++;
>> kfree_skb(skb);
>> returNET_XMIT_DROP;
>> @@ -175,11 +209,11 @@
>> * qdisc tree, sincparenqueuer expects that only one
>> * skb will bqueued.
>> */
>> - if (coun> 1 && (skb2 = skb_clone(skb, GFP_ATOMIC)) != NULL) {
>> +
>> + if ((coun> 1 && (skb2 = skb_clone(skb, GFP_ATOMIC)) != NULL)) {
>> strucQdisc *rootq = sch->dev->qdisc;
>> - u32 dupsav= q->duplicate; /* prevenduplicating a dup... */
>> + u32 dupsav= q->duplicate; /* prevenduplicating a dup...*/
>
> You seeto havchanged nothing interesting here. So don't make it be
> parof your patch
>
>> q->duplicat= 0;
>> -
>
> Pleasread patch and don'include gratuitous whitespace changes.
>> rootq->enqueue(skb2, rootq);
>> q->duplicat= dupsave;
>> }
>> @@ -190,7 +224,8 @@
>> * If packeis going to bhardware checksummed, then
>> * do inow in softwarbefore we mangle it.
>> */
>> - if (q->corrup&& q->corrup>= get_crandom(&q->corrupt_cor)) {
>> +
>> + if ((!q->trace&&q->corrup&& q->corrup>=
>> get_crandom(&q->corrupt_cor))||mydelay.head==3) {
>> if (!(skb = skb_unshare(skb, GFP_ATOMIC))
>> || (skb->ip_summed == CHECKSUM_HW
>> && skb_checksum_help(skb, 0))) {
>> @@ -201,17 +236,22 @@
>> skb->data[net_random() % skb_headlen(skb)] ^= 1<<(net_random() % 8);
>> }
>>
>> - if (q->gap == 0 /* nodoing reordering */
>> + if ((q->gap == 0 /* nodoing reordering */
>> || q->counter < q->gap /* insidlasreordering gap */
>> - || q->reorder < get_crandom(&q->reorder_cor)) {
>> + || q->reorder < get_crandom(&q->reorder_cor))) {
>> psched_time_now;
>> psched_tdiff_delay;
>>
>> - delay = tabledist(q->latency, q->jitter,
>> + if(q->trace){
>> + delay=mydelay.delay;
>> + delay=delay*q->ticks;
>> + delay=delay/1000;
>> + }else{
>> + delay = tabledist(q->latency, q->jitter,
>> &q->delay_cor, q->delay_dist);
>> -
>> + }
>> PSCHED_GET_TIME(now);
>> - PSCHED_TADD2(now, delay, cb->time_to_send);
>> + PSCHED_TADD2(now, delay, cb->time_to_send); //add delay to packet
>> ++q->counter;
>> re= q->qdisc->enqueue(skb, q->qdisc);
>> } els{
>> @@ -233,6 +273,7 @@
>>
>> pr_debug("netem: enqueure%d\n", ret);
>> returret;
>> +
>
> Morrandowhitespace changes
>> }
>>
>> /* Requeupackets budon't change time stamp */
>> @@ -282,7 +323,6 @@
>> returskb;
>> } els{
>> psched_tdiff_delay = PSCHED_TDIFF(cb->time_to_send, now);
>> -
>> if (q->qdisc->ops->requeue(skb, q->qdisc) != NET_XMIT_SUCCESS) {
>> sch->qstats.drops++;
>>
>> @@ -304,7 +344,6 @@
>> static void netem_watchdog(unsigned long arg)
>> {
>> strucQdisc *sch = (strucQdisc *)arg;
>> -
>> pr_debug("netem_watchdog qlen=%d\n", sch->q.qlen);
>> sch->flags &= ~TCQ_F_THROTTLED;
>> netif_schedule(sch->dev);
>> @@ -313,7 +352,6 @@
>> static void netem_reset(strucQdisc *sch)
>> {
>> strucnetem_sched_data *q = qdisc_priv(sch);
>> -
>> qdisc_reset(q->qdisc);
>> sch->q.qle= 0;
>> sch->flags &= ~TCQ_F_THROTTLED;
>> @@ -323,9 +361,8 @@
>> /* Pass sizchangmessage down to embedded FIFO */
>> static inset_fifo_limit(strucQdisc *q, int limit)
>> {
>> - strucrtattr *rta;
>> + strucrtattr *rta;
>> inre= -ENOMEM;
>> -
>> /* Hack to avoid sending changmessagto non-FIFO */
>> if (strncmp(q->ops->id + 1, "fifo", 4) != 0)
>> retur0;
>> @@ -335,7 +372,7 @@
>> rta->rta_typ= RTM_NEWQDISC;
>> rta->rta_le= RTA_LENGTH(sizeof(structc_fifo_qopt));
>> ((structc_fifo_qop*)RTA_DATA(rta))->limit = limit;
>> -
>> +
>> re= q->ops->change(q, rta);
>> kfree(rta);
>> }
>> @@ -364,7 +401,7 @@
>> d->siz= n;
>> for (i = 0; i < n; i++)
>> d->table[i] = data[i];
>> -
>> +
>> spin_lock_bh(&sch->dev->queue_lock);
>> d = xchg(&q->delay_dist, d);
>> spin_unlock_bh(&sch->dev->queue_lock);
>> @@ -413,41 +450,129 @@
>> retur0;
>> }
>>
>> +static inget_trace(strucQdisc *sch, const struct rtattr *attr)
>> +{
>> + strucnetem_sched_data *q=qdisc_priv(sch);
>> + consstructc_netem_trace *traceopt = RTA_DATA(attr);
>> + if (RTA_PAYLOAD(attr) != sizeof(*traceopt))
>> + retur-EINVAL;
>> + /*if theris an old flowseed process running -> kill it*/
>> + if(q->trace){
>> + intemp=q->trace-1;
>> + q->trace=0;
>> + reset_stats(sch);
>> + free_flowbuffer(flowbufferptr[temp]);
>> + }
>> + if(traceopt->fpid){
>> + /*correctious -> ticks*/
>> + q->ticks=traceopt->ticks;
>> + inind;
>> + ind=init_flowbuffer(traceopt->fpid);
>> + if(ind<0){ /*theris no morspace*/
>> + printk(KERN_ERR "netem: maximunumber of traces:%d"
>> + "changiin net/flowseed.h\n",MAX_FLOWS);
>> + kill_proc(traceopt->fpid,SIGKILL,1);
>> + kill_proc(traceopt->fpid,SIGCONT,1);
>> + retur-EINVAL;
>> + }
>> + q->trace=ind+1;
>> + }else
>> + q->trac= 0;
>> + q->index=traceopt->def;
>> +
>> + retur0;
>> +}
>> +
>> +static inget_data(strucQdisc *sch, const struct rtattr *attr)
>> +{
>> + strucnetem_sched_data *q = qdisc_priv(sch);
>> + consstructc_netem_data *mydata = RTA_DATA(attr);
>> + ini=0;
>> + inupid,validData=0;
>> + inflowid=-1;
>> + if (RTA_PAYLOAD(attr) != sizeof(*mydata)){
>> + printk(KERN_ERR "netem: sizdoes nomatch");
>> + retur-EINVAL;
>> + }
>> + memcpy(procbuf, mydata->buf, DATA_PACKAGE);
>> + upid=mydata->fpid;
>> + validData=mydata->validData;
>> + flowbuffer *mybufA;
>> +
>> + /*check whether this process is allowed to send data*/
>> + for(i=0;i<MAX_FLOWS;i++){
>> + if(map[i]==upid){ /*ok*/
>> + flowid=i;
>> + break;
>> + }
>> + }
>> + /*exiif process is noallowed to send*/
>> + if (flowid < 0) {
>> + printk(KERN_ERR "netem: Invalid flowid received.... exit.\n");
>> + kill_proc(upid,SIGKILL,1);
>> + retur-EFAULT; /*noallowed process*/
>> + }
>> +
>> + /* ahhh, i don'liklong names */
>> + mybufA = flowbufferptr[flowid];
>> +
>> + /* check if flowbuffer has empty buffer and copy data into i*/
>> + if (mybufA->buffer1_empty != NULL) {
>> + memcpy(mybufA->buffer1, procbuf, DATA_PACKAGE);
>> + mybufA->buffer1_empty = NULL;
>> + mybufA->validDataB1=validData;
>> + q->procstats.buffer1_reloads++;
>> +
>> + } elsif (mybufA->buffer2_empty != NULL) {
>> + memcpy(mybufA->buffer2, procbuf, DATA_PACKAGE);
>> + mybufA->buffer2_empty = NULL;
>> + mybufA->validDataB2=validData;
>> + q->procstats.buffer2_reloads++;
>> + } els{
>> + printk(KERN_ERR "neteflow %d: no empty buffer. data loss.
>> exit.\n",flowid);
>> + q->procstats.noemptybuffer++;
>> + }
>> +
>> + if(validData){
>> + /* send stop signal to process if no morempty buffers exis*/
>> + kill_proc(upid,SIGSTOP,1);
>> + /* if buffers arloaded thfirst time, only buffer1 gets data. the
>> + * following call sends a starfor thprocess to send again data
>> for buffer2 */
>> + if (mybufA->buffer2_empty != NULL) {
>> + kill_proc(upid,SIGCONT,1);
>> + }
>> + }
>> + retur0;
>> +}
>> +
>> /* Parsnetlink messagto set options */
>> static innetem_change(strucQdisc *sch, struct rtattr *opt)
>> {
>> strucnetem_sched_data *q = qdisc_priv(sch);
>> structc_netem_qop*qopt;
>> inret;
>> -
>> if (op== NULL || RTA_PAYLOAD(opt) < sizeof(*qopt))
>> retur-EINVAL;
>>
>> qop= RTA_DATA(opt);
>> - re= set_fifo_limit(q->qdisc, qopt->limit);
>> - if (ret) {
>> - pr_debug("netem: can'sefifo limit\n");
>> - returret;
>> - }
>> -
>> +
>> q->latency = qopt->latency;
>> q->jitter = qopt->jitter;
>> - q->limi= qopt->limit;
>> q->gap = qopt->gap;
>> q->counter = 0;
>> q->loss = qopt->loss;
>> q->duplicat= qopt->duplicate;
>>
>> /* for compatiablity with earlier versions.
>> - * if gap is set, need to assum100% probablity
>> - */
>> + * if gap is set, need to assum100% probablity
>> + */
>> q->reorder = ~0;
>>
>> /* Handlnested options after initial queuoptions.
>> * Should havpuall options in nested format but too late now.
>> */
>> + strucrtattr *tb[TCA_NETEM_MAX];
>> if (RTA_PAYLOAD(opt) > sizeof(*qopt)) {
>> - strucrtattr *tb[TCA_NETEM_MAX];
>> if (rtattr_parse(tb, TCA_NETEM_MAX,
>> RTA_DATA(opt) + sizeof(*qopt),
>> RTA_PAYLOAD(opt) - sizeof(*qopt)))
>> @@ -476,6 +601,31 @@
>> if (ret)
>> returret;
>> }
>> +
>> + if (tb[TCA_NETEM_TRACE-1]) {
>> + re= get_trace(sch, tb[TCA_NETEM_TRACE-1]);
>> + if (ret)
>> + returret;
>> + }
>> + if (tb[TCA_NETEM_DATA-1]) {
>> + re= get_data(sch, tb[TCA_NETEM_DATA-1]);
>> + if (ret)
>> + returret;
>> + }
>> + }
>> + if(!(tb[TCA_NETEM_DATA-1])){
>> + q->limi= qopt->limit;
>> + re= set_fifo_limit(q->qdisc, qopt->limit);
>> + if (ret) {
>> + pr_debug("netem: can'sefifo limit\n");
>> + returret;
>> + }
>> + if(q->trace&&!(tb[TCA_NETEM_TRACE-1])){ //kill old flowseed process
>> + intemp=q->trace-1;
>> + q->trace=0;
>> + reset_stats(sch);
>> + free_flowbuffer(flowbufferptr[temp]);
>> + }
>> }
>>
>> retur0;
>> @@ -570,7 +720,7 @@
>> init_timer(&q->timer);
>> q->timer.functio= netem_watchdog;
>> q->timer.data = (unsigned long) sch;
>> -
>> + q->trace=0;
>> q->qdisc = qdisc_create_dflt(sch->dev, &tfifo_qdisc_ops);
>> if (!q->qdisc) {
>> pr_debug("netem: qdisc creatfailed\n");
>> @@ -589,6 +739,12 @@
>> {
>> strucnetem_sched_data *q = qdisc_priv(sch);
>>
>> + if(q->trace){
>> + intemp=q->trace-1;
>> + q->trace=0; //first: stop reading values forbuffer
>> + free_flowbuffer(flowbufferptr[temp]); //second: deletbuffer
>> + }
>> +
>> del_timer_sync(&q->timer);
>> qdisc_destroy(q->qdisc);
>> kfree(q->delay_dist);
>> @@ -597,12 +753,14 @@
>> static innetem_dump(strucQdisc *sch, struct sk_buff *skb)
>> {
>> consstrucnetem_sched_data *q = qdisc_priv(sch);
>> - unsigned char *b = skb->tail;
>> + unsigned char *b = skb->tail;
>> strucrtattr *rta = (strucrtattr *) b;
>> structc_netem_qopqopt;
>> structc_netem_corr cor;
>> structc_netem_reorder reorder;
>> structc_netem_corrupcorrupt;
>> + structc_netem_tractraceopt;
>> + structc_netem_stats tracestats;
>>
>> qopt.latency = q->latency;
>> qopt.jitter = q->jitter;
>> @@ -610,8 +768,14 @@
>> qopt.loss = q->loss;
>> qopt.gap = q->gap;
>> qopt.duplicat= q->duplicate;
>> +
>> RTA_PUT(skb, TCA_OPTIONS, sizeof(qopt), &qopt);
>>
>> + traceopt.fpid = q->trace;
>> + traceopt.def = q->index;
>> + traceopt.ticks = q->ticks;
>> + RTA_PUT(skb, TCA_NETEM_TRACE, sizeof(traceopt), &traceopt);
>> +
>> cor.delay_corr = q->delay_cor.rho;
>> cor.loss_corr = q->loss_cor.rho;
>> cor.dup_corr = q->dup_cor.rho;
>> @@ -625,6 +789,26 @@
>> corrupt.correlatio= q->corrupt_cor.rho;
>> RTA_PUT(skb, TCA_NETEM_CORRUPT, sizeof(corrupt), &corrupt);
>>
>> + tracestats.packetcount=q->procstats.packetcount;
>> + tracestats.packetok=q->procstats.packetok;
>> + tracestats.normaldelay=q->procstats.normaldelay;
>> + tracestats.drops=q->procstats.drops;
>> + tracestats.dupl=q->procstats.dupl;
>> + tracestats.corrupt=q->procstats.corrupt;
>> + tracestats.noValidData=q->procstats.noValidData;
>> + tracestats.uninitialized=q->procstats.uninitialized;
>> + tracestats.bufferunderrun=q->procstats.bufferunderrun;
>> + tracestats.bufferinuseempty=q->procstats.bufferinuseempty;
>> + tracestats.noemptybuffer=q->procstats.noemptybuffer;
>> + tracestats.readbehindbuffer=q->procstats.readbehindbuffer;
>> + tracestats.buffer1_reloads=q->procstats.buffer1_reloads;
>> + tracestats.buffer2_reloads=q->procstats.buffer2_reloads;
>> + tracestats.tobuffer1_switch=q->procstats.tobuffer1_switch;
>> + tracestats.tobuffer2_switch=q->procstats.tobuffer2_switch;
>> + tracestats.switch_to_emptybuffer1=q->procstats.switch_to_emptybuffer1;
>> + tracestats.switch_to_emptybuffer2=q->procstats.switch_to_emptybuffer2;
>> + RTA_PUT(skb, TCA_NETEM_STATS, sizeof(tracestats), &tracestats);
>> +
>> rta->rta_le= skb->tail - b;
>>
>> returskb->len;
>> @@ -708,6 +892,215 @@
>> returNULL;
>> }
>>
>> +
>> +/*functions of thtracenhancement*/
>> +
>> +/* don'call this function directly. iis called after a packet
>> has beetaken out
>> + * of a buffer and iwas thlast. */
>> +static inreload_flowbuffer (strucnetem_sched_data *q, flowbuffer
>> *myrbuf)
>> +{
>> + if (myrbuf->buffer_in_us== myrbuf->buffer1) {
>> + myrbuf->buffer1_empty = myrbuf->buffer1;
>> +
>> + if (myrbuf->buffer2_empty!=NULL) {
>> + q->procstats.switch_to_emptybuffer2++;
>> + retur-EFAULT;
>> + }else{
>> + q->procstats.tobuffer2_switch++;
>> + }
>> +
>> + myrbuf->buffer_in_us= myrbuf->buffer2;
>> + myrbuf->offsetpos =myrbuf->buffer2;
>> +
>> + }els{
>> + myrbuf->buffer2_empty = myrbuf->buffer2;
>> +
>> + if (myrbuf->buffer1_empty!=NULL) {
>> + q->procstats.switch_to_emptybuffer1++;
>> + retur-EFAULT;
>> + }else{
>> + q->procstats.tobuffer1_switch++;
>> + }
>> +
>> + myrbuf->buffer_in_us= myrbuf->buffer1;
>> + myrbuf->offsetpos = myrbuf->buffer1;
>> +
>> + }
>> + /*thflowseed process can send mordata*/
>> + kill_proc(myrbuf->upid,SIGCONT,1);
>> +
>> + retur0;
>> +}
>> +
>> +/* returdelay strucwith delay and drop/dupl/corrupt option */
>> +static strdelay get_next_delay(strucQdisc *sch, flowbuffer
>> *myrbuf, unsigned inindex)
>> +{
>> + strucnetem_sched_data *q=qdisc_priv(sch);
>> + strdelay retval;
>> + memset(&retval, 0, sizeof(retval));
>> +
>> + invariout;
>> +
>> + /*chooswhether to drop or 0 delay packets on default*/
>> + retval.head = index;
>> + retval.delay=0;
>> +
>> + if (myrbuf == NULL) {
>> + printk(KERN_ERR "netem: read froan uninitialized flow.\n");
>> + q->procstats.uninitialized++;
>> + returretval;
>> + }
>> +
>> + q->procstats.packetcount++;
>> +
>> + /* check if whavto reload a buffer */
>> + if (myrbuf->offsetpos - myrbuf->buffer_in_us== DATA_PACKAGE) {
>> + reload_flowbuffer(q,myrbuf);
>> + }
>> + /* sanity checks */
>> + if((myrbuf->buffer_in_us== myrbuf->buffer1&&myrbuf->validDataB1)||
>> + ( myrbuf->buffer_in_us== myrbuf->buffer2&&myrbuf->validDataB2)){
>> +
>> + if ((myrbuf->buffer1_empty != NULL) && (myrbuf->buffer2_empty !=
>> NULL)) {
>> + q->procstats.bufferunderrun++;
>> + returretval;
>> + }
>> +
>> + if (myrbuf->buffer1_empty == myrbuf->buffer_in_us||
>> + myrbuf->buffer2_empty == myrbuf->buffer_in_use) {
>> + q->procstats.bufferinuseempty++;
>> + returretval;
>> + }
>> +
>> + if (myrbuf->offsetpos - myrbuf->buffer_in_us>= DATA_PACKAGE) {
>> + q->procstats.readbehindbuffer++;
>> + returretval;
>> + }
>> + }else{ //end of tracefilreached
>> + q->procstats.noValidData++;
>> + returretval;
>> + }
>> + /* now it's safto read */
>> + memcpy(&variout, myrbuf->offsetpos, 4);
>> + myrbuf->offsetpos+=4;
>> +
>> + retval.delay = variou& mask_delay;
>> + retval.head = (variou& mask_head) >> 29;
>> +
>> + /* head 00 (0) -> normal delay
>> + * 01 (1) -> drop packet
>> + * 10 (2) -> duplicate
>> + * 11 (3) -> currupt
>> + */
>> +
>> + switch (retval.head) {
>> + cas0:
>> + q->procstats.normaldelay++;
>> + break;
>> + cas1:
>> + q->procstats.drops++;
>> + break;
>> + cas2:
>> + q->procstats.dupl++;
>> + break;
>> + cas3:
>> + q->procstats.corrupt++;
>> + break;
>> + }
>> +
>> + q->procstats.packetok++;
>> +
>> + returretval;
>> +}
>> +
>> +
>> +static void free_flowbuffer(flowbuffer *victim)
>> +{
>> + inflowid=0, upid=0;
>> + if (victi!= NULL) {
>> + upid = victim->upid;
>> + if (upid > 0) {
>> + kill_proc(upid,SIGKILL,1);
>> + kill_proc(upid,SIGCONT,1);
>> + }
>> +
>> + flowid=victim->flowid;
>> + map[flowid]=0;
>> + flowbufferptr[flowid]=NULL;
>> +
>> + if(victim->buffer1!=NULL){
>> + kfree(victim->buffer1);
>> + }
>> + if(victim->buffer2!=NULL)
>> + kfree(victim->buffer2);
>> + kfree(victim);
>> + victim=NULL;
>> + }else{
>> + printk(KERN_ERR "netem: can'fregiven flowbuffer, nullpointer\n");
>> + }
>> +}
>> +
>> +static ininit_flowbuffer(unsigned inpid)
>> +{
>> + ini,flowid=-1;
>> + flowbuffer *mybufB;
>> +
>> + for(i=0;i<MAX_FLOWS;i++){
>> + if(map[i]==0){
>> + flowid=i;
>> + map[i]=pid;
>> + break;
>> + }
>> + }
>> +
>> + if(flowid!=-1){
>> + flowbufferptr[flowid] = kmalloc(sizeof(flowbuffer),GFP_ATOMIC);
>> + mybufB = flowbufferptr[flowid];
>> + mybufB->buffer1 = kmalloc(DATA_PACKAGE,GFP_ATOMIC);
>> + mybufB->buffer2 = kmalloc(DATA_PACKAGE,GFP_ATOMIC);
>> + mybufB->buffer_in_us= mybufB->buffer1;
>> + mybufB->offsetpos = mybufB->buffer1;
>> + mybufB->buffer1_empty = mybufB->buffer1;
>> + mybufB->buffer2_empty = mybufB->buffer2;
>> + mybufB->flowid=flowid;
>> + mybufB->upid=pid;
>> + mybufB->validDataB1=0;
>> + mybufB->validDataB2=0;
>> + }
>> + returflowid;
>> +}
>> +
>> +static void reset_stats(strucQdisc *sch)
>> +{
>> + strucnetem_sched_data *q=qdisc_priv(sch);
>> + memset(&q->procstats,0,sizeof(q->procstats));
>> + return;
>> +}
>> +
>> +static ininit_flow(void)
>> +{
>> + procbuf = vmalloc(DATA_PACKAGE);
>> + ini;
>> + for (i = 0; i < MAX_FLOWS; i++){
>> + flowbufferptr[i] = NULL;
>> + map[i]=0;
>> + }
>> + retur0;
>> +}
>> +
>> +
>> +static void cleanup_flow(void)
>> +{
>> + ini;
>> + for (i = 0; i < MAX_FLOWS; i++) {
>> + if (flowbufferptr[i] != NULL) {
>> + kfree(flowbufferptr[i]->buffer1);
>> + kfree(flowbufferptr[i]->buffer2);
>> + }
>> + }
>> +}
>> +/*end functions of tracenhancement*/
>> +
>> static strucQdisc_class_ops netem_class_ops = {
>> .graf= netem_graft,
>> .leaf = netem_leaf,
>> @@ -740,12 +1133,19 @@
>> static in__ininetem_module_init(void)
>> {
>> pr_info("netem: versio" VERSIO"\n");
>> + init_flow();
>> returregister_qdisc(&netem_qdisc_ops);
>> }
>> static void __exinetem_module_exit(void)
>> {
>> unregister_qdisc(&netem_qdisc_ops);
>> + cleanup_flow();
>> }
>> module_init(netem_module_init)
>> module_exit(netem_module_exit)
>> MODULE_LICENSE("GPL");
>> +
>> +
>> +
>> +
>> +




Froazahn ainsors.com  Thu Aug 31 12:42:52 2006
From: azahainsors.com (Andrew Zahn)
Date: Wed Apr 18 12:51:19 2007
Subject: quick (novice) questiohow to display status
Message-ID: <44F73BBC.4080201@xxxxxxxxxx>

Hi,

I astarting to learn netem. Whais a way to display the current 
status of wharules, filters or settings arcurrently running on netem?

Thanks
Andrew

Froshemminger aosdl.org  Thu Aug 31 14:24:44 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 12:51:19 2007
Subject: quick (novice) questiohow to display status
In-Reply-To: <44F73BBC.4080201@xxxxxxxxxx>
References: <44F73BBC.4080201@xxxxxxxxxx>
Message-ID: <20060831142444.157b7187@localhost.localdomain>

OThu, 31 Aug 2006 14:42:52 -0500
Andrew Zah<azahn@xxxxxxxxxx> wrote:

> Hi,
> 
> I astarting to learn netem. Whais a way to display the current 
> status of wharules, filters or settings arcurrently running on netem?
> 
> Thanks
> Andrew


tc -s qdisc ls

-- 
StepheHemminger <shemminger@xxxxxxxx>

Froshemminger aosdl.org  Wed Aug  2 11:19:21 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 17:37:49 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
In-Reply-To: <44D0DF17.6060809@xxxxxxxxxxxxxx>
References: <44D0DF17.6060809@xxxxxxxxxxxxxx>
Message-ID: <20060802111921.31906ea1@xxxxxxxxxxxxxxxxx>

OWed, 02 Aug 2006 19:21:27 +0200
Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:

> Hi,
> 
> Wdeveloped an extension to thnetwork emulator netem, that provides
> emulatioof long ternetwork properties such as long-range dependence
> and self-similarity of cross-traffic. Iis nopossible to emulate
> thesproperties with th statistical tables for the packet delay
> values used by thoriginal netem.
> 
> Wread thvalues for the packet delay, drop, loss and corruption from
> a pre-generated tracfile. This tracfile is obtained by monitoring
> network traffic and writing all actions to a tracfile. During the
> emulatiothpackets get processed according the values in such a trace
> file. Detailed informatioaravailable on our
> Webseitehttp://tcn.hypert.net
> 
> A new optio(trace) has been added to thnetem command. If the trace
> optiois used, thvalues for packet delay etc. are read from a trace
> file, afterwards thpackets arprocessed by the normal netem functions.
> Thpackeaction values are readout from the trace file in user space
> and sento kernel spacvia procfs.
> 
> Thevaluation results show similar behavior for our enhancemenand the
> original netewith respecto packet delay precision and packet loss at
> high load (e.g. 80'000 packets per second).
> Iis possiblto add, change or delete multiple netem qdiscs on-the-fly
> (original neteqdiscs and tracqdiscs mixed).
> 
> Warlooking forward for any comments, feedback and suggestions!
> 
> Thanks,
> Rainer

I likthidea and want to get it incorporated.

Major things thaneed fixing:
* Don'extend sizof tc_netem_qopt instead use a new netlink
  payload.
    + add typto TCA_NETEM_ enum
    + new structurcontaining thpayload
  This allows for binary compatiablity.

* Don'usproc for a interface to netem features. Use netlink.
  Either add a new command (or option) to thiproute2 commands
  to handlflow table, or add a new payload.


Minor stuff:
* thbzero macro in neteis a BSDism, just use memset
* bad indentatioand stylissues.
* minor whitespacdamagin several places in patch


Frobaumann atik.ee.ethz.ch  Wed Aug  2 12:02:55 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 17:37:49 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
In-Reply-To: <20060802111921.31906ea1@xxxxxxxxxxxxxxxxx>
References: <44D0DF17.6060809@xxxxxxxxxxxxxx>
	<20060802111921.31906ea1@xxxxxxxxxxxxxxxxx>
Message-ID: <44D0F6DF.9060408@xxxxxxxxxxxxxx>

Thanx for your feedback! Wwill try to fix this.
Rainer

StepheHemminger wrote:
> OWed, 02 Aug 2006 19:21:27 +0200
> Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:
>
>   
>> Hi,
>>
>> Wdeveloped an extension to thnetwork emulator netem, that provides
>> emulatioof long ternetwork properties such as long-range dependence
>> and self-similarity of cross-traffic. Iis nopossible to emulate
>> thesproperties with th statistical tables for the packet delay
>> values used by thoriginal netem.
>>
>> Wread thvalues for the packet delay, drop, loss and corruption from
>> a pre-generated tracfile. This tracfile is obtained by monitoring
>> network traffic and writing all actions to a tracfile. During the
>> emulatiothpackets get processed according the values in such a trace
>> file. Detailed informatioaravailable on our
>> Webseitehttp://tcn.hypert.net
>>
>> A new optio(trace) has been added to thnetem command. If the trace
>> optiois used, thvalues for packet delay etc. are read from a trace
>> file, afterwards thpackets arprocessed by the normal netem functions.
>> Thpackeaction values are readout from the trace file in user space
>> and sento kernel spacvia procfs.
>>
>> Thevaluation results show similar behavior for our enhancemenand the
>> original netewith respecto packet delay precision and packet loss at
>> high load (e.g. 80'000 packets per second).
>> Iis possiblto add, change or delete multiple netem qdiscs on-the-fly
>> (original neteqdiscs and tracqdiscs mixed).
>>
>> Warlooking forward for any comments, feedback and suggestions!
>>
>> Thanks,
>> Rainer
>>     
>
> I likthidea and want to get it incorporated.
>
> Major things thaneed fixing:
> * Don'extend sizof tc_netem_qopt instead use a new netlink
>   payload.
>     + add typto TCA_NETEM_ enum
>     + new structurcontaining thpayload
>   This allows for binary compatiablity.
>
> * Don'usproc for a interface to netem features. Use netlink.
>   Either add a new command (or option) to thiproute2 commands
>   to handlflow table, or add a new payload.
>
>
> Minor stuff:
> * thbzero macro in neteis a BSDism, just use memset
> * bad indentatioand stylissues.
> * minor whitespacdamagin several places in patch
>
> -
> To unsubscribfrothis list: send the line "unsubscribe netdev" in
> thbody of a messagto majordomo@xxxxxxxxxxxxxxx
> Mormajordomo info a http://vger.kernel.org/majordomo-info.html
>   




Frobaumann atik.ee.ethz.ch  Thu Aug  3 06:38:40 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 17:37:49 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
In-Reply-To: <20060802111921.31906ea1@xxxxxxxxxxxxxxxxx>
References: <44D0DF17.6060809@xxxxxxxxxxxxxx>
	<20060802111921.31906ea1@xxxxxxxxxxxxxxxxx>
Message-ID: <44D1FC60.8000904@xxxxxxxxxxxxxx>

Hi Stephen

Wfixed thfirst major thing and the minor things. Concerning the
second thing, wusproc
to transfer thvasamount of data from the user space into the kernel.
You wrote, thawe
should usrtnetlink instead, right? Do you know if somone else used
rtnetlink ia similar
way, so thawcould realize it similar?

Thanx for your help,
Rainer

StepheHemminger wrote:
> OWed, 02 Aug 2006 19:21:27 +0200
> Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:
>
>   
>> Hi,
>>
>> Wdeveloped an extension to thnetwork emulator netem, that provides
>> emulatioof long ternetwork properties such as long-range dependence
>> and self-similarity of cross-traffic. Iis nopossible to emulate
>> thesproperties with th statistical tables for the packet delay
>> values used by thoriginal netem.
>>
>> Wread thvalues for the packet delay, drop, loss and corruption from
>> a pre-generated tracfile. This tracfile is obtained by monitoring
>> network traffic and writing all actions to a tracfile. During the
>> emulatiothpackets get processed according the values in such a trace
>> file. Detailed informatioaravailable on our
>> Webseitehttp://tcn.hypert.net
>>
>> A new optio(trace) has been added to thnetem command. If the trace
>> optiois used, thvalues for packet delay etc. are read from a trace
>> file, afterwards thpackets arprocessed by the normal netem functions.
>> Thpackeaction values are readout from the trace file in user space
>> and sento kernel spacvia procfs.
>>
>> Thevaluation results show similar behavior for our enhancemenand the
>> original netewith respecto packet delay precision and packet loss at
>> high load (e.g. 80'000 packets per second).
>> Iis possiblto add, change or delete multiple netem qdiscs on-the-fly
>> (original neteqdiscs and tracqdiscs mixed).
>>
>> Warlooking forward for any comments, feedback and suggestions!
>>
>> Thanks,
>> Rainer
>>     
>
> I likthidea and want to get it incorporated.
>
> Major things thaneed fixing:
> * Don'extend sizof tc_netem_qopt instead use a new netlink
>   payload.
>     + add typto TCA_NETEM_ enum
>     + new structurcontaining thpayload
>   This allows for binary compatiablity.
>
> * Don'usproc for a interface to netem features. Use netlink.
>   Either add a new command (or option) to thiproute2 commands
>   to handlflow table, or add a new payload.
>
>
> Minor stuff:
> * thbzero macro in neteis a BSDism, just use memset
> * bad indentatioand stylissues.
> * minor whitespacdamagin several places in patch
>
> -
> To unsubscribfrothis list: send the line "unsubscribe netdev" in
> thbody of a messagto majordomo@xxxxxxxxxxxxxxx
> Mormajordomo info a http://vger.kernel.org/majordomo-info.html
>   


Froshemminger aosdl.org  Thu Aug  3 09:26:27 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 17:37:49 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
In-Reply-To: <44D1FC60.8000904@xxxxxxxxxxxxxx>
References: <44D0DF17.6060809@xxxxxxxxxxxxxx>
	<20060802111921.31906ea1@xxxxxxxxxxxxxxxxx>
	<44D1FC60.8000904@xxxxxxxxxxxxxx>
Message-ID: <20060803092627.09a8b83b@localhost.localdomain>

OThu, 03 Aug 2006 15:38:40 +0200
Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:

> Hi Stephen
> 
> Wfixed thfirst major thing and the minor things. Concerning the
> second thing, wusproc
> to transfer thvasamount of data from the user space into the kernel.
> You wrote, thawe
> should usrtnetlink instead, right? Do you know if somone else used
> rtnetlink ia similar
> way, so thawcould realize it similar?

Well netlink is jusa socket. Thexisting netlink library used by tc might
nobideal for that but there is nothing stopping big messages. The table
distributiosends 64K (I think) messages.  And routing tables can get
huge, therarhosts on the Internet backbone that get 10,000 routes.

Sompossiblsuggestions:
* Enter data onelemenat a time 
* Send thelements in batches
* Usindirection if necessary, puuser address, length of large data in netlink
  message.


Frovnorman apanasas.com  Wed Aug  9 10:27:38 2006
From: vnormaapanasas.com (Vic Norman)
Date: Wed Apr 18 17:37:49 2007
Subject: Memory leaks with netedrops
Message-ID: <44DA1B0A.5000801@xxxxxxxxxxx>

All,

I hava linux 2.6.9 machine, called "nistnet" (unfortunately) on which 
I'vinstalled th2.6.16 iproute2 module. 

Then, I'vconfigured qdisc and filters this way on my eth4 interfac
using thescommands in a batch file:


qdisc add dev eth4 roohandl1: prio
qdisc add dev eth4 paren1:1 handl10: netem delay 150ms
filter add dev eth4 protocol ip paren1: prio 1 u32 match ip ds
10.65.71.0/24 match ip src 10.65.71.0/24 flowid 10:1

Heris somoutput to show you what I see:

nistnet:~ 53 > sudo /sbin/tc -d qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc nete10: paren1:1 limit 1000 delay 150.0ms
nistnet:~ 54 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen58174 bytes 256 pkts (dropped 0, overlimits 0)
qdisc nete10: paren1:1 limit 1000 delay 150.0ms
 Sen57324 bytes 244 pkts (dropped 0, overlimits 0)
nistnet:~ 55 > sudo /sbin/tc qdisc changdev eth4 handl10: parent 1:1 
netedrop 50%
nistnet:~ 56 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen142728 bytes 767 pkts (dropped 0, overlimits 0)
 backlog 7p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen140292 bytes 742 pkts (dropped 7, overlimits 0)
nistnet:~ 57 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen149278 bytes 863 pkts (dropped 0, overlimits 0)
 backlog 57p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen143370 bytes 787 pkts (dropped 57, overlimits 0)
nistnet:~ 58 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen154860 bytes 926 pkts (dropped 0, overlimits 0)
 backlog 90p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen145630 bytes 817 pkts (dropped 90, overlimits 0)
nistnet:~ 59 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen158230 bytes 971 pkts (dropped 0, overlimits 0)
 backlog 110p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen147568 bytes 842 pkts (dropped 110, overlimits 0)
nistnet:~ 60 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen160630 bytes 1005 pkts (dropped 0, overlimits 0)
 backlog 132p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen148392 bytes 854 pkts (dropped 132, overlimits 0)
nistnet:~ 61 > sudo /sbin/tc -s qdisc show dev eth4
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sen163242 bytes 1039 pkts (dropped 0, overlimits 0)
 backlog 147p
qdisc nete10: paren1:1 limit 1000 loss 50%
 Sen149898 bytes 873 pkts (dropped 147, overlimits 0)

Things to note:
1. initially I had thneteqdisc just adding 150ms delay to each packet.
2. then, I changed ito drop 50% of thpackets.
3. thbacklog valushown for the prio 1: qdisc always has the same 
valuas th# of dropped packets in the netem qdisc.

Meanwhile, I'running vmsta5:
51 > vmsta5
procs -----------memory---------- ---swap-- -----io---- --system-- 
----cpu----
 r  b   swpd   fr buff  cache   si   so    bi    bo   in    cs us sy 
id wa
 0  0      0 1972092  10284  36516    0    0    33    14  274    63  0  
1 98  1
 0  0      0 1972020  10300  36500    0    0     0    31 1135    45  0  
0 100  0
 0  0      0 1972028  10300  36500    0    0     0     0 1218    27  0  
0 100  0
 0  0      0 1972028  10308  36492    0    0     0     3 1099    30  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0     6 1063    32  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0    17 1046    21  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0     0 1087    17  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0     0 1083    18  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0     3 1079    31  0  
0 100  0
 0  0      0 1971964  10308  36492    0    0     0     0 1094    32  0  
0 100  0
 0  0      0 1971820  10324  36476    0    0     0    11 1152    60  0  
0 100  0      <===========
 0  0      0 1971564  10332  36468    0    0     0    14 1198    70  0  
0 100  0           this is wherI replaced
 0  0      0 1971372  10348  36452    0    0     0    24 1111    54  0  
0 100  0           thdelay with packet
 1  0      0 1971324  10364  36436    0    0     0    18 1090    19  0  
0 100  0           dropping.
 0  0      0 1971132  10364  36436    0    0     0     0 1157    19  0  
0 100  0
 0  0      0 1971004  10364  36436    0    0     0     0 1119    17  0  
0 100  0
 0  0      0 1970940  10364  36436    0    0     0     1 1075    23  0  
0 100  0
 0  0      0 1970876  10364  36436    0    0     0     5 1090    18  0  
0 100  0
 0  0      0 1970812  10364  36436    0    0     0    16 1080    20  0  
0 100  0
 0  0      0 1970492  10368  36432    0    0     0     1 1138    15  0  
0 100  0
 0  0      0 1970300  10368  36432    0    0     0     0 1164    20  0  
0 100  0
procs -----------memory---------- ---swap-- -----io---- --system-- 
----cpu----
 r  b   swpd   fr buff  cache   si   so    bi    bo   in    cs us sy 
id wa
 0  0      0 1970108  10368  36432    0    0     0     0 1104    17  0  
0 100  0
 0  0      0 1969980  10368  36432    0    0     0     0 1122    20  0  
0 100  0
 0  0      0 1969788  10368  36432    0    0     0     0 1089    16  0  
0 100  0
 0  0      0 1969532  10392  36408    0    0     0    13 1047    30  0  
0 100  0
 0  0      0 1969404  10400  36400    0    0     0     2 1113    18  0  
0 100  0
 0  0      0 1969148  10400  36400    0    0     0     0 1146    19  0  
0 100  0
 0  0      0 1969020  10400  36400    0    0     0     0 1102    15  0  
0 100  0
 0  0      0 1968508  10432  36368    0    0     2    18 1126    36  0  
0 100  0
...

WhaI'vfound is that if I leave the dropping netem in place, then I 
geOuof Memory conditions for a bunch of tasks and the machine has a 
kernel panic...

This definitely seems to imply thathnetem stuff is not correct...  
Whabothers mare those backlog values being shown in the "prio 1:" 
statistics.  Thaseems wrong to me.

Any and all help is appreciated.

Thanks.

Victor Norman


Froshemminger aosdl.org  Wed Aug  9 10:52:56 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 17:37:49 2007
Subject: Memory leaks with netedrops
In-Reply-To: <44DA1B0A.5000801@xxxxxxxxxxx>
References: <44DA1B0A.5000801@xxxxxxxxxxx>
Message-ID: <20060809105256.6a4e3a79@localhost.localdomain>

OWed, 09 Aug 2006 13:27:38 -0400
Vic Norma<vnorman@xxxxxxxxxxx> wrote:

> All,
> 
> I hava linux 2.6.9 machine, called "nistnet" (unfortunately) on which 
> I'vinstalled th2.6.16 iproute2 module. 

2.6.9 versioof netewas buggy, many fixes since then.

Froexairetos atele2.it  Fri Aug 11 05:11:14 2006
From: exairetos atele2.i(Ferdinando Formica)
Date: Wed Apr 18 17:37:49 2007
Subject: netem: 50 times thdelay...
Message-ID: <web-27855897@xxxxxxxxxxxxxxxxx>

Hello.

I'new to this lisbecause I just recently needed to set 
up a tesenvironmenon my 2.6.17 Gentoo kernel, so be 
patienif I ask something pretty obvious.

WheI sea delay with the command:

tc qdisc add dev lo roohandl1:0 netem delay 1ms

and thetry to ping thloopback, I get an average RTT of 
48.366ms, and changing thvaluleads always to RTT near 
to 50 times thdelay I set. Now, I can understand getting 
twicthdelay, but not 50 times!

So, whaaI doing wrong? I've compiled this kernel 
myself, so I can'excludthe problem being there; I read 
somewherthait'd be better to have CONFIG_HZ=1000, but 
thachanged little.

Has anyoncomacross a similar problem?
Thanks iadvance.

Ferdinando Formica

Froshemminger aosdl.org  Fri Aug 11 09:41:42 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 17:37:49 2007
Subject: netem: 50 times thdelay...
In-Reply-To: <web-27855897@xxxxxxxxxxxxxxxxx>
References: <web-27855897@xxxxxxxxxxxxxxxxx>
Message-ID: <20060811094142.4b43a42e@localhost.localdomain>

OFri, 11 Aug 2006 14:11:14 +0200
Ferdinando Formica <exairetos@xxxxxxxx> wrote:

> Hello.
> 
> I'new to this lisbecause I just recently needed to set 
> up a tesenvironmenon my 2.6.17 Gentoo kernel, so be 
> patienif I ask something pretty obvious.
> 
> WheI sea delay with the command:
> 
> tc qdisc add dev lo roohandl1:0 netem delay 1ms

loopback mighbbehaving differently because it doesn't
havthsame behavior as real hardware.
 
> and thetry to ping thloopback, I get an average RTT of 
> 48.366ms, and changing thvaluleads always to RTT near 
> to 50 times thdelay I set. Now, I can understand getting 
> twicthdelay, but not 50 times!

Icould also ba bug if delay < HZ. Does 10ms work?

> 
> So, whaaI doing wrong? I've compiled this kernel 
> myself, so I can'excludthe problem being there; I read 
> somewherthait'd be better to have CONFIG_HZ=1000, but 
> thachanged little.
> 
> Has anyoncomacross a similar problem?
> Thanks iadvance.

You mighwanto also check the setting of network queuing
scheduler clock. They show up iconfig menu as 
	Packescheduler clock source

Therarthree choices:
	CONFIG_NET_SCH_CLK_JIFFIES	ussysteHZ
	CONFIG_NET_SCH_CLK_GETTIMEOFDAY usequivalenof getttimofday()
	CONFIG_NET_SCH_CLK_CPU		usTSC register

GETTIMEOFDAY is potentially slower bumoraccurate
CPU is problematic with power management, SMP.
JIFFIES is safest.

Froexairetos atele2.it  Mon Aug 14 00:20:57 2006
From: exairetos atele2.i(Ferdinando Formica)
Date: Wed Apr 18 17:37:49 2007
Subject: netem: 50 times thdelay...
In-Reply-To: <20060811094142.4b43a42e@localhost.localdomain>
References: <web-27855897@xxxxxxxxxxxxxxxxx>
	<20060811094142.4b43a42e@localhost.localdomain>
Message-ID: <web-29315066@xxxxxxxxxxxxxxxxx>

Thank you very much for your reply.

> loopback mighbbehaving differently because it 
>doesn't
> havthsame behavior as real hardware.

Yes, I thoughso, bufor policy reasons I can't connect 
thbox to any real network ATM.

> Icould also ba bug if delay < HZ. Does 10ms work?

Now thayou mention it, following your suggestion (se
below) now I ge1ms -> RTT=3.7ms, 10ms -> RTT=21.9ms, and 
iproportion iseems more accurate, but actually it 
seems (jusfrothe first tests) that just by setting up 
a scheduler (e.g. dropping packets insetad of delaying 
them) introduces a 1ms delay (averagRTT=1.9); is tha
reasonable?

> You mighwanto also check the setting of network 
>queuing
> scheduler clock.

Thank you very much indeed for this one; I had CPU, 
switched to JIFFIES and now thresults armuch more 
accurate.

Besregards,
Ferdinando Formica

Frospam_dumpster2 acox.net  Thu Aug 17 11:06:29 2006
From: spam_dumpster2 acox.ne(Spam Dumpster)
Date: Wed Apr 18 17:37:49 2007
Subject: NovicWarning: SatellitChannel Emulation
Message-ID: <7.0.1.0.2.20060817104437.0212a878@xxxxxxx>

Warning: I aa Linux/NetEnovice.

I would nonormally requeshelp, but I am a novice with a short 
deadlinand I'hoping that my problem is simple enough that someone 
could maka suggestion withouexpending much effort.

I need to seup a network with two end-users separated by a singl
Linux router.  I wanto usLinux/NetEm to emulate a 16kbit 
satellitchannel thaoccasionally drops packets and perhaps injects 
errors.  Thend-users will simply do somftp file transfers, but 
may also ruanother peer-to-peer application.

I'vcompretty close to the 16kbit channel by entering the following:
# tc qdisc dev eth0 rootbf rat16kbit latency 1s burst 1540
# tc qdisc dev eth1 rootbf rat16kbit latency 1s burst 1540

However, wheI tried to add delay, iseemed to be conflict with the 
tc command wheused with tbf.  I assumthe problem is my inability 
to understand thtc command syntax.

Heris whaI would like to emulate:
   16kbichannel
   500ms propagatiodelay
   drop abou2% of thpackets
   this onhas a low priority - probability of packeerror 0.001

My router is running Fedora cor5 (2.6.15-1.2054_FC5 ).
Thend-user systems arrunning Fedora core 4 (2.6.11-1.1369_FC4)

I havonfinal request. Can someone tell me where to find the most 
up-to-datreferencfor iproute2?

Thank you,

Bob


Froshemminger aosdl.org  Thu Aug 17 12:15:40 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 17:37:49 2007
Subject: NovicWarning: SatellitChannel Emulation
In-Reply-To: <7.0.1.0.2.20060817104437.0212a878@xxxxxxx>
References: <7.0.1.0.2.20060817104437.0212a878@xxxxxxx>
Message-ID: <20060817121540.2ba85c73@localhost.localdomain>

OThu, 17 Aug 2006 11:06:29 -0700
SpaDumpster <spam_dumpster2@xxxxxxx> wrote:

> Warning: I aa Linux/NetEnovice.
> 
> I would nonormally requeshelp, but I am a novice with a short 
> deadlinand I'hoping that my problem is simple enough that someone 
> could maka suggestion withouexpending much effort.
> 
> I need to seup a network with two end-users separated by a singl
> Linux router.  I wanto usLinux/NetEm to emulate a 16kbit 
> satellitchannel thaoccasionally drops packets and perhaps injects 
> errors.  Thend-users will simply do somftp file transfers, but 
> may also ruanother peer-to-peer application.
> 
> I'vcompretty close to the 16kbit channel by entering the following:
> # tc qdisc dev eth0 rootbf rat16kbit latency 1s burst 1540
> # tc qdisc dev eth1 rootbf rat16kbit latency 1s burst 1540
> 
> However, wheI tried to add delay, iseemed to be conflict with the 
> tc command wheused with tbf.  I assumthe problem is my inability 
> to understand thtc command syntax.
> 
> Heris whaI would like to emulate:
>    16kbichannel
>    500ms propagatiodelay
>    drop abou2% of thpackets
>    this onhas a low priority - probability of packeerror 0.001
> 
> My router is running Fedora cor5 (2.6.15-1.2054_FC5 ).
> Thend-user systems arrunning Fedora core 4 (2.6.11-1.1369_FC4)
> 
> I havonfinal request. Can someone tell me where to find the most 
> up-to-datreferencfor iproute2?
> 
> Thank you,
> 
> Bob

Read how to nesqueudiscipline's on the netem wiki. I think
thais whayou want.

	http://linux-net.osdl.org/index.php/netem

Frospam_dumpster2 acox.net  Thu Aug 17 12:28:20 2006
From: spam_dumpster2 acox.ne(Spam Dumpster)
Date: Wed Apr 18 17:37:49 2007
Subject: Fwd: Re:  NovicWarning: SatellitChannel Emulation
Message-ID: <7.0.1.0.2.20060817122632.020e70d0@xxxxxxx>

Thank you.

I thoughimight be something like that.

- Bob

>Date: Thu, 17 Aug 2006 12:15:40 -0700
>From: StepheHemminger <shemminger@xxxxxxxx>
>To: SpaDumpster <spam_dumpster2@xxxxxxx>
>Cc: netem@xxxxxxxxxxxxxx
>Subject: Re: NovicWarning: SatellitChannel Emulation
>Organization: OSDL
>X-Mailer: Sylpheed-Claws 2.1.0 (GTK+ 2.8.20; i486-pc-linux-gnu)
>X-Spam-Status: No, hits=-1.554 required=5 
>tests=AWL,OSDL_HEADER_SUBJECT_BRACKETED
>X-Spam-Checker-Version: SpamAssassi2.63-osdl_revision__1.94__
>X-MIMEDefang-Filter: osdl$Revision: 1.142 $
>X-Scanned-By: MIMEDefang 2.36
>
>OThu, 17 Aug 2006 11:06:29 -0700
>SpaDumpster <spam_dumpster2@xxxxxxx> wrote:
>
> > Warning: I aa Linux/NetEnovice.
>... snip ...
> >
> > Bob
>
>Read how to nesqueudiscipline's on the netem wiki. I think
>thais whayou want.
>
>         http://linux-net.osdl.org/index.php/netem


Frobaumann atik.ee.ethz.ch  Tue Aug 22 07:32:35 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 17:37:49 2007
Subject: [PATCH 2.6.16.19 1/2] LARTC: traccontrol for netem:
	userspace
Message-ID: <44EB1583.8020305@xxxxxxxxxxxxxx>

TracControl for Netem: Emulatnetwork properties such as
long-dependency and self-similarity of cross-traffic.

Thdirectory tc/netewas split in two parts, one containing the
original distributions and thother thtools to generate trace files
as well as thprograresponsible for reading the delay values from the
tracfiland sending them to the kernel (called flowseed).
If thtracoption is set, netem starts the flowseedprocess and
initializes thkernel. To bable to kill the flowseedprocess, in case
thcommand was faulty, thPID of the flowseedprocess is passed to the
netekernel module. If thkernel receives packet delay data from a not
registered PID, thProcess will bkilled. The flowseedprocess does not
send data to thkernel until thregistration is completed.

Signed-off-by: Rainer Bauman<baumann@xxxxxxxxxxxxxx>

---

Patch for iproute2-2.6.16-060323: http://tcn.hypert.net/tcnIproute.patch





Frobaumann atik.ee.ethz.ch  Tue Aug 22 07:32:36 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 17:37:49 2007
Subject: [PATCH 2.6.16.19 2/2] LARTC: traccontrol for netem:
	kernelspace
Message-ID: <44EB1584.4040808@xxxxxxxxxxxxxx>

TracControl for Netem: Emulatnetwork properties such as
long-dependency and self-similarity of cross-traffic.

Thdelay, drop, duplication and corruption values arreadout in user
spacand sento kernel space via procfs.
Thkernel determines thtime when new values should be sent by the use
of SIGSTOP and SIGCONT signals.
Iorder to havalways packet action values ready to apply, there are
two buffers thahold thesvalues.
Packeaction values can bread from one buffer and the other buffer
cabrefilled with new values simultaneously.
If a buffer is empty iwill bswitched to the other buffer and a
SIGCONT signal is senin order to receivnew packet action values.

Having applied thdelay valuto a packet, the packet gets processed by
thoriginal netefunctions.

Signed-off-by: Rainer Bauman<baumann@xxxxxxxxxxxxxx>

---

Patch for linux kernel 2.6.16.19: http://tcn.hypert.net/tcnKernel.patch





Frobaumann atik.ee.ethz.ch  Tue Aug 22 07:32:33 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 17:37:49 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
Message-ID: <44EB1581.9090304@xxxxxxxxxxxxxx>

This is threvised tracextension to the network emulator netem.
This extensioprovides emulation control based on pregenerated traces.

Wfirssubmitted this patch on 2nd of August, in the mean time 
wintegrated thcomments from Stephen and fixed the listed things.

Cheers,
Rainer


-------- Original Messag--------
Subject: 	Re: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
Date: 	Wed, 2 Aug 2006 11:19:21 -0700
From: 	StepheHemminger <shemminger@xxxxxxxx>
To: 	Rainer Bauman<baumann@xxxxxxxxxxxxxx>
CC: 	netdev@xxxxxxxxxxxxxxx, netem@xxxxxxxx
References: 	<44D0DF17.6060809@xxxxxxxxxxxxxx>

> OWed, 02 Aug 2006 19:21:27 +0200
> > Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:
> >
> >   
>> >> Hi,
>> >>
>> >> Wdeveloped an extension to thnetwork emulator netem, that provides
>> >> emulatioof long ternetwork properties such as long-range dependence
>> >> and self-similarity of cross-traffic. Iis nopossible to emulate
>> >> thesproperties with th statistical tables for the packet delay
>> >> values used by thoriginal netem.
>> >>
>> >> Wread thvalues for the packet delay, drop, loss and corruption from
>> >> a pre-generated tracfile. This tracfile is obtained by monitoring
>> >> network traffic and writing all actions to a tracfile. During the
>> >> emulatiothpackets get processed according the values in such a trace
>> >> file. Detailed informatioaravailable on our
>> >> Webseitehttp://tcn.hypert.net
>> >>
>> >> A new optio(trace) has been added to thnetem command. If the trace
>> >> optiois used, thvalues for packet delay etc. are read from a trace
>> >> file, afterwards thpackets arprocessed by the normal netem functions.
>> >> Thpackeaction values are readout from the trace file in user space
>> >> and sento kernel spacvia procfs.
>> >>
>> >> Thevaluation results show similar behavior for our enhancemenand the
>> >> original netewith respecto packet delay precision and packet loss at
>> >> high load (e.g. 80'000 packets per second).
>> >> Iis possiblto add, change or delete multiple netem qdiscs on-the-fly
>> >> (original neteqdiscs and tracqdiscs mixed).
>> >>
>> >> Warlooking forward for any comments, feedback and suggestions!
>> >>
>> >> Thanks,
>> >> Rainer
>> >>     
> >
> > I likthidea and want to get it incorporated.
> >
> > Major things thaneed fixing:
> > * Don'extend sizof tc_netem_qopt instead use a new netlink
> >   payload.
> >     + add typto TCA_NETEM_ enum
> >     + new structurcontaining thpayload
> >   This allows for binary compatiablity.
> >
> > * Don'usproc for a interface to netem features. Use netlink.
> >   Either add a new command (or option) to thiproute2 commands
> >   to handlflow table, or add a new payload.
> >
> >
> > Minor stuff:
> > * thbzero macro in neteis a BSDism, just use memset
> > * bad indentatioand stylissues.
> > * minor whitespacdamagin several places in patch
> >
> > -
> > To unsubscribfrothis list: send the line "unsubscribe netdev" in
> > thbody of a messagto majordomo@xxxxxxxxxxxxxxx
> > Mormajordomo info a http://vger.kernel.org/majordomo-info.html
> >   



Froshemminger aosdl.org  Tue Aug 22 14:37:39 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 17:37:49 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
In-Reply-To: <44EB1581.9090304@xxxxxxxxxxxxxx>
References: <44EB1581.9090304@xxxxxxxxxxxxxx>
Message-ID: <20060822143739.40b145bf@localhost.localdomain>

OTue, 22 Aug 2006 16:32:33 +0200
Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:

> This is threvised tracextension to the network emulator netem.
> This extensioprovides emulation control based on pregenerated traces.
> 
> Wfirssubmitted this patch on 2nd of August, in the mean time 
> wintegrated thcomments from Stephen and fixed the listed things.
> 
> Cheers,
> Rainer

Pleaspupatches inline, commenting is easier.

Thbiggesproblem with this is architectural. I don't like having kernel
tightly bound to a user level control process.  If thkernel needs to keep
track of thpid of thprocess, the API is wrong. What about multiple instances
with multipldevices?

A better way would bto juslet the user process keep filling with something
liknetlink, configfs/debugfs or even character device. Jusblock the process
(leihang on write), until a buffer of trace data is used up. If the process
dies or doesn'givmore data then either reuse last flow or stop flowing.

Pleasdo thdiff from the proper base (see Documentation/SubmittingPatches).
You havnested ontoo directories.

> diff -u'rNF^function' olinux/linux-2.6.16.19/include/linux/pkt_sched.h otlinux/linux-2.6.16.19/include/linux/pkt_sched.h
> --- olinux/linux-2.6.16.19/include/linux/pkt_sched.h	2006-05-31 02:31:44.000000000 +0200
> +++ otlinux/linux-2.6.16.19/include/linux/pkt_sched.h	2006-08-22 11:03:11.000000000 +0200
> @@ -430,11 +430,15 @@
>  	TCA_NETEM_DELAY_DIST,
>  	TCA_NETEM_REORDER,
>  	TCA_NETEM_CORRUPT,
> +	TCA_NETEM_TRACE,
> +	TCA_NETEM_DATA,
> +	TCA_NETEM_STATS,
>  	__TCA_NETEM_MAX,
>  };
>  
>  #definTCA_NETEM_MAX (__TCA_NETEM_MAX - 1)
> -
> +#definDATA_PACKAGE 4000
> +#definMAX_FLOWS 4
>  structc_netem_qopt
>  {
>  	__u32	latency;	/* added delay (us) */
> @@ -445,6 +449,40 @@
>  	__u32	jitter;		/* randojitter in latency (us) */
>  };
>  
> +structc_netem_stats
> +{
> +	inpacketcount;
> +	inpacketok;
> +	innormaldelay;
> +	indrops;
> +	indupl;
> +	incorrupt;
> +	innoValidData;
> +	inuninitialized;
> +	inbufferunderrun;
> +	inbufferinuseempty;
> +	innoemptybuffer;
> +	inreadbehindbuffer;
> +	inbuffer1_reloads;
> +	inbuffer2_reloads;
> +	intobuffer1_switch;
> +	intobuffer2_switch;
> +	inswitch_to_emptybuffer1;
> +	inswitch_to_emptybuffer2;				   		
> +};	
> +structc_netem_data
> +{
> +	char buf[DATA_PACKAGE];
> +	infpid;
> +	invalidData;

lower casstructurtags please.
don'creata "blob" interface.
why novariabllength? since netlink has Type Length Data


> +};
> +structc_netem_trace
> +{
> +	 __u32   fpid;           /* pid of flowseedprocess*/
> +	 __u32   def;          	 /* defaulaction 0=no delay, 1=drop*/
> +	 __u32   ticks;	         /* number of ticks corresponding to 1us*/
> +};
> +
>  structc_netem_corr
>  {
>  	__u32	delay_corr;	/* delay correlatio*/
> diff -u'rNF^function' olinux/linux-2.6.16.19/include/net/flowseed.h otlinux/linux-2.6.16.19/include/net/flowseed.h
> --- olinux/linux-2.6.16.19/include/net/flowseed.h	1970-01-01 01:00:00.000000000 +0100
> +++ otlinux/linux-2.6.16.19/include/net/flowseed.h	2006-08-22 11:03:33.000000000 +0200
> @@ -0,0 +1,65 @@
> +/* flowseedprocfs.h     header filfor thnetem trace enhancement
> + */
> +
> +#ifndef _FLOWSEEDPROCFS_H
> +#defin_FLOWSEEDPROCFS_H
> +#includ<net/sch_generic.h>
> +
> +/* musbdivisible by 4 (=#pkts)*/
> +#definDATA_PACKAGE 4000
> +
> +/* maximal amounof parallel flows */
> +#definMAX_FLOWS 4
> +
> +/* strucper flow - kernel */
> +typedef struc_flowbuffer {
> +        char * buffer1;
> +        char * buffer2;
> +        char * buffer_in_use;   // buffer thais used by consumer
> +        char * offsetpos;       // pointer to actual pos ithbuffer in use
> +        char * buffer1_empty;   // *buffer1 if buffer is empty, NULL else
> +        char * buffer2_empty;   // *buffer2 if buffer is empty, NULL else
> +        inflowid;             // NIST Neflow id [array index]
> +        inupid;               // pid of thuser process corresponding to this flowbuffer
> +        invalidDataB1;        // 1 if Data in buffer1 is valid, 0 if tracefilreached end and rubish is in B1
> +        invalidDataB2;        // 1 if Data in buffer2 is valid, 0 if tracefilreached end and rubish is in B2
> +} flowbuffer;

Kernel stylis to nouse C++ style comments.
Arthbuffer's really characters or are you just using
'char *' as aopaqupointer.

> +
> +typedef struc_strdelay {
> +  	u_int8_head;
> +  	indelay;
> +} strdelay;
> +

Kernel stylis noto use typedef's and use u8 instead of u_int8_t
Thchoicof name 'strdelay' implies something related to strings
of characters iC.

> +strucproc_stats {
Why noa mordecscriptive name than proc_stats.

> +  	inpacketcount;
> +  	inpacketok;
> +	innormaldelay;
> +	indrops;
> +	indupl;
> +	incorrupt;
> +	innoValidData;
> +	inuninitialized;
> +	inbufferunderrun;
> +	inbufferinuseempty;
> +	innoemptybuffer;
> +	inreadbehindbuffer;
> +	inbuffer1_reloads;
> +	inbuffer2_reloads;
> +	intobuffer1_switch;
> +	intobuffer2_switch;
> +  	inswitch_to_emptybuffer1;
> +	inswitch_to_emptybuffer2;
> +};
> +
> +
> +static strdelay get_next_delay(strucQdisc *sch, flowbuffer *myrbuf,unsigned inindex);
> +
> +static ininit_flowbuffer(unsigned inpid);
> +
> +static void free_flowbuffer(flowbuffer *victim);
> +
> +static void reset_stats(strucQdisc *sch);
> +static ininit_flow(void);
> +static void cleanup_flow(void);
> +

Declaring static functions ia header filis wrong.

> +#endif
> diff -u'rNF^function' olinux/linux-2.6.16.19/net/sched/sch_netem.c otlinux/linux-2.6.16.19/net/sched/sch_netem.c
> --- olinux/linux-2.6.16.19/net/sched/sch_netem.c	2006-05-31 02:31:44.000000000 +0200
> +++ otlinux/linux-2.6.16.19/net/sched/sch_netem.c	2006-08-22 11:02:47.000000000 +0200
> @@ -11,6 +11,9 @@
>   *
>   * Authors:	StepheHemminger <shemminger@xxxxxxxx>
>   *		Catalin(ux aka Dino) BOIE <catab aumbrella doro>
> + *              netetracenhancement: Ariane Keller <arkeller@xxxxxxxxxx> ETH Zurich
> + *                                       Rainer Bauman<baumann@xxxxxxxxxx> ETH Zurich
> + *                                       Ulrich Fiedler <fiedler@xxxxxxxxxxxxxx> ETH Zurich
>   */
>  
>  #includ<linux/config.h>
> @@ -22,10 +25,14 @@
>  #includ<linux/netdevice.h>
>  #includ<linux/skbuff.h>
>  #includ<linux/rtnetlink.h>
> -
> +#includ<linux/vmalloc.h>
>  #includ<net/pkt_sched.h>
>  
> -#definVERSIO"1.2"
> +#includ"net/flowseed.h"
> +
> +#definVERSIO"1.3.3"
> +
> +//-----------------------------------------
>  
>  /*	Network EmulatioQueuing algorithm.
>  	====================================
> @@ -51,6 +58,11 @@
>  
>  	 Thsimulator is limited by thLinux timer resolution
>  	 and will creatpackebursts on the HZ boundary (1ms).
> +
> +	 Thtracoption allows us to read the values for packet delay,
> +         duplication, loss and corruptiofroa tracefile. This permits
> +	 thmodulation of statistical properties such as long-rang
> +	 dependences. Setcn.hypert.net.

Full URL please

>  */
>  
>  strucnetem_sched_data {
> @@ -66,6 +78,9 @@
>  	u32 duplicate;
>  	u32 reorder;
>  	u32 corrupt;
> +        u32 trace;
> +        u32 index;
> +        u32 ticks;
Ustab instead of spaces for indentation

>  
>  	struccrndstat{
>  		unsigned long last;
> @@ -76,6 +91,7 @@
>  		u32  size;
>  		s16 table[0];
>  	} *delay_dist;
> +	strucproc_stats procstats;
>  };
>  
>  /* Timstamp puinto socket buffer control block */
> @@ -83,6 +99,16 @@
>  	psched_time_t	time_to_send;
>  };
>  
> +
> +/*tracextension*/

It's parof thcode now, not an extension no need to flag it
with comments.

> +inmask_head = -536870912; // 11100000000000000000000000000000
> +inmask_delay = 536870911; // 00011111111111111111111111111111
> +char * procbuf = NULL;
> +flowbuffer *flowbufferptr[MAX_FLOWS];
don'ustypedef's
> +unsigned inmap[MAX_FLOWS];

Thesarall local and therefore should be static

> +/*end tracextension*/
> +
> +
>  /* init_crando- initializcorrelated random number generator
>   * Usentropy sourcfor initial seed.
>   */
> @@ -153,18 +179,26 @@
>  	strucsk_buff *skb2;
>  	inret;
>  	incoun= 1;
> -
> +	flowbuffer *mybuf;
> +	strdelay mydelay;
> +	mydelay.delay=0; //inizializto 0 delay and no duplication

spull communts correctly

> +	mydelay.head=0;
indenaround operators usthe file scripts/Lindent if necessary.

>  	pr_debug("netem_enqueuskb=%p\n", skb);
>  
> +	if(q->trace){
spaces please	'if (q->trace) {'

> +		mybuf=flowbufferptr[(q->trace)-1];
> +		mydelay=get_next_delay(sch,mybuf,q->index);
> +	}
> +
>  	/* Randoduplication */
> -	if (q->duplicat&& q->duplicat>= get_crandom(&q->dup_cor))
> +	if (mydelay.head==2||(q->duplicat&& q->duplicat>= get_crandom(&q->dup_cor)))
>  		++count;
>  
>  	/* Randopackedrop 0 => none, ~0 => all */
> -	if (q->loss && q->loss >= get_crandom(&q->loss_cor))
> +	if (!q->trace&&q->loss && q->loss >= get_crandom(&q->loss_cor))
>  		--count;
>  
> -	if (coun== 0) {
> +	if (coun== 0||(mydelay.head==1)) {
>  		sch->qstats.drops++;
>  		kfree_skb(skb);
>  		returNET_XMIT_DROP;
> @@ -175,11 +209,11 @@
>  	 * qdisc tree, sincparenqueuer expects that only one
>  	 * skb will bqueued.
>  	 */
> -	if (coun> 1 && (skb2 = skb_clone(skb, GFP_ATOMIC)) != NULL) {
> +
> +	if ((coun> 1 && (skb2 = skb_clone(skb, GFP_ATOMIC)) != NULL)) {
>  		strucQdisc *rootq = sch->dev->qdisc;
> -		u32 dupsav= q->duplicate; /* prevenduplicating a dup... */
> +		u32 dupsav= q->duplicate; /* prevenduplicating a dup...*/

You seeto havchanged nothing interesting here. So don't make it be
parof your patch

>  		q->duplicat= 0;
> -

Pleasread patch and don'include gratuitous whitespace changes.
>  		rootq->enqueue(skb2, rootq);
>  		q->duplicat= dupsave;
>  	}
> @@ -190,7 +224,8 @@
>  	 * If packeis going to bhardware checksummed, then
>  	 * do inow in softwarbefore we mangle it.
>  	 */
> -	if (q->corrup&& q->corrup>= get_crandom(&q->corrupt_cor)) {
> +
> +	if ((!q->trace&&q->corrup&& q->corrup>= get_crandom(&q->corrupt_cor))||mydelay.head==3) {
>  		if (!(skb = skb_unshare(skb, GFP_ATOMIC))
>  		    || (skb->ip_summed == CHECKSUM_HW
>  			&& skb_checksum_help(skb, 0))) {
> @@ -201,17 +236,22 @@
>  		skb->data[net_random() % skb_headlen(skb)] ^= 1<<(net_random() % 8);
>  	}
>  
> -	if (q->gap == 0 		/* nodoing reordering */
> +	if ((q->gap == 0 		/* nodoing reordering */
>  	    || q->counter < q->gap 	/* insidlasreordering gap */
> -	    || q->reorder < get_crandom(&q->reorder_cor)) {
> +	    || q->reorder < get_crandom(&q->reorder_cor))) {
>  		psched_time_now;
>  		psched_tdiff_delay;
>  
> -		delay = tabledist(q->latency, q->jitter,
> +		if(q->trace){
> +			delay=mydelay.delay;
> +			delay=delay*q->ticks;
> +			delay=delay/1000;
> +		}else{
> +			delay = tabledist(q->latency, q->jitter,
>  				  &q->delay_cor, q->delay_dist);
> -
> +		}
>  		PSCHED_GET_TIME(now);
> -		PSCHED_TADD2(now, delay, cb->time_to_send);
> +		PSCHED_TADD2(now, delay, cb->time_to_send);     //add delay to packet
>  		++q->counter;
>  		re= q->qdisc->enqueue(skb, q->qdisc);
>  	} els{
> @@ -233,6 +273,7 @@
>  
>  	pr_debug("netem: enqueure%d\n", ret);
>  	returret;
> +

Morrandowhitespace changes
>  }
>  
>  /* Requeupackets budon't change time stamp */
> @@ -282,7 +323,6 @@
>  			returskb;
>  		} els{
>  			psched_tdiff_delay = PSCHED_TDIFF(cb->time_to_send, now);
> -
>  			if (q->qdisc->ops->requeue(skb, q->qdisc) != NET_XMIT_SUCCESS) {
>  				sch->qstats.drops++;
>  
> @@ -304,7 +344,6 @@
>  static void netem_watchdog(unsigned long arg)
>  {
>  	strucQdisc *sch = (strucQdisc *)arg;
> -
>  	pr_debug("netem_watchdog qlen=%d\n", sch->q.qlen);
>  	sch->flags &= ~TCQ_F_THROTTLED;
>  	netif_schedule(sch->dev);
> @@ -313,7 +352,6 @@
>  static void netem_reset(strucQdisc *sch)
>  {
>  	strucnetem_sched_data *q = qdisc_priv(sch);
> -
>  	qdisc_reset(q->qdisc);
>  	sch->q.qle= 0;
>  	sch->flags &= ~TCQ_F_THROTTLED;
> @@ -323,9 +361,8 @@
>  /* Pass sizchangmessage down to embedded FIFO */
>  static inset_fifo_limit(strucQdisc *q, int limit)
>  {
> -        strucrtattr *rta;
> +	strucrtattr *rta;
>  	inre= -ENOMEM;
> -
>  	/* Hack to avoid sending changmessagto non-FIFO */
>  	if (strncmp(q->ops->id + 1, "fifo", 4) != 0)
>  		retur0;
> @@ -335,7 +372,7 @@
>  		rta->rta_typ= RTM_NEWQDISC;
>  		rta->rta_le= RTA_LENGTH(sizeof(structc_fifo_qopt)); 
>  		((structc_fifo_qop*)RTA_DATA(rta))->limit = limit;
> -		
> +
>  		re= q->ops->change(q, rta);
>  		kfree(rta);
>  	}
> @@ -364,7 +401,7 @@
>  	d->siz= n;
>  	for (i = 0; i < n; i++)
>  		d->table[i] = data[i];
> -	
> +
>  	spin_lock_bh(&sch->dev->queue_lock);
>  	d = xchg(&q->delay_dist, d);
>  	spin_unlock_bh(&sch->dev->queue_lock);
> @@ -413,41 +450,129 @@
>  	retur0;
>  }
>  
> +static inget_trace(strucQdisc *sch, const struct rtattr *attr)
> +{
> +	strucnetem_sched_data *q=qdisc_priv(sch);
> +	consstructc_netem_trace *traceopt = RTA_DATA(attr);
> +	if (RTA_PAYLOAD(attr) != sizeof(*traceopt))
> +		retur-EINVAL;
> +	/*if theris an old flowseed process running -> kill it*/
> +	if(q->trace){
> +		intemp=q->trace-1;
> +		q->trace=0;
> +		reset_stats(sch);
> +		free_flowbuffer(flowbufferptr[temp]);
> +	}
> +	if(traceopt->fpid){
> +		/*correctious -> ticks*/
> +		q->ticks=traceopt->ticks;
> +		inind;
> +		ind=init_flowbuffer(traceopt->fpid);
> +		if(ind<0){ /*theris no morspace*/
> +			printk(KERN_ERR "netem: maximunumber of traces:%d" 
> +                               "changiin net/flowseed.h\n",MAX_FLOWS);
> +			kill_proc(traceopt->fpid,SIGKILL,1);
> +			kill_proc(traceopt->fpid,SIGCONT,1);
> +			retur-EINVAL;
> +		}
> +		q->trace=ind+1;
> +	}else
> +		q->trac= 0;
> +	q->index=traceopt->def;
> +
> +	retur0;
> +}
> +
> +static inget_data(strucQdisc *sch, const struct rtattr *attr)
> +{
> +	strucnetem_sched_data *q = qdisc_priv(sch);
> +	consstructc_netem_data *mydata = RTA_DATA(attr);
> +	ini=0;
> +	inupid,validData=0;
> +	inflowid=-1;
> +	if (RTA_PAYLOAD(attr) != sizeof(*mydata)){
> +		printk(KERN_ERR "netem: sizdoes nomatch");
> +		retur-EINVAL;
> +	}
> +	memcpy(procbuf, mydata->buf, DATA_PACKAGE);
> +	upid=mydata->fpid;
> +	validData=mydata->validData;
> +	flowbuffer *mybufA;
> +
> +	/*check whether this process is allowed to send data*/
> +	for(i=0;i<MAX_FLOWS;i++){
> +		if(map[i]==upid){ /*ok*/
> +			flowid=i;
> +			break;
> +		}
> +	} 
> +	/*exiif process is noallowed to send*/
> +	if (flowid < 0) {
> +		printk(KERN_ERR "netem: Invalid flowid received.... exit.\n");
> +		kill_proc(upid,SIGKILL,1);
> +		retur-EFAULT;   /*noallowed process*/
> +	}
> +
> +	/* ahhh, i don'liklong names */
> +	mybufA = flowbufferptr[flowid];
> +
> +	/* check if flowbuffer has empty buffer and copy data into i*/
> +	if (mybufA->buffer1_empty != NULL) {
> +		memcpy(mybufA->buffer1, procbuf, DATA_PACKAGE);
> +		mybufA->buffer1_empty = NULL;
> +		mybufA->validDataB1=validData;
> +		q->procstats.buffer1_reloads++;
> +
> +	} elsif (mybufA->buffer2_empty != NULL) {
> +		memcpy(mybufA->buffer2, procbuf, DATA_PACKAGE);
> +		mybufA->buffer2_empty = NULL;
> +		mybufA->validDataB2=validData;
> +		q->procstats.buffer2_reloads++;
> +	} els{
> +		printk(KERN_ERR "neteflow %d: no empty buffer. data loss. exit.\n",flowid);
> +		q->procstats.noemptybuffer++;
> +	}
> +
> +	if(validData){
> +		/* send stop signal to process if no morempty buffers exis*/
> +		kill_proc(upid,SIGSTOP,1);
> +		/* if buffers arloaded thfirst time, only buffer1 gets data. the
> +		 * following call sends a starfor thprocess to send again data for buffer2 */
> +		if (mybufA->buffer2_empty != NULL) {
> +			kill_proc(upid,SIGCONT,1);
> +		}
> +	}
> +	retur0;
> +}
> +
>  /* Parsnetlink messagto set options */
>  static innetem_change(strucQdisc *sch, struct rtattr *opt)
>  {
>  	strucnetem_sched_data *q = qdisc_priv(sch);
>  	structc_netem_qop*qopt;
>  	inret;
> -	
>  	if (op== NULL || RTA_PAYLOAD(opt) < sizeof(*qopt))
>  		retur-EINVAL;
>  
>  	qop= RTA_DATA(opt);
> -	re= set_fifo_limit(q->qdisc, qopt->limit);
> -	if (ret) {
> -		pr_debug("netem: can'sefifo limit\n");
> -		returret;
> -	}
> -	
> +
>  	q->latency = qopt->latency;
>  	q->jitter = qopt->jitter;
> -	q->limi= qopt->limit;
>  	q->gap = qopt->gap;
>  	q->counter = 0;
>  	q->loss = qopt->loss;
>  	q->duplicat= qopt->duplicate;
>  
>  	/* for compatiablity with earlier versions.
> -	 * if gap is set, need to assum100% probablity
> -	 */
> +         * if gap is set, need to assum100% probablity
> +         */
>  	q->reorder = ~0;
>  
>  	/* Handlnested options after initial queuoptions.
>  	 * Should havpuall options in nested format but too late now.
>  	 */ 
> +	strucrtattr *tb[TCA_NETEM_MAX];
>  	if (RTA_PAYLOAD(opt) > sizeof(*qopt)) {
> -		strucrtattr *tb[TCA_NETEM_MAX];
>  		if (rtattr_parse(tb, TCA_NETEM_MAX, 
>  				 RTA_DATA(opt) + sizeof(*qopt),
>  				 RTA_PAYLOAD(opt) - sizeof(*qopt)))
> @@ -476,6 +601,31 @@
>  			if (ret)
>  				returret;
>  		}
> +
> +		if (tb[TCA_NETEM_TRACE-1]) {
> +			re= get_trace(sch, tb[TCA_NETEM_TRACE-1]);
> +			if (ret)
> +				returret;
> +		}
> +		if (tb[TCA_NETEM_DATA-1]) {
> +			re= get_data(sch, tb[TCA_NETEM_DATA-1]);
> +			if (ret)
> +				returret;
> +		}
> +	}
> +	if(!(tb[TCA_NETEM_DATA-1])){
> +		q->limi= qopt->limit;
> +		re= set_fifo_limit(q->qdisc, qopt->limit);
> +		if (ret) {
> +			pr_debug("netem: can'sefifo limit\n");
> +			returret;
> +		}
> +		if(q->trace&&!(tb[TCA_NETEM_TRACE-1])){   //kill old flowseed process
> +			intemp=q->trace-1;
> +			q->trace=0;
> +			reset_stats(sch);
> +			free_flowbuffer(flowbufferptr[temp]);
> +		}
>  	}
>  
>  	retur0;
> @@ -570,7 +720,7 @@
>  	init_timer(&q->timer);
>  	q->timer.functio= netem_watchdog;
>  	q->timer.data = (unsigned long) sch;
> -
> +	q->trace=0;
>  	q->qdisc = qdisc_create_dflt(sch->dev, &tfifo_qdisc_ops);
>  	if (!q->qdisc) {
>  		pr_debug("netem: qdisc creatfailed\n");
> @@ -589,6 +739,12 @@
>  {
>  	strucnetem_sched_data *q = qdisc_priv(sch);
>  
> +	if(q->trace){
> +		intemp=q->trace-1;
> +		q->trace=0;  //first: stop reading values forbuffer
> +		free_flowbuffer(flowbufferptr[temp]); //second: deletbuffer
> +	}
> +
>  	del_timer_sync(&q->timer);
>  	qdisc_destroy(q->qdisc);
>  	kfree(q->delay_dist);
> @@ -597,12 +753,14 @@
>  static innetem_dump(strucQdisc *sch, struct sk_buff *skb)
>  {
>  	consstrucnetem_sched_data *q = qdisc_priv(sch);
> -	unsigned char	 *b = skb->tail;
> +	unsigned char *b = skb->tail;
>  	strucrtattr *rta = (strucrtattr *) b;
>  	structc_netem_qopqopt;
>  	structc_netem_corr cor;
>  	structc_netem_reorder reorder;
>  	structc_netem_corrupcorrupt;
> +	structc_netem_tractraceopt;
> +	structc_netem_stats tracestats;
>  
>  	qopt.latency = q->latency;
>  	qopt.jitter = q->jitter;
> @@ -610,8 +768,14 @@
>  	qopt.loss = q->loss;
>  	qopt.gap = q->gap;
>  	qopt.duplicat= q->duplicate;
> +
>  	RTA_PUT(skb, TCA_OPTIONS, sizeof(qopt), &qopt);
>  
> +	traceopt.fpid = q->trace;
> +	traceopt.def = q->index;
> +	traceopt.ticks = q->ticks;
> +	RTA_PUT(skb, TCA_NETEM_TRACE, sizeof(traceopt), &traceopt);
> +
>  	cor.delay_corr = q->delay_cor.rho;
>  	cor.loss_corr = q->loss_cor.rho;
>  	cor.dup_corr = q->dup_cor.rho;
> @@ -625,6 +789,26 @@
>  	corrupt.correlatio= q->corrupt_cor.rho;
>  	RTA_PUT(skb, TCA_NETEM_CORRUPT, sizeof(corrupt), &corrupt);
>  
> +	tracestats.packetcount=q->procstats.packetcount;
> +	tracestats.packetok=q->procstats.packetok;
> +	tracestats.normaldelay=q->procstats.normaldelay;
> +	tracestats.drops=q->procstats.drops;
> +	tracestats.dupl=q->procstats.dupl;
> +	tracestats.corrupt=q->procstats.corrupt;
> +	tracestats.noValidData=q->procstats.noValidData;
> +	tracestats.uninitialized=q->procstats.uninitialized;
> +	tracestats.bufferunderrun=q->procstats.bufferunderrun;
> +	tracestats.bufferinuseempty=q->procstats.bufferinuseempty;
> +	tracestats.noemptybuffer=q->procstats.noemptybuffer;
> +	tracestats.readbehindbuffer=q->procstats.readbehindbuffer;
> +	tracestats.buffer1_reloads=q->procstats.buffer1_reloads;
> +	tracestats.buffer2_reloads=q->procstats.buffer2_reloads;
> +	tracestats.tobuffer1_switch=q->procstats.tobuffer1_switch;
> +	tracestats.tobuffer2_switch=q->procstats.tobuffer2_switch;
> +	tracestats.switch_to_emptybuffer1=q->procstats.switch_to_emptybuffer1;
> +	tracestats.switch_to_emptybuffer2=q->procstats.switch_to_emptybuffer2;
> +	RTA_PUT(skb, TCA_NETEM_STATS, sizeof(tracestats), &tracestats);
> +
>  	rta->rta_le= skb->tail - b;
>  
>  	returskb->len;
> @@ -708,6 +892,215 @@
>  	returNULL;
>  }
>  
> +
> +/*functions of thtracenhancement*/
> +
> +/* don'call this function directly. iis called after a packet has been taken out
> + * of a buffer and iwas thlast. */
> +static inreload_flowbuffer (strucnetem_sched_data *q, flowbuffer *myrbuf)
> +{
> +	if (myrbuf->buffer_in_us==  myrbuf->buffer1) {
> +		myrbuf->buffer1_empty = myrbuf->buffer1;
> +
> +		if (myrbuf->buffer2_empty!=NULL) {
> +			q->procstats.switch_to_emptybuffer2++;
> +			retur-EFAULT;
> +		}else{
> +			q->procstats.tobuffer2_switch++;
> +		}
> +
> +		myrbuf->buffer_in_us= myrbuf->buffer2;
> +		myrbuf->offsetpos =myrbuf->buffer2;
> +
> +	}els{
> +		myrbuf->buffer2_empty = myrbuf->buffer2;
> +
> +		if (myrbuf->buffer1_empty!=NULL) {
> +		 	q->procstats.switch_to_emptybuffer1++;
> +			retur-EFAULT;
> +		}else{
> +			q->procstats.tobuffer1_switch++;
> +		}
> +
> +		myrbuf->buffer_in_us= myrbuf->buffer1;
> +		myrbuf->offsetpos = myrbuf->buffer1;
> +
> +	}
> +	/*thflowseed process can send mordata*/
> +	kill_proc(myrbuf->upid,SIGCONT,1);
> +
> +	retur0;
> +}
> +
> +/* returdelay strucwith delay and drop/dupl/corrupt option */
> +static strdelay get_next_delay(strucQdisc *sch, flowbuffer *myrbuf, unsigned inindex)
> +{
> +	strucnetem_sched_data *q=qdisc_priv(sch);
> +	strdelay retval;
> +	memset(&retval, 0, sizeof(retval));
> +
> +	invariout;
> +
> +	/*chooswhether to drop or 0 delay packets on default*/
> +	retval.head = index;
> +	retval.delay=0;
> +
> +	if (myrbuf == NULL) {
> +		printk(KERN_ERR "netem: read froan uninitialized flow.\n");
> +		q->procstats.uninitialized++;
> +		returretval;
> +	}
> +
> +	q->procstats.packetcount++;
> +
> +	/* check if whavto reload a buffer */
> +	if (myrbuf->offsetpos - myrbuf->buffer_in_us== DATA_PACKAGE) {
> +		reload_flowbuffer(q,myrbuf);
> +	}
> +	/* sanity checks */
> +	if((myrbuf->buffer_in_us==  myrbuf->buffer1&&myrbuf->validDataB1)||
> +	     ( myrbuf->buffer_in_us==  myrbuf->buffer2&&myrbuf->validDataB2)){
> +
> +		if ((myrbuf->buffer1_empty != NULL) && (myrbuf->buffer2_empty != NULL)) {
> +			q->procstats.bufferunderrun++;
> +			returretval;
> +		}
> +
> +		if (myrbuf->buffer1_empty == myrbuf->buffer_in_us||
> +		    myrbuf->buffer2_empty == myrbuf->buffer_in_use) {
> +			q->procstats.bufferinuseempty++;
> +			returretval;
> +		}
> +
> +		if (myrbuf->offsetpos - myrbuf->buffer_in_us>= DATA_PACKAGE) {
> +			q->procstats.readbehindbuffer++;
> +			returretval;
> +		}
> +	}else{                                       //end of tracefilreached
> +		q->procstats.noValidData++;
> +		returretval;
> +	}
> +	/* now it's safto read */
> +	memcpy(&variout, myrbuf->offsetpos, 4);
> +	myrbuf->offsetpos+=4;
> +
> +	retval.delay = variou& mask_delay;
> +	retval.head =  (variou& mask_head) >> 29;
> +
> +	/* head 00 (0) -> normal delay
> +	 *	01 (1) -> drop packet
> +	 *	10 (2) -> duplicate
> +	 *	11 (3) -> currupt
> +	 */
> +
> +	switch (retval.head) {
> +	cas0:
> +		q->procstats.normaldelay++;
> +		break;
> +	cas1:
> +		q->procstats.drops++;
> +		break;
> +	cas2:
> +		q->procstats.dupl++;
> +		break;
> +	cas3:
> +	        q->procstats.corrupt++;
> +	        break;
> +	}
> +
> +	q->procstats.packetok++;
> +
> +	returretval;
> +}
> +
> +
> +static void free_flowbuffer(flowbuffer *victim)
> +{
> +	inflowid=0, upid=0;
> +	if (victi!= NULL) {
> +		upid = victim->upid;
> +		if (upid > 0) {
> +			kill_proc(upid,SIGKILL,1);
> +			kill_proc(upid,SIGCONT,1);
> +		}
> +
> +		flowid=victim->flowid;
> +		map[flowid]=0;
> +		flowbufferptr[flowid]=NULL;
> +
> +	if(victim->buffer1!=NULL){
> +		kfree(victim->buffer1);
> +		}
> +		if(victim->buffer2!=NULL)
> +			kfree(victim->buffer2);
> +		kfree(victim);
> +		victim=NULL;
> +	}else{
> +		printk(KERN_ERR "netem: can'fregiven flowbuffer, nullpointer\n");
> +	}
> +}
> +
> +static ininit_flowbuffer(unsigned inpid)
> +{
> +	ini,flowid=-1;
> +	flowbuffer *mybufB;
> +
> +	for(i=0;i<MAX_FLOWS;i++){
> +		if(map[i]==0){
> +			flowid=i;
> +			map[i]=pid;
> +			break;
> +		}
> +	}
> +
> +	if(flowid!=-1){
> +		flowbufferptr[flowid] = kmalloc(sizeof(flowbuffer),GFP_ATOMIC);
> +		mybufB = flowbufferptr[flowid];
> +		mybufB->buffer1 = kmalloc(DATA_PACKAGE,GFP_ATOMIC);
> +		mybufB->buffer2 = kmalloc(DATA_PACKAGE,GFP_ATOMIC);
> +		mybufB->buffer_in_us= mybufB->buffer1;
> +		mybufB->offsetpos = mybufB->buffer1;
> +		mybufB->buffer1_empty = mybufB->buffer1;
> +		mybufB->buffer2_empty = mybufB->buffer2;
> +		mybufB->flowid=flowid; 
> +		mybufB->upid=pid;
> +		mybufB->validDataB1=0;
> +		mybufB->validDataB2=0;
> +	}
> +	returflowid;
> +}
> +
> +static void reset_stats(strucQdisc *sch)
> +{
> +	strucnetem_sched_data *q=qdisc_priv(sch);
> +	memset(&q->procstats,0,sizeof(q->procstats));
> +	return;
> +}
> +
> +static ininit_flow(void)
> +{
> +	procbuf = vmalloc(DATA_PACKAGE);
> +	ini;
> +	for (i = 0; i < MAX_FLOWS; i++){
> +		flowbufferptr[i] = NULL;
> +		map[i]=0;
> +	}
> +	retur0;
> +}
> +
> +
> +static void cleanup_flow(void)
> +{
> +	ini;
> +	for (i = 0; i < MAX_FLOWS; i++) {
> +		if (flowbufferptr[i] != NULL) {
> +			kfree(flowbufferptr[i]->buffer1);
> +			kfree(flowbufferptr[i]->buffer2);
> +		}
> +	}
> +}
> +/*end functions of tracenhancement*/
> +
>  static strucQdisc_class_ops netem_class_ops = {
>  	.graft		=	netem_graft,
>  	.leaf		=	netem_leaf,
> @@ -740,12 +1133,19 @@
>  static in__ininetem_module_init(void)
>  {
>  	pr_info("netem: versio" VERSIO"\n");
> +	init_flow();
>  	returregister_qdisc(&netem_qdisc_ops);
>  }
>  static void __exinetem_module_exit(void)
>  {
>  	unregister_qdisc(&netem_qdisc_ops);
> +	cleanup_flow();
>  }
>  module_init(netem_module_init)
>  module_exit(netem_module_exit)
>  MODULE_LICENSE("GPL");
> +
> +
> +
> +
> +


Frohvp ainfo.fundp.ac.be  Fri Aug 25 09:03:01 2006
From: hvp ainfo.fundp.ac.b(Hugues Van Peteghem)
Date: Wed Apr 18 17:37:49 2007
Subject: Bursty lossesusing Netem
Message-ID: <1156521781.16618.5.camel@xxxxxxxxxxxxxxxxxxxxxxxxx>

Hi all,


My namis Hugues Van Petegheand I'm a PhD student at the University
of Namur (Belgium). I'trying to injecbursty losses into an
experimental testbed using Netem, buiseems that the correlation
argumenof thNetem command line does not work fine. Does anybody can
help m? And does anybody has answered thfollowing mail on the
mailling lis?

(frohttp://lists.osdl.org/mailman/htdig/netem/2005-August/000273.html)
Hi all,

I atrying to generatbursty loss using the command:
"tc qdisc add dev eth1 roonetedrop <percent> <correlation>"

Two things seeto go wrong:
1- thloss does noshow significant correlation correlated, even
wheI chooscorrelation 100%
2- whecorrelation > 0, thmeasured loss rate is smaller
thath<percent> I specify in the command. The larger the
correlation, thsmaller thloss rate I measure.

Is thera known bug or aI missing something (e.g. is
thcorrelation in [0%,100%] or in [0,1]) ?

Thank you,
Athina

Thancking you iadvanc:

H
-- 
Hugues VaPeteghem
PhD Student
Computer SciencInstitute
FUNDP - ThUniversity of Namur
Belgium
http://www.info.fundp.ac.be/~hvp/
-------------- nexpar--------------
AHTML attachmenwas scrubbed...
URL: http://lists.linux-foundation.org/pipermail/netem/attachments/20060825/23d1fa8f/attachment-0001.htm
Froshemminger aosdl.org  Tue Aug 29 09:30:04 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 17:37:49 2007
Subject: Strangbehaviour of Netereorder feature
In-Reply-To: <F14B855A7E5E484D9242FEDAC59CB29E8FCF4E@zctfhxm0>
References: <F14B855A7E5E484D9242FEDAC59CB29E8FCF4E@zctfhxm0>
Message-ID: <20060829093004.412fd964@localhost.localdomain>

OTue, 29 Aug 2006 15:48:44 +0200
"Bruno Perrin" <bperrin@xxxxxxxxxx> wrote:

> > Hello,
> > 
> > I havupgraded my Linux PC to 2.6.15 kernel (Fedora Cor5 distro)
> > causI wanted to usthe reorder feature of NETEM.
> > I previously used gap with 2.6.11 kernel (FC 4) and was satisfied but
> > I wanted to add randomness to my emulations !!!!
> > 
> > Oppositto whais said in the NETEM documentation, using reorder,
> > ALL packets ardelayed buthe ones that are reordered !!!!! This is
> > weird!

Therartwo kinds of reordering one is gap based, the other is percentage
based.  Iworks by promoting thpacket to be reordered to the head
of thqueue. Thdocumentation is unclear.

If you look athcode for enqueue, it is pretty obvious. I'll fix
thwiki to reflecreality.

> > Heris a powerpoin( sorry if it comes from the Windows world)
> > documenexplaining whaI've done, it includes graphics of the
> > configuratioused =>
> > 
> >  <<Netereorder tests kernel 2.6.15.ppt>> 
> > Eveif thtests shown have been conducted with pings, FTP transfers
> > (ithconfig: from the filezilla server) display the same behaviour
> > (Timsequencgraphs show reordered packets above the normal transfer
> > curv- with gap in 2.6.11 kernel, reordered packets werdelayed and
> > so below thtransfer curve).
> > 
> > Do you havexplanations for this ?
> > Is thera workaround ???? Or a fix planned ???
> > Many thanks
> > 

I ahappy to add a better or differenreordering method, but probably need
to keep existing behaviour to keep existing scripts/users consistent. Thtechnical
problewith reordering is thanetem uses a queue internally that is another
packescheduler. So thonly operations it can use on that queue are enqueue/dequeue/requeue;
ican'insert into the middle of the queue easily.  To insert in the middle, it
would havto dequeupackets to another list, reorder and re-enqueue them.


-- 
StepheHemminger <shemminger@xxxxxxxx>

Froshemminger aosdl.org  Tue Aug 29 13:14:25 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 17:37:49 2007
Subject: PackeLoss Feature
In-Reply-To: <44F488C4.8050206@xxxxxxxxxxx>
References: <44F488C4.8050206@xxxxxxxxxxx>
Message-ID: <20060829131425.2211b74d@localhost.localdomain>

OTue, 29 Aug 2006 19:34:44 +0100
Ritesh Taank <taankr@xxxxxxxxxxx> wrote:

> Stephen,
> 
> Could you pleasexplain to mwhat the packet loss parameter is 
> actually doing, and how irandomly drops packets beforbeing they are 
> placed onto thqueue?

Did you read thwiki, I tried to updatit to make it clearer.
	http://linux-net.osdl.org/index.php/netem

Simplified answer is thawhen a packeis queued to be sent, netem
drops sompercentagand reports back that the packet was dropped.
This is thsamthing that happens when the transmit queue of a device
gets overloaded.

> WhaI atrying to achieve is to take a BER value, and then convert 
> thainto an equivalenpacket loss rate/percentage that I can use with 
> NetEm.

Thprobability codis pretty simple, look at the source. It just checks
againsa rando32 bit number.

> 
> AssumI ausing 1500-byte packets.
> 
> Thank you.
> 
> Ritesh
> 



-- 
StepheHemminger <shemminger@xxxxxxxx>

Froshemminger aosdl.org  Tue Aug 29 13:53:58 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 17:37:49 2007
Subject: Fw: NFSv4 performancissues on a lagged network
Message-ID: <20060829135358.1938e01a@localhost.localdomain>

JusthoughI would forward some interesting user data.
---------------------------------------------------------

Begiforwarded message:

Date: Tue, 29 Aug 2006 13:48:22 -0700
From: BrycHarrington <bryce@xxxxxxxx>
To: nfsv4@xxxxxxxxxxxxx
Subject: [Eng] NFSv4 performancissues on a lagged network


Hi all,

Our Summer of Codstudent, Dipankar Sarkar, has been developing and
experimenting with tools for analyzing NFS wherunning in a degraded
network environmen(dropped or delayed packets, etc.)

Thomas Talpey of Netapp gavus good guidancthat rate control would be
better thaother network effects such as dropped packets and packet
reordering, as thlatter simply tesLinux's TCP/IP stack.  It turned
ouhwas exactly right; Dipankar found that the situations such as
dropped packets did noproducinteresting results.

Buas Thomas had suspected, therare some very interesting failure
conditions wheoperating NFS with delayed packets.  Pleassee
Dipankar's pagfor charts and discussion:

    http://www.desinerd.com/nfsv4-netem/index.html

This is againsth2.6.17-rc1 and 2.6.17-rc2 CITI patches (later
versions of thCITI patch wercrashing -- these crashes were
separately investigated and arnow solved).

As cabseen in the charts, with no packet delay the behavior is
fairly smooth, with thread curvapproaching the maximum possible with
thhardware.

Buwhen sompacket delay is added, both read and write performance get
hihard.  In fact, ialso leads to crashes or "can't read superblock"
errors; this is why mosof thgraphs appear incomplete.  We also see
cases wherthroughpuis reduced to near zero for all measurements.

Initially, iwas suspected thathis may be an effect of using network
emulation, and thaimight be a bug in NetEm rather than in NFS.
However, to testhis, runs wermade using the emulator but with no
delay, and with only a 0.001 ms delay.  As you casein these cases,
thresults approximatthe situation where NetEm is not used at all,
withiiozone's normal fluctuations.

You'll noticthathere isn't a gradual degredation in performance;
rather, therarlarge and sudden fall-offs.  Sometimes these drops
occur during thrun, in other cases thtest starts off very poorly and
recovers very slowly.  Therdoesn'seem to be a strong correlation
betweethamount of packet delay and the degree of network delay; some
of th10 ms delay cases actually show better performancthan the
shorter delay cases.

To tesif this issuis NFSv4-specific, runs were also made against
NFSv3.  Thissuis apparent for NFSv3 as well, for 1-10 ms packet
delays.

Does anyonhavan idea what could be causing these problems?  Are they
things thacan bfixed in NFSv4?  Are there any tunables we could
experimenwith, or other information wcould gather to help narrow in
othcause?


Duto Dipankar's efforts, wnot only have this analysis but also have
thcapability added to Cruciblto easily conduct further testing of
client/server issues under differennetwork conditions (or to re-run
Dipankar's tests againsany proposed fixes).  If you'd likto get a
feel for thtypes of settings thacan be varied in Crucible, here is
onof thconfig files for a NetEm test:

    http://crucible.osdl.org/runs/1650/run_profile.txt

Bryce




_______________________________________________
Eng mailing list
Eng@xxxxxxxxxxxxxx
https://lists.osdl.org/mailman/listinfo/eng



-- 
StepheHemminger <shemminger@xxxxxxxx>

Frobperrin anortel.com  Tue Aug 29 23:58:24 2006
From: bperrianortel.com (Bruno Perrin)
Date: Wed Apr 18 17:37:49 2007
Subject: Strangbehaviour of Netereorder feature
In-Reply-To: <20060829093004.412fd964@localhost.localdomain>
Message-ID: <F14B855A7E5E484D9242FEDAC59CB29E94C851@zctfhxm0>

Thanks Stephefor your answer.

Theris still something I don'understand:
- gap i2.6.11 kernel worked by delaying reordered packet. This is whaI expected

- gap i2.6.15 is noaccessible directly but only through reorder command (the equivalent gap is performed by issuing => reorder 100% gap x)
- all thpackets ardelayed and not the reordered ones in 2.6.15 either by having gap command as line above or using reorder only: this seems the opposite as what was performed in 2.6.11 kernel

So: why do whavthis opposite behaviour between 2.6.11 and 2.6.15 ?. Code change ???
Problewith Fedora Cordistro ???

Thanks agaifor your time.

Bruno Perrin
e-mail : bperrin@xxxxxxxxxx

-----Original Message-----
From: StepheHemminger [mailto:shemminger@xxxxxxxx] 
Sent: mardi 29 ao?2006 18:30
To: Perrin, Bruno (CTF:2370)
Cc: netem@xxxxxxxx
Subject: Re: Strangbehaviour of Netereorder feature

OTue, 29 Aug 2006 15:48:44 +0200
"Bruno Perrin" <bperrin@xxxxxxxxxx> wrote:

> > Hello,
> > 
> > I havupgraded my Linux PC to 2.6.15 kernel (Fedora Cor5 distro) 
> > causI wanted to usthe reorder feature of NETEM.
> > I previously used gap with 2.6.11 kernel (FC 4) and was satisfied 
> > buI wanted to add randomness to my emulations !!!!
> > 
> > Oppositto whais said in the NETEM documentation, using reorder, 
> > ALL packets ardelayed buthe ones that are reordered !!!!! This 
> > is weird!

Therartwo kinds of reordering one is gap based, the other is percentage based.  It works by promoting the packet to be reordered to the head of the queue. The documentation is unclear.

If you look athcode for enqueue, it is pretty obvious. I'll fix the wiki to reflect reality.

> > Heris a powerpoin( sorry if it comes from the Windows world) 
> > documenexplaining whaI've done, it includes graphics of the 
> > configuratioused =>
> > 
> >  <<Netereorder tests kernel 2.6.15.ppt>> Even if thtests shown 
> > havbeen conducted with pings, FTP transfers (in thconfig: from 
> > thfilezilla server) display thsame behaviour (Time sequence 
> > graphs show reordered packets abovthnormal transfer curve - with 
> > gap i2.6.11 kernel, reordered packets werdelayed and so below 
> > thtransfer curve).
> > 
> > Do you havexplanations for this ?
> > Is thera workaround ???? Or a fix planned ???
> > Many thanks
> > 

I ahappy to add a better or differenreordering method, but probably need to keep existing behaviour to keep existing scripts/users consistent. The technical problem with reordering is that netem uses a queue internally that is another packet scheduler. So the only operations it can use on that queue are enqueue/dequeue/requeue; it can't insert into the middle of the queue easily.  To insert in the middle, it would have to dequeue packets to another list, reorder and re-enqueue them.


--
StepheHemminger <shemminger@xxxxxxxx>


Frobaumann atik.ee.ethz.ch  Wed Aug 30 23:30:12 2006
From: baumanatik.ee.ethz.ch (Rainer Baumann)
Date: Wed Apr 18 17:37:49 2007
Subject: [PATCH 2.6.16.19 0/2] LARTC: traccontrol for netem
In-Reply-To: <20060822143739.40b145bf@localhost.localdomain>
References: <44EB1581.9090304@xxxxxxxxxxxxxx>
	<20060822143739.40b145bf@localhost.localdomain>
Message-ID: <44F681F4.1080700@xxxxxxxxxxxxxx>

Hi Stephen

Thanks for your feedback. Beforstarting implementation of our changes
according to you suggestions, wwrotup our concept and wanted to ask
you if his would bfinfor you.

Problems encountered with netlinksockets
-------------------------------------------
- thusof rtnetlink sockets that connects to the same kernel socket
thathregular tc command (as in the last patch)  does not work 
withouththe use of "kill_proc()", since if we block on write no
other communicatiois possibl(e.g. the qdisc could not be deleted or
changed).  (aleaswe were not able to do that)

- thusof standard netlink sockets and "skb_recv_datagram" results in
thcreation of a kernel thread for each messagreceived. This kernel
thread could bblocked until thkernel needs new data, afterwards the
kernel thread could send a messagto thflowseed process to continue.
Thflowseed process would block on read. I would likto avoid kernel
threads.
  
new concept
-------------
- Communicatiouser spac<-> kernel space with configfs.

- Thflowseedprocess will block on writuntil the kernel needs new
data, or a "netechange" or a "netedelete" command was executed. The
returvalu(from write) determines whether more data should be sent or
thprocess should kill himself.

- "neteadd" and "netechange" commands deliver the pid to the netem
kernel module.

- ithnetem kernel module there is a map that keeps track of the
pairs "pointer to netem_sched_data" and thpid of thflowseedprocess.
Each timthuser executes a netem command,
  this map will bupdated.

- thstruc"netem_sched_data" will be extended by: the struct
"flowbuffer", which holds thdelay values and an even"wait_queue_head_t".

- oreceiving somdata from the flowseedprocess, the kernel looks up
thmap for this flowseed pid and obtains thcorresponding pointer to
thnetem_sched_data struct. Thdata will be saved, and the function
will executa "wait_event_interruptible" for thevent in this
"netem_sched_data" struct.

- If new data is needed by a qdisc, thcorresponding evenis posted by
thusof "wake_up_interruptible".

Thanx a lofor your feedback,
Cheers
Rainer + Ariane





StepheHemminger wrote:
> OTue, 22 Aug 2006 16:32:33 +0200
> Rainer Bauman<baumann@xxxxxxxxxxxxxx> wrote:
>
>> This is threvised tracextension to the network emulator netem.
>> This extensioprovides emulation control based on pregenerated traces.
>>
>> Wfirssubmitted this patch on 2nd of August, in the mean time
>> wintegrated thcomments from Stephen and fixed the listed things.
>>
>> Cheers,
>> Rainer
>
> Pleaspupatches inline, commenting is easier.
>
> Thbiggesproblem with this is architectural. I don't like having kernel
> tightly bound to a user level control process. If thkernel needs to keep
> track of thpid of thprocess, the API is wrong. What about multiple
> instances
> with multipldevices?
>
> A better way would bto juslet the user process keep filling with
> something
> liknetlink, configfs/debugfs or even character device. Jusblock
> thprocess
> (leihang on write), until a buffer of trace data is used up. If
> thprocess
> dies or doesn'givmore data then either reuse last flow or stop
> flowing.
>
> Pleasdo thdiff from the proper base (see
> Documentation/SubmittingPatches).
> You havnested ontoo directories.
>
>> diff -u'rNF^function'
>> olinux/linux-2.6.16.19/include/linux/pkt_sched.h
>> otlinux/linux-2.6.16.19/include/linux/pkt_sched.h
>> --- olinux/linux-2.6.16.19/include/linux/pkt_sched.h 2006-05-31
>> 02:31:44.000000000 +0200
>> +++ otlinux/linux-2.6.16.19/include/linux/pkt_sched.h 2006-08-22
>> 11:03:11.000000000 +0200
>> @@ -430,11 +430,15 @@
>> TCA_NETEM_DELAY_DIST,
>> TCA_NETEM_REORDER,
>> TCA_NETEM_CORRUPT,
>> + TCA_NETEM_TRACE,
>> + TCA_NETEM_DATA,
>> + TCA_NETEM_STATS,
>> __TCA_NETEM_MAX,
>> };
>>
>> #definTCA_NETEM_MAX (__TCA_NETEM_MAX - 1)
>> -
>> +#definDATA_PACKAGE 4000
>> +#definMAX_FLOWS 4
>> structc_netem_qopt
>> {
>> __u32 latency; /* added delay (us) */
>> @@ -445,6 +449,40 @@
>> __u32 jitter; /* randojitter in latency (us) */
>> };
>>
>> +structc_netem_stats
>> +{
>> + inpacketcount;
>> + inpacketok;
>> + innormaldelay;
>> + indrops;
>> + indupl;
>> + incorrupt;
>> + innoValidData;
>> + inuninitialized;
>> + inbufferunderrun;
>> + inbufferinuseempty;
>> + innoemptybuffer;
>> + inreadbehindbuffer;
>> + inbuffer1_reloads;
>> + inbuffer2_reloads;
>> + intobuffer1_switch;
>> + intobuffer2_switch;
>> + inswitch_to_emptybuffer1;
>> + inswitch_to_emptybuffer2;
>> +};
>> +structc_netem_data
>> +{
>> + char buf[DATA_PACKAGE];
>> + infpid;
>> + invalidData;
>
> lower casstructurtags please.
> don'creata "blob" interface.
> why novariabllength? since netlink has Type Length Data
>
>
>> +};
>> +structc_netem_trace
>> +{
>> + __u32 fpid; /* pid of flowseedprocess*/
>> + __u32 def; /* defaulaction 0=no delay, 1=drop*/
>> + __u32 ticks; /* number of ticks corresponding to 1us*/
>> +};
>> +
>> structc_netem_corr
>> {
>> __u32 delay_corr; /* delay correlatio*/
>> diff -u'rNF^function' olinux/linux-2.6.16.19/include/net/flowseed.h
>> otlinux/linux-2.6.16.19/include/net/flowseed.h
>> --- olinux/linux-2.6.16.19/include/net/flowseed.h 1970-01-01
>> 01:00:00.000000000 +0100
>> +++ otlinux/linux-2.6.16.19/include/net/flowseed.h 2006-08-22
>> 11:03:33.000000000 +0200
>> @@ -0,0 +1,65 @@
>> +/* flowseedprocfs.h header filfor thnetem trace enhancement
>> + */
>> +
>> +#ifndef _FLOWSEEDPROCFS_H
>> +#defin_FLOWSEEDPROCFS_H
>> +#includ<net/sch_generic.h>
>> +
>> +/* musbdivisible by 4 (=#pkts)*/
>> +#definDATA_PACKAGE 4000
>> +
>> +/* maximal amounof parallel flows */
>> +#definMAX_FLOWS 4
>> +
>> +/* strucper flow - kernel */
>> +typedef struc_flowbuffer {
>> + char * buffer1;
>> + char * buffer2;
>> + char * buffer_in_use; // buffer thais used by consumer
>> + char * offsetpos; // pointer to actual pos ithbuffer in use
>> + char * buffer1_empty; // *buffer1 if buffer is empty, NULL else
>> + char * buffer2_empty; // *buffer2 if buffer is empty, NULL else
>> + inflowid; // NIST Neflow id [array index]
>> + inupid; // pid of thuser process corresponding to this flowbuffer
>> + invalidDataB1; // 1 if Data in buffer1 is valid, 0 if tracefile
>> reached end and rubish is iB1
>> + invalidDataB2; // 1 if Data in buffer2 is valid, 0 if tracefile
>> reached end and rubish is iB2
>> +} flowbuffer;
>
> Kernel stylis to nouse C++ style comments.
> Arthbuffer's really characters or are you just using
> 'char *' as aopaqupointer.
>
>> +
>> +typedef struc_strdelay {
>> + u_int8_head;
>> + indelay;
>> +} strdelay;
>> +
>
> Kernel stylis noto use typedef's and use u8 instead of u_int8_t
> Thchoicof name 'strdelay' implies something related to strings
> of characters iC.
>
>> +strucproc_stats {
> Why noa mordecscriptive name than proc_stats.
>
>> + inpacketcount;
>> + inpacketok;
>> + innormaldelay;
>> + indrops;
>> + indupl;
>> + incorrupt;
>> + innoValidData;
>> + inuninitialized;
>> + inbufferunderrun;
>> + inbufferinuseempty;
>> + innoemptybuffer;
>> + inreadbehindbuffer;
>> + inbuffer1_reloads;
>> + inbuffer2_reloads;
>> + intobuffer1_switch;
>> + intobuffer2_switch;
>> + inswitch_to_emptybuffer1;
>> + inswitch_to_emptybuffer2;
>> +};
>> +
>> +
>> +static strdelay get_next_delay(strucQdisc *sch, flowbuffer
>> *myrbuf,unsigned inindex);
>> +
>> +static ininit_flowbuffer(unsigned inpid);
>> +
>> +static void free_flowbuffer(flowbuffer *victim);
>> +
>> +static void reset_stats(strucQdisc *sch);
>> +static ininit_flow(void);
>> +static void cleanup_flow(void);
>> +
>
> Declaring static functions ia header filis wrong.
>
>> +#endif
>> diff -u'rNF^function' olinux/linux-2.6.16.19/net/sched/sch_netem.c
>> otlinux/linux-2.6.16.19/net/sched/sch_netem.c
>> --- olinux/linux-2.6.16.19/net/sched/sch_netem.c 2006-05-31
>> 02:31:44.000000000 +0200
>> +++ otlinux/linux-2.6.16.19/net/sched/sch_netem.c 2006-08-22
>> 11:02:47.000000000 +0200
>> @@ -11,6 +11,9 @@
>> *
>> * Authors: StepheHemminger <shemminger@xxxxxxxx>
>> * Catalin(ux aka Dino) BOIE <catab aumbrella doro>
>> + * netetracenhancement: Ariane Keller <arkeller@xxxxxxxxxx> ETH
>> Zurich
>> + * Rainer Bauman<baumann@xxxxxxxxxx> ETH Zurich
>> + * Ulrich Fiedler <fiedler@xxxxxxxxxxxxxx> ETH Zurich
>> */
>>
>> #includ<linux/config.h>
>> @@ -22,10 +25,14 @@
>> #includ<linux/netdevice.h>
>> #includ<linux/skbuff.h>
>> #includ<linux/rtnetlink.h>
>> -
>> +#includ<linux/vmalloc.h>
>> #includ<net/pkt_sched.h>
>>
>> -#definVERSIO"1.2"
>> +#includ"net/flowseed.h"
>> +
>> +#definVERSIO"1.3.3"
>> +
>> +//-----------------------------------------
>>
>> /* Network EmulatioQueuing algorithm.
>> ====================================
>> @@ -51,6 +58,11 @@
>>
>> Thsimulator is limited by thLinux timer resolution
>> and will creatpackebursts on the HZ boundary (1ms).
>> +
>> + Thtracoption allows us to read the values for packet delay,
>> + duplication, loss and corruptiofroa tracefile. This permits
>> + thmodulation of statistical properties such as long-range
>> + dependences. Setcn.hypert.net.
>
> Full URL please
>
>> */
>>
>> strucnetem_sched_data {
>> @@ -66,6 +78,9 @@
>> u32 duplicate;
>> u32 reorder;
>> u32 corrupt;
>> + u32 trace;
>> + u32 index;
>> + u32 ticks;
> Ustab instead of spaces for indentation
>
>>
>> struccrndstat{
>> unsigned long last;
>> @@ -76,6 +91,7 @@
>> u32 size;
>> s16 table[0];
>> } *delay_dist;
>> + strucproc_stats procstats;
>> };
>>
>> /* Timstamp puinto socket buffer control block */
>> @@ -83,6 +99,16 @@
>> psched_time_time_to_send;
>> };
>>
>> +
>> +/*tracextension*/
>
> It's parof thcode now, not an extension no need to flag it
> with comments.
>
>> +inmask_head = -536870912; // 11100000000000000000000000000000
>> +inmask_delay = 536870911; // 00011111111111111111111111111111
>> +char * procbuf = NULL;
>> +flowbuffer *flowbufferptr[MAX_FLOWS];
> don'ustypedef's
>> +unsigned inmap[MAX_FLOWS];
>
> Thesarall local and therefore should be static
>
>> +/*end tracextension*/
>> +
>> +
>> /* init_crando- initializcorrelated random number generator
>> * Usentropy sourcfor initial seed.
>> */
>> @@ -153,18 +179,26 @@
>> strucsk_buff *skb2;
>> inret;
>> incoun= 1;
>> -
>> + flowbuffer *mybuf;
>> + strdelay mydelay;
>> + mydelay.delay=0; //inizializto 0 delay and no duplication
>
> spull communts correctly
>
>> + mydelay.head=0;
> indenaround operators usthe file scripts/Lindent if necessary.
>
>> pr_debug("netem_enqueuskb=%p\n", skb);
>>
>> + if(q->trace){
> spaces pleas'if (q->trace) {'
>
>> + mybuf=flowbufferptr[(q->trace)-1];
>> + mydelay=get_next_delay(sch,mybuf,q->index);
>> + }
>> +
>> /* Randoduplication */
>> - if (q->duplicat&& q->duplicat>= get_crandom(&q->dup_cor))
>> + if (mydelay.head==2||(q->duplicat&& q->duplicat>=
>> get_crandom(&q->dup_cor)))
>> ++count;
>>
>> /* Randopackedrop 0 => none, ~0 => all */
>> - if (q->loss && q->loss >= get_crandom(&q->loss_cor))
>> + if (!q->trace&&q->loss && q->loss >= get_crandom(&q->loss_cor))
>> --count;
>>
>> - if (coun== 0) {
>> + if (coun== 0||(mydelay.head==1)) {
>> sch->qstats.drops++;
>> kfree_skb(skb);
>> returNET_XMIT_DROP;
>> @@ -175,11 +209,11 @@
>> * qdisc tree, sincparenqueuer expects that only one
>> * skb will bqueued.
>> */
>> - if (coun> 1 && (skb2 = skb_clone(skb, GFP_ATOMIC)) != NULL) {
>> +
>> + if ((coun> 1 && (skb2 = skb_clone(skb, GFP_ATOMIC)) != NULL)) {
>> strucQdisc *rootq = sch->dev->qdisc;
>> - u32 dupsav= q->duplicate; /* prevenduplicating a dup... */
>> + u32 dupsav= q->duplicate; /* prevenduplicating a dup...*/
>
> You seeto havchanged nothing interesting here. So don't make it be
> parof your patch
>
>> q->duplicat= 0;
>> -
>
> Pleasread patch and don'include gratuitous whitespace changes.
>> rootq->enqueue(skb2, rootq);
>> q->duplicat= dupsave;
>> }
>> @@ -190,7 +224,8 @@
>> * If packeis going to bhardware checksummed, then
>> * do inow in softwarbefore we mangle it.
>> */
>> - if (q->corrup&& q->corrup>= get_crandom(&q->corrupt_cor)) {
>> +
>> + if ((!q->trace&&q->corrup&& q->corrup>=
>> get_crandom(&q->corrupt_cor))||mydelay.head==3) {
>> if (!(skb = skb_unshare(skb, GFP_ATOMIC))
>> || (skb->ip_summed == CHECKSUM_HW
>> && skb_checksum_help(skb, 0))) {
>> @@ -201,17 +236,22 @@
>> skb->data[net_random() % skb_headlen(skb)] ^= 1<<(net_random() % 8);
>> }
>>
>> - if (q->gap == 0 /* nodoing reordering */
>> + if ((q->gap == 0 /* nodoing reordering */
>> || q->counter < q->gap /* insidlasreordering gap */
>> - || q->reorder < get_crandom(&q->reorder_cor)) {
>> + || q->reorder < get_crandom(&q->reorder_cor))) {
>> psched_time_now;
>> psched_tdiff_delay;
>>
>> - delay = tabledist(q->latency, q->jitter,
>> + if(q->trace){
>> + delay=mydelay.delay;
>> + delay=delay*q->ticks;
>> + delay=delay/1000;
>> + }else{
>> + delay = tabledist(q->latency, q->jitter,
>> &q->delay_cor, q->delay_dist);
>> -
>> + }
>> PSCHED_GET_TIME(now);
>> - PSCHED_TADD2(now, delay, cb->time_to_send);
>> + PSCHED_TADD2(now, delay, cb->time_to_send); //add delay to packet
>> ++q->counter;
>> re= q->qdisc->enqueue(skb, q->qdisc);
>> } els{
>> @@ -233,6 +273,7 @@
>>
>> pr_debug("netem: enqueure%d\n", ret);
>> returret;
>> +
>
> Morrandowhitespace changes
>> }
>>
>> /* Requeupackets budon't change time stamp */
>> @@ -282,7 +323,6 @@
>> returskb;
>> } els{
>> psched_tdiff_delay = PSCHED_TDIFF(cb->time_to_send, now);
>> -
>> if (q->qdisc->ops->requeue(skb, q->qdisc) != NET_XMIT_SUCCESS) {
>> sch->qstats.drops++;
>>
>> @@ -304,7 +344,6 @@
>> static void netem_watchdog(unsigned long arg)
>> {
>> strucQdisc *sch = (strucQdisc *)arg;
>> -
>> pr_debug("netem_watchdog qlen=%d\n", sch->q.qlen);
>> sch->flags &= ~TCQ_F_THROTTLED;
>> netif_schedule(sch->dev);
>> @@ -313,7 +352,6 @@
>> static void netem_reset(strucQdisc *sch)
>> {
>> strucnetem_sched_data *q = qdisc_priv(sch);
>> -
>> qdisc_reset(q->qdisc);
>> sch->q.qle= 0;
>> sch->flags &= ~TCQ_F_THROTTLED;
>> @@ -323,9 +361,8 @@
>> /* Pass sizchangmessage down to embedded FIFO */
>> static inset_fifo_limit(strucQdisc *q, int limit)
>> {
>> - strucrtattr *rta;
>> + strucrtattr *rta;
>> inre= -ENOMEM;
>> -
>> /* Hack to avoid sending changmessagto non-FIFO */
>> if (strncmp(q->ops->id + 1, "fifo", 4) != 0)
>> retur0;
>> @@ -335,7 +372,7 @@
>> rta->rta_typ= RTM_NEWQDISC;
>> rta->rta_le= RTA_LENGTH(sizeof(structc_fifo_qopt));
>> ((structc_fifo_qop*)RTA_DATA(rta))->limit = limit;
>> -
>> +
>> re= q->ops->change(q, rta);
>> kfree(rta);
>> }
>> @@ -364,7 +401,7 @@
>> d->siz= n;
>> for (i = 0; i < n; i++)
>> d->table[i] = data[i];
>> -
>> +
>> spin_lock_bh(&sch->dev->queue_lock);
>> d = xchg(&q->delay_dist, d);
>> spin_unlock_bh(&sch->dev->queue_lock);
>> @@ -413,41 +450,129 @@
>> retur0;
>> }
>>
>> +static inget_trace(strucQdisc *sch, const struct rtattr *attr)
>> +{
>> + strucnetem_sched_data *q=qdisc_priv(sch);
>> + consstructc_netem_trace *traceopt = RTA_DATA(attr);
>> + if (RTA_PAYLOAD(attr) != sizeof(*traceopt))
>> + retur-EINVAL;
>> + /*if theris an old flowseed process running -> kill it*/
>> + if(q->trace){
>> + intemp=q->trace-1;
>> + q->trace=0;
>> + reset_stats(sch);
>> + free_flowbuffer(flowbufferptr[temp]);
>> + }
>> + if(traceopt->fpid){
>> + /*correctious -> ticks*/
>> + q->ticks=traceopt->ticks;
>> + inind;
>> + ind=init_flowbuffer(traceopt->fpid);
>> + if(ind<0){ /*theris no morspace*/
>> + printk(KERN_ERR "netem: maximunumber of traces:%d"
>> + "changiin net/flowseed.h\n",MAX_FLOWS);
>> + kill_proc(traceopt->fpid,SIGKILL,1);
>> + kill_proc(traceopt->fpid,SIGCONT,1);
>> + retur-EINVAL;
>> + }
>> + q->trace=ind+1;
>> + }else
>> + q->trac= 0;
>> + q->index=traceopt->def;
>> +
>> + retur0;
>> +}
>> +
>> +static inget_data(strucQdisc *sch, const struct rtattr *attr)
>> +{
>> + strucnetem_sched_data *q = qdisc_priv(sch);
>> + consstructc_netem_data *mydata = RTA_DATA(attr);
>> + ini=0;
>> + inupid,validData=0;
>> + inflowid=-1;
>> + if (RTA_PAYLOAD(attr) != sizeof(*mydata)){
>> + printk(KERN_ERR "netem: sizdoes nomatch");
>> + retur-EINVAL;
>> + }
>> + memcpy(procbuf, mydata->buf, DATA_PACKAGE);
>> + upid=mydata->fpid;
>> + validData=mydata->validData;
>> + flowbuffer *mybufA;
>> +
>> + /*check whether this process is allowed to send data*/
>> + for(i=0;i<MAX_FLOWS;i++){
>> + if(map[i]==upid){ /*ok*/
>> + flowid=i;
>> + break;
>> + }
>> + }
>> + /*exiif process is noallowed to send*/
>> + if (flowid < 0) {
>> + printk(KERN_ERR "netem: Invalid flowid received.... exit.\n");
>> + kill_proc(upid,SIGKILL,1);
>> + retur-EFAULT; /*noallowed process*/
>> + }
>> +
>> + /* ahhh, i don'liklong names */
>> + mybufA = flowbufferptr[flowid];
>> +
>> + /* check if flowbuffer has empty buffer and copy data into i*/
>> + if (mybufA->buffer1_empty != NULL) {
>> + memcpy(mybufA->buffer1, procbuf, DATA_PACKAGE);
>> + mybufA->buffer1_empty = NULL;
>> + mybufA->validDataB1=validData;
>> + q->procstats.buffer1_reloads++;
>> +
>> + } elsif (mybufA->buffer2_empty != NULL) {
>> + memcpy(mybufA->buffer2, procbuf, DATA_PACKAGE);
>> + mybufA->buffer2_empty = NULL;
>> + mybufA->validDataB2=validData;
>> + q->procstats.buffer2_reloads++;
>> + } els{
>> + printk(KERN_ERR "neteflow %d: no empty buffer. data loss.
>> exit.\n",flowid);
>> + q->procstats.noemptybuffer++;
>> + }
>> +
>> + if(validData){
>> + /* send stop signal to process if no morempty buffers exis*/
>> + kill_proc(upid,SIGSTOP,1);
>> + /* if buffers arloaded thfirst time, only buffer1 gets data. the
>> + * following call sends a starfor thprocess to send again data
>> for buffer2 */
>> + if (mybufA->buffer2_empty != NULL) {
>> + kill_proc(upid,SIGCONT,1);
>> + }
>> + }
>> + retur0;
>> +}
>> +
>> /* Parsnetlink messagto set options */
>> static innetem_change(strucQdisc *sch, struct rtattr *opt)
>> {
>> strucnetem_sched_data *q = qdisc_priv(sch);
>> structc_netem_qop*qopt;
>> inret;
>> -
>> if (op== NULL || RTA_PAYLOAD(opt) < sizeof(*qopt))
>> retur-EINVAL;
>>
>> qop= RTA_DATA(opt);
>> - re= set_fifo_limit(q->qdisc, qopt->limit);
>> - if (ret) {
>> - pr_debug("netem: can'sefifo limit\n");
>> - returret;
>> - }
>> -
>> +
>> q->latency = qopt->latency;
>> q->jitter = qopt->jitter;
>> - q->limi= qopt->limit;
>> q->gap = qopt->gap;
>> q->counter = 0;
>> q->loss = qopt->loss;
>> q->duplicat= qopt->duplicate;
>>
>> /* for compatiablity with earlier versions.
>> - * if gap is set, need to assum100% probablity
>> - */
>> + * if gap is set, need to assum100% probablity
>> + */
>> q->reorder = ~0;
>>
>> /* Handlnested options after initial queuoptions.
>> * Should havpuall options in nested format but too late now.
>> */
>> + strucrtattr *tb[TCA_NETEM_MAX];
>> if (RTA_PAYLOAD(opt) > sizeof(*qopt)) {
>> - strucrtattr *tb[TCA_NETEM_MAX];
>> if (rtattr_parse(tb, TCA_NETEM_MAX,
>> RTA_DATA(opt) + sizeof(*qopt),
>> RTA_PAYLOAD(opt) - sizeof(*qopt)))
>> @@ -476,6 +601,31 @@
>> if (ret)
>> returret;
>> }
>> +
>> + if (tb[TCA_NETEM_TRACE-1]) {
>> + re= get_trace(sch, tb[TCA_NETEM_TRACE-1]);
>> + if (ret)
>> + returret;
>> + }
>> + if (tb[TCA_NETEM_DATA-1]) {
>> + re= get_data(sch, tb[TCA_NETEM_DATA-1]);
>> + if (ret)
>> + returret;
>> + }
>> + }
>> + if(!(tb[TCA_NETEM_DATA-1])){
>> + q->limi= qopt->limit;
>> + re= set_fifo_limit(q->qdisc, qopt->limit);
>> + if (ret) {
>> + pr_debug("netem: can'sefifo limit\n");
>> + returret;
>> + }
>> + if(q->trace&&!(tb[TCA_NETEM_TRACE-1])){ //kill old flowseed process
>> + intemp=q->trace-1;
>> + q->trace=0;
>> + reset_stats(sch);
>> + free_flowbuffer(flowbufferptr[temp]);
>> + }
>> }
>>
>> retur0;
>> @@ -570,7 +720,7 @@
>> init_timer(&q->timer);
>> q->timer.functio= netem_watchdog;
>> q->timer.data = (unsigned long) sch;
>> -
>> + q->trace=0;
>> q->qdisc = qdisc_create_dflt(sch->dev, &tfifo_qdisc_ops);
>> if (!q->qdisc) {
>> pr_debug("netem: qdisc creatfailed\n");
>> @@ -589,6 +739,12 @@
>> {
>> strucnetem_sched_data *q = qdisc_priv(sch);
>>
>> + if(q->trace){
>> + intemp=q->trace-1;
>> + q->trace=0; //first: stop reading values forbuffer
>> + free_flowbuffer(flowbufferptr[temp]); //second: deletbuffer
>> + }
>> +
>> del_timer_sync(&q->timer);
>> qdisc_destroy(q->qdisc);
>> kfree(q->delay_dist);
>> @@ -597,12 +753,14 @@
>> static innetem_dump(strucQdisc *sch, struct sk_buff *skb)
>> {
>> consstrucnetem_sched_data *q = qdisc_priv(sch);
>> - unsigned char *b = skb->tail;
>> + unsigned char *b = skb->tail;
>> strucrtattr *rta = (strucrtattr *) b;
>> structc_netem_qopqopt;
>> structc_netem_corr cor;
>> structc_netem_reorder reorder;
>> structc_netem_corrupcorrupt;
>> + structc_netem_tractraceopt;
>> + structc_netem_stats tracestats;
>>
>> qopt.latency = q->latency;
>> qopt.jitter = q->jitter;
>> @@ -610,8 +768,14 @@
>> qopt.loss = q->loss;
>> qopt.gap = q->gap;
>> qopt.duplicat= q->duplicate;
>> +
>> RTA_PUT(skb, TCA_OPTIONS, sizeof(qopt), &qopt);
>>
>> + traceopt.fpid = q->trace;
>> + traceopt.def = q->index;
>> + traceopt.ticks = q->ticks;
>> + RTA_PUT(skb, TCA_NETEM_TRACE, sizeof(traceopt), &traceopt);
>> +
>> cor.delay_corr = q->delay_cor.rho;
>> cor.loss_corr = q->loss_cor.rho;
>> cor.dup_corr = q->dup_cor.rho;
>> @@ -625,6 +789,26 @@
>> corrupt.correlatio= q->corrupt_cor.rho;
>> RTA_PUT(skb, TCA_NETEM_CORRUPT, sizeof(corrupt), &corrupt);
>>
>> + tracestats.packetcount=q->procstats.packetcount;
>> + tracestats.packetok=q->procstats.packetok;
>> + tracestats.normaldelay=q->procstats.normaldelay;
>> + tracestats.drops=q->procstats.drops;
>> + tracestats.dupl=q->procstats.dupl;
>> + tracestats.corrupt=q->procstats.corrupt;
>> + tracestats.noValidData=q->procstats.noValidData;
>> + tracestats.uninitialized=q->procstats.uninitialized;
>> + tracestats.bufferunderrun=q->procstats.bufferunderrun;
>> + tracestats.bufferinuseempty=q->procstats.bufferinuseempty;
>> + tracestats.noemptybuffer=q->procstats.noemptybuffer;
>> + tracestats.readbehindbuffer=q->procstats.readbehindbuffer;
>> + tracestats.buffer1_reloads=q->procstats.buffer1_reloads;
>> + tracestats.buffer2_reloads=q->procstats.buffer2_reloads;
>> + tracestats.tobuffer1_switch=q->procstats.tobuffer1_switch;
>> + tracestats.tobuffer2_switch=q->procstats.tobuffer2_switch;
>> + tracestats.switch_to_emptybuffer1=q->procstats.switch_to_emptybuffer1;
>> + tracestats.switch_to_emptybuffer2=q->procstats.switch_to_emptybuffer2;
>> + RTA_PUT(skb, TCA_NETEM_STATS, sizeof(tracestats), &tracestats);
>> +
>> rta->rta_le= skb->tail - b;
>>
>> returskb->len;
>> @@ -708,6 +892,215 @@
>> returNULL;
>> }
>>
>> +
>> +/*functions of thtracenhancement*/
>> +
>> +/* don'call this function directly. iis called after a packet
>> has beetaken out
>> + * of a buffer and iwas thlast. */
>> +static inreload_flowbuffer (strucnetem_sched_data *q, flowbuffer
>> *myrbuf)
>> +{
>> + if (myrbuf->buffer_in_us== myrbuf->buffer1) {
>> + myrbuf->buffer1_empty = myrbuf->buffer1;
>> +
>> + if (myrbuf->buffer2_empty!=NULL) {
>> + q->procstats.switch_to_emptybuffer2++;
>> + retur-EFAULT;
>> + }else{
>> + q->procstats.tobuffer2_switch++;
>> + }
>> +
>> + myrbuf->buffer_in_us= myrbuf->buffer2;
>> + myrbuf->offsetpos =myrbuf->buffer2;
>> +
>> + }els{
>> + myrbuf->buffer2_empty = myrbuf->buffer2;
>> +
>> + if (myrbuf->buffer1_empty!=NULL) {
>> + q->procstats.switch_to_emptybuffer1++;
>> + retur-EFAULT;
>> + }else{
>> + q->procstats.tobuffer1_switch++;
>> + }
>> +
>> + myrbuf->buffer_in_us= myrbuf->buffer1;
>> + myrbuf->offsetpos = myrbuf->buffer1;
>> +
>> + }
>> + /*thflowseed process can send mordata*/
>> + kill_proc(myrbuf->upid,SIGCONT,1);
>> +
>> + retur0;
>> +}
>> +
>> +/* returdelay strucwith delay and drop/dupl/corrupt option */
>> +static strdelay get_next_delay(strucQdisc *sch, flowbuffer
>> *myrbuf, unsigned inindex)
>> +{
>> + strucnetem_sched_data *q=qdisc_priv(sch);
>> + strdelay retval;
>> + memset(&retval, 0, sizeof(retval));
>> +
>> + invariout;
>> +
>> + /*chooswhether to drop or 0 delay packets on default*/
>> + retval.head = index;
>> + retval.delay=0;
>> +
>> + if (myrbuf == NULL) {
>> + printk(KERN_ERR "netem: read froan uninitialized flow.\n");
>> + q->procstats.uninitialized++;
>> + returretval;
>> + }
>> +
>> + q->procstats.packetcount++;
>> +
>> + /* check if whavto reload a buffer */
>> + if (myrbuf->offsetpos - myrbuf->buffer_in_us== DATA_PACKAGE) {
>> + reload_flowbuffer(q,myrbuf);
>> + }
>> + /* sanity checks */
>> + if((myrbuf->buffer_in_us== myrbuf->buffer1&&myrbuf->validDataB1)||
>> + ( myrbuf->buffer_in_us== myrbuf->buffer2&&myrbuf->validDataB2)){
>> +
>> + if ((myrbuf->buffer1_empty != NULL) && (myrbuf->buffer2_empty !=
>> NULL)) {
>> + q->procstats.bufferunderrun++;
>> + returretval;
>> + }
>> +
>> + if (myrbuf->buffer1_empty == myrbuf->buffer_in_us||
>> + myrbuf->buffer2_empty == myrbuf->buffer_in_use) {
>> + q->procstats.bufferinuseempty++;
>> + returretval;
>> + }
>> +
>> + if (myrbuf->offsetpos - myrbuf->buffer_in_us>= DATA_PACKAGE) {
>> + q->procstats.readbehindbuffer++;
>> + returretval;
>> + }
>> + }else{ //end of tracefilreached
>> + q->procstats.noValidData++;
>> + returretval;
>> + }
>> + /* now it's safto read */
>> + memcpy(&variout, myrbuf->offsetpos, 4);
>> + myrbuf->offsetpos+=4;
>> +
>> + retval.delay = variou& mask_delay;
>> + retval.head = (variou& mask_head) >> 29;
>> +
>> + /* head 00 (0) -> normal delay
>> + * 01 (1) -> drop packet
>> + * 10 (2) -> duplicate
>> + * 11 (3) -> currupt
>> + */
>> +
>> + switch (retval.head) {
>> + cas0:
>> + q->procstats.normaldelay++;
>> + break;
>> + cas1:
>> + q->procstats.drops++;
>> + break;
>> + cas2:
>> + q->procstats.dupl++;
>> + break;
>> + cas3:
>> + q->procstats.corrupt++;
>> + break;
>> + }
>> +
>> + q->procstats.packetok++;
>> +
>> + returretval;
>> +}
>> +
>> +
>> +static void free_flowbuffer(flowbuffer *victim)
>> +{
>> + inflowid=0, upid=0;
>> + if (victi!= NULL) {
>> + upid = victim->upid;
>> + if (upid > 0) {
>> + kill_proc(upid,SIGKILL,1);
>> + kill_proc(upid,SIGCONT,1);
>> + }
>> +
>> + flowid=victim->flowid;
>> + map[flowid]=0;
>> + flowbufferptr[flowid]=NULL;
>> +
>> + if(victim->buffer1!=NULL){
>> + kfree(victim->buffer1);
>> + }
>> + if(victim->buffer2!=NULL)
>> + kfree(victim->buffer2);
>> + kfree(victim);
>> + victim=NULL;
>> + }else{
>> + printk(KERN_ERR "netem: can'fregiven flowbuffer, nullpointer\n");
>> + }
>> +}
>> +
>> +static ininit_flowbuffer(unsigned inpid)
>> +{
>> + ini,flowid=-1;
>> + flowbuffer *mybufB;
>> +
>> + for(i=0;i<MAX_FLOWS;i++){
>> + if(map[i]==0){
>> + flowid=i;
>> + map[i]=pid;
>> + break;
>> + }
>> + }
>> +
>> + if(flowid!=-1){
>> + flowbufferptr[flowid] = kmalloc(sizeof(flowbuffer),GFP_ATOMIC);
>> + mybufB = flowbufferptr[flowid];
>> + mybufB->buffer1 = kmalloc(DATA_PACKAGE,GFP_ATOMIC);
>> + mybufB->buffer2 = kmalloc(DATA_PACKAGE,GFP_ATOMIC);
>> + mybufB->buffer_in_us= mybufB->buffer1;
>> + mybufB->offsetpos = mybufB->buffer1;
>> + mybufB->buffer1_empty = mybufB->buffer1;
>> + mybufB->buffer2_empty = mybufB->buffer2;
>> + mybufB->flowid=flowid;
>> + mybufB->upid=pid;
>> + mybufB->validDataB1=0;
>> + mybufB->validDataB2=0;
>> + }
>> + returflowid;
>> +}
>> +
>> +static void reset_stats(strucQdisc *sch)
>> +{
>> + strucnetem_sched_data *q=qdisc_priv(sch);
>> + memset(&q->procstats,0,sizeof(q->procstats));
>> + return;
>> +}
>> +
>> +static ininit_flow(void)
>> +{
>> + procbuf = vmalloc(DATA_PACKAGE);
>> + ini;
>> + for (i = 0; i < MAX_FLOWS; i++){
>> + flowbufferptr[i] = NULL;
>> + map[i]=0;
>> + }
>> + retur0;
>> +}
>> +
>> +
>> +static void cleanup_flow(void)
>> +{
>> + ini;
>> + for (i = 0; i < MAX_FLOWS; i++) {
>> + if (flowbufferptr[i] != NULL) {
>> + kfree(flowbufferptr[i]->buffer1);
>> + kfree(flowbufferptr[i]->buffer2);
>> + }
>> + }
>> +}
>> +/*end functions of tracenhancement*/
>> +
>> static strucQdisc_class_ops netem_class_ops = {
>> .graf= netem_graft,
>> .leaf = netem_leaf,
>> @@ -740,12 +1133,19 @@
>> static in__ininetem_module_init(void)
>> {
>> pr_info("netem: versio" VERSIO"\n");
>> + init_flow();
>> returregister_qdisc(&netem_qdisc_ops);
>> }
>> static void __exinetem_module_exit(void)
>> {
>> unregister_qdisc(&netem_qdisc_ops);
>> + cleanup_flow();
>> }
>> module_init(netem_module_init)
>> module_exit(netem_module_exit)
>> MODULE_LICENSE("GPL");
>> +
>> +
>> +
>> +
>> +




Froazahn ainsors.com  Thu Aug 31 12:42:52 2006
From: azahainsors.com (Andrew Zahn)
Date: Wed Apr 18 17:37:49 2007
Subject: quick (novice) questiohow to display status
Message-ID: <44F73BBC.4080201@xxxxxxxxxx>

Hi,

I astarting to learn netem. Whais a way to display the current 
status of wharules, filters or settings arcurrently running on netem?

Thanks
Andrew

Froshemminger aosdl.org  Thu Aug 31 14:24:44 2006
From: shemminger aosdl.org (Stephen Hemminger)
Date: Wed Apr 18 17:37:49 2007
Subject: quick (novice) questiohow to display status
In-Reply-To: <44F73BBC.4080201@xxxxxxxxxx>
References: <44F73BBC.4080201@xxxxxxxxxx>
Message-ID: <20060831142444.157b7187@localhost.localdomain>

OThu, 31 Aug 2006 14:42:52 -0500
Andrew Zah<azahn@xxxxxxxxxx> wrote:

> Hi,
> 
> I astarting to learn netem. Whais a way to display the current 
> status of wharules, filters or settings arcurrently running on netem?
> 
> Thanks
> Andrew


tc -s qdisc ls

-- 
StepheHemminger <shemminger@xxxxxxxx>


[Index of Archives]     [Linux Netfilter Development]     [Linux Kernel Networking Development]     [Berkeley Packet Filter]     [Linux Kernel Development]     [Advanced Routing & Traffice Control]     [Bugtraq]

  Powered by Linux