I did a number of tests and there doesn't appear to be any noticeable differences between using CPU and JIFFIES (HZ=1000) as packet scheduler source. I didn't mention that the outgoing interface on the core router has 2 ip addresses. One is vlan tagged for the test network im running and the other is not tagged, so the vast majority of outgoing traffic will be tagged on the outgoing interface. Question 1. When sending a packet on a tagged interface, the 4 byte vlan header is added between the source address and the ethertype field. As far as I know the tag is added and removed in the device driver and so it does not show up when using ethereal or tcpdump. The driver in use is the e1000 which supports VLANS. As a result the minimum pkt size for tagged packets is 68bytes and maximum is 1522 bytes. When using MPU and overhead in htb, does that mean that the overhead should be increased to 42 (38 + 4 for vlan header) and MPU = 88 (42 + 46). There are both tagged and untagged packets frames on this interface. I have run tests using MPU of 84 and overhead of 38 and mpu 88 and overhead of 42 and increasing the ceil to 90, 95, 97 and 99Mbit. There are spikes in the delay and the higher I increase the ceil, the higher the spikes are. I have graphs of the tests I done. The filename of the graph should explain the settings that were used in the test. Hysteresis is set to 0, CPU as the packet scheduler clock source, mpu and overhead ( 38 and 84 respectively) or vlan mpu and overhead (42 and 88 respectively) and then the ceiling at either 90,95 or 99 mbit. 21957_hysteresis_CPU_mpu_overhead_95_ceil.png 22762_hysteresis_CPU_vlan_mpu_overhead_90_ceil.png 22875_hysteresis_CPU_vlan_mpu_overhead_99_ceil.png 24135_hysteresis_CPU_mpu_overhead_90_ceil.png 24143_hysteresis_CPU_vlan_mpu_overhead_95_ceil.png 24262_hysteresis_CPU_mpu_overhead_99_ceil.png They can be found at http://www.compsoc.nuigalway.ie/~jlynch/htb/ MPU and overhead seems to be used mainly in places where the size of frames are fixed. Does it make a difference using it with Ethernet where the frame size is variable ? When you said you could ceil at 99mbit ok did you look at the max delay ? Did u notice spikes like what are in my graphs ? Do you have any idea what could be causing these spikes ?? Other observations: I was using values from /proc/net/dev to measure the throughput going through the core router and I noticed that different network drivers increment the packet counters differently. (ie for e1000 for a packet with max Ethernet payload of 1500 bytes byte counter incremented by 1518 bytes which includes eth header and trailer. 3c59x on the other hand increments by 1514 bytes which does not include the eth trailer. For vlan tagged packets e1000 increments byte counter by 1522 bytes, 3c59x increments by 1518 bytes. Have you come across this before ? Also according to the source of tc, specifically in tc_util.c it refers to this page http://physics.nist.gov/cuu/Units/binary.html as to how rates are specified in tc. So i presume tc uses this to specify rate, ceil etc. This doesnt seem to be mentioned anywhere. static const struct rate_suffix { const char *name; double scale; } suffixes[] = { { "bit", 1. }, { "Kibit", 1024. }, { "kbit", 1000. }, { "mibit", 1024.*1024. }, { "mbit", 1000000. }, { "gibit", 1024.*1024.*1024. }, { "gbit", 1000000000. }, { "tibit", 1024.*1024.*1024.*1024. }, { "tbit", 1000000000000. }, { "Bps", 8. }, { "KiBps", 8.*1024. }, { "KBps", 8000. }, { "MiBps", 8.*1024*1024. }, { "MBps", 8000000. }, { "GiBps", 8.*1024.*1024.*1024. }, { "GBps", 8000000000. }, { "TiBps", 8.*1024.*1024.*1024.*1024. }, { "TBps", 8000000000000. }, { NULL } }; Jonathan On Thu, 2005-08-11 at 17:36 +0100, Andy Furniss wrote: > Jonathan Lynch wrote: > > > > > I dont understand what you mean when you say "if you could sample truly > > randomly you would get a proper distribution". > > > > Also having the timers synchronized will allow for more accurate > > measurements of the delay. I cant see how this would have an impact on > > the pattern. > > I mean it's possibly just to do with the test if a 0ms - 1ms delay is > expected then you could see patterns arising depending on how you > measure delay/clock drift or something. > > Now I have two pings that do intervels < 1 sec - the inetutils GNU ping > guys implemented it for me :-), and I also have the iputils one I can > simulate a stream better. > > While doing this I noticed that iputils ping actually gives lower > latency readings when sending many pps. Using tcpdump deltas I can see > the network latency is the same however many pps I do - it's just that > when measuring <1ms delays and doing many pps it seems that some code > gets cached (guess) and the reported delay changes as a result. > > I mention that just to illustrate that measuring small delays can be > misleading and influenced by the exact nature of your setup. > > > > > > > I have the default setting which is to set it to jiffies. There is a > > comment in the kernal config for Packet scheduler clock source that > > mentions that Jiffies "its resolution is too low for accurate shaping > > except at very low speed". I will recompile the kernel and try the CPU > > option tomorrow to see if there is any change. > > Maybe not in the case of htb - I use CPU and see similar results, the > comment about accurate shaping was probably written when HZ=100, but I > suppose it will be better for something :-) > > > > > > > I didnt realise such options existed for htb (mpu + overhead). These parameters are not mentioned in the man pages or in the htb manual. > > I presume I have to patch tc to get these features ?. > > > There is mention on the htb page - it was added as a patch so was not > designed in, which explains why burst doesn't use it. > > You don't need to patch recent iproute2 it's already in there. > > Andy. _______________________________________________ LARTC mailing list LARTC@xxxxxxxxxxxxxxx http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
