I did so privately, but couldn't help thanking Martin publicly for the detailed reply. I'm sure not only Ewa, but also many other 'spectators' like me, learned a awful lot from this thread. Kudos ! G. Gonçalo Luiz On 6 April 2016 at 10:23, Ewa Janczukowicz <janczukowicz.ewa@xxxxxxxxx> wrote: > Hello Martin, > > Thank you so much for taking time to answer my questions in such a detailed way. > > You have provided the information I was missing to understand how the > priorities work, hence now I understand the importance of the quantum > parameter, but also the behavior that I was observing during my HTB > tests. > > For me quantum was mainly applicable in case of the same priorities, > thus I have left it to default, i.e. calculated according to > configured rates. But given the information you have provided, I > definitely have a better understanding and I will see how I can adapt > this parameter. > > Thank you again for your help. > > Have a great day, > Ewa > > > > > > > > On Tue, Apr 5, 2016 at 7:41 PM, Martin A. Brown <martin@xxxxxxxxxxxx> wrote: >> >> Greetings Ewa, >> >> You seem to be exercising HTB quite well. >> >>>I have questions about how exactly priorities work in HTB. I have >>>read the documentation and different websites, but it still appears >>>not really clear to me, especially that I do not observe the >>>“right” behavior. >> >> I think the 'prio' is one of the under-explained parameters to HTB. >> >>>I was trying to give an absolute priority to real-time traffic, but >>>concurrent TCP traffic with lower priority was still sometimes >>>taking over (even though its rate was limited to minimum so was >>>getting almost nothing in the green mode since I know that queues >>>in green mode are always sent before those in yellow). >> >> It's quite good that you have set the 'rate' on the concurrent TCP >> traffic class to be quite low, because as you point out, green >> classes always get to meet their Assured Rate, the HTB 'rate' >> parameter. >> >> [ >> >> For others who may be reading this message, here's a review of our >> references to class colors: >> >> * green: the current dequeue rate of an HTB class is below its >> 'rate' (in the theory, this is called 'Assured Rate' [1]) >> >> * yellow: the current dequeue rate of an HTB class is between >> its 'rate' and its 'ceil' >> >> * red: the current dequeue rate of an HTB class equals or exceeds >> the 'ceil' (it is possible to go slightly over 'ceil' when >> borrowing and dequeuing 'quantum' bytes) >> ] >> >> My main (somewhat rhetorical) question in reply to you is: >> >> How large is the quantum? >> >> In the rest of my reply, I will try to describe the interaction >> between the 'quantum' and 'prio' parameters in HTB operation. >> >>>Thus I would like to ask how exactly the priority is treated in >>>HTB. >>> >> >From the documentation, I understand that in case of the same >>>priority, the quantum and DRR is used. So no problem here. >> >> Here's are some framing rules, that may help (it's quite likely to >> be review for you). >> >> * each class is either a leaf class or an ancestor class (a.k.a. >> inner class) >> >> * leaf classes enqueue, hold and dequeue packets (they do all of >> the actual work) >> >> * ancestor classes only offer control over borrowing; they hold no >> packets, but rather control the distribution of shared resources >> >> * the 'prio' and 'quantum' parameters are only used in leaf >> classes ('prio' and 'quantum' have no real meaning in parent >> classes, even if they accept the parameters) >> >> * lower numbered values for the 'prio' of a class represent higher >> priority >> >> * borrowing does not kick in until a leaf class has hit its >> 'rate' (when the class becomes yellow) >> >> * for ancestor classes, the 'rate' and 'ceil' parameters are only >> used to determine when that ancestor can lend to children (I >> think that only 'ceil' is used, maybe others who know can >> comment authoritatively) >> >> * the HTB qdisc will only service the leaf classes looking for >> packets to dequeue if at least one leaf is green or if there are >> ancestors are capable of lending tokens >> >> At any point, either there are tokens available, in which case, HTB >> can send, or there are no tokens available, in which case HTB waits >> until tokens are available again. Here are a couple of (the many) >> possible states for an HTB tree: >> >> * all leaves in an HTB tree may be completely 'red'; every leaf >> class is at its 'ceil'; no packets will be dequeued >> >> * all (or some) leaves in an HTB tree are 'yellow', but all >> ancestor classes are 'red'; no packets dequeued >> >> * all (or some) leaves in an HTB tree are 'yellow', and some >> ancestor classes are 'green' or 'yellow'; HTB will allow leaf >> classes to borrow from ancestors and dequeue packets up to >> 'quantum' bytes in 'prio' order >> >> * all (or some) leaves in an HTB tree are 'green' (ancestor >> classes don't matter here); allow leaf classes to dequeue >> packets up to 'quantum' bytes in 'prio' order >> >> Here's another way to look at it. This is what HTB does in an >> infinite loop (of course, this is an inexact approximation in >> pseudo-code): >> >> while exist(green_classes) >> for leaf in ordered_by_prio(green_classes) >> dequeue_and_charge_parents(leaf, quantum_bytes) >> >> while exist(yellow_classes) >> for leaf in ordered_by_prio(yellow_classes) >> if tokens_available(ancestors(leaf)) >> dequeue_and_charge_parents(leaf, quantum_bytes) >> >> So, as you can see, any leaf class that has not spent its 'ceil' yet >> gets an opportunity to send 'quantum' bytes, during each time it is >> possible to send. The above is, of course, an approximation, since >> packets can arrive at any time to any class....but, this is roughly >> how HTB behaves. >> >> And, now to try to answer your questions directly. >> >>>However, in case of queues with different priorities (but on the >>>same level), how is it exactly divided? >> >> All leaf classes are visited in order (by 'prio'). Each leaf class >> gets to send up to 'quantum' bytes. Then, the next leaf class gets >> an opportunity. And, this is the magic of 'quantum'. >> >> The 'quantum' parameter has significant possible effects. By >> setting the 'quantum' parameter really large on one class, you can >> starve other classes. But, by setting the 'quantum' very low, you >> can cause more work, because HTB has to visit each class more times >> to dequeue the same number of bytes (packets). >> >> You must balance the possibility of sibling starvation against the >> amount of work incurred. >> >> For reference, HTB clamps the 'quantum' values at the following >> minima and maxima (from 1489ff. in sch_htb.c [0]): >> >> cl->quantum = 1000; # -- minimum quantum >> cl->quantum = 200000; # -- maximum quantum >> >> If you have not set the 'quantum' on any of your classes, then it is >> calculated sanely for you based on the 'rate'. >> >>>Does the highest priority send packets as long as its queue is full >>>or also only for a given quantum? >> >> Hopefully the above explanation answers your question. The highest >> priority leaf class does not get to send as long as its queue is >> full. It gets to send 'quantum' bytes for each round of service, >> but that highest priority leaf class will always get the first >> opportunity to send and to borrow from ancestors. >> >>>If it sends as long as its queue is full, how come the traffic with >>>lower priority does not get stalled? >> >> And, this is precisely why the 'quantum' parameter exists. It's not >> really that important when classes are green, but as soon as classes >> turn yellow, it is the interaction of 'prio' and 'quantum' which >> offers us control over resource sharing among siblings. >> >> With 'quantum', you get to determine how much each yellow sibling >> class gets to transmit at a single service. With 'prio', you >> determine which sibling transmits first. >> >> Analogy: If your higher priority sister has a prodigious appetite >> (large 'quantum' value), then she may eat most of the available >> stew! THe horrible beast! But, maybe in a different world, your >> higher priority sister eats only enough to keep a bird alive (small >> 'quantum' value). Considerate sister! In that case, there's plenty >> of stew left for you and even your lower priority siblings. >> >>>Additionally, is this behavior the same when both queues are in >>>yellow and in green mode (thus at parent and leaf level)? >> >> Also, with the reference to the yellow and green, I see that you >> have read Martin Devera's theory page [1] on HTB! Wonderfully >> helpful and dense stuff! >> >> Have a wonderful day, >> >> -Martin >> >> [0] http://lxr.free-electrons.com/source/net/sched/sch_htb.c#L1489 >> [1] http://luxik.cdi.cz/~devik/qos/htb/manual/theory.htm >> >> -- >> Martin A. Brown >> http://linux-ip.net/ > -- > To unsubscribe from this list: send the line "unsubscribe lartc" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe lartc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html