I did so privately, but couldn't help thanking Martin publicly for the
detailed reply.
I'm sure not only Ewa, but also many other 'spectators' like me,
learned a awful lot from this thread.
Kudos !
G.
Gonçalo Luiz
On 6 April 2016 at 10:23, Ewa Janczukowicz <janczukowicz.ewa@xxxxxxxxx> wrote:
> Hello Martin,
>
> Thank you so much for taking time to answer my questions in such a detailed way.
>
> You have provided the information I was missing to understand how the
> priorities work, hence now I understand the importance of the quantum
> parameter, but also the behavior that I was observing during my HTB
> tests.
>
> For me quantum was mainly applicable in case of the same priorities,
> thus I have left it to default, i.e. calculated according to
> configured rates. But given the information you have provided, I
> definitely have a better understanding and I will see how I can adapt
> this parameter.
>
> Thank you again for your help.
>
> Have a great day,
> Ewa
>
>
>
>
>
>
>
> On Tue, Apr 5, 2016 at 7:41 PM, Martin A. Brown <martin@xxxxxxxxxxxx> wrote:
>>
>> Greetings Ewa,
>>
>> You seem to be exercising HTB quite well.
>>
>>>I have questions about how exactly priorities work in HTB. I have
>>>read the documentation and different websites, but it still appears
>>>not really clear to me, especially that I do not observe the
>>>“right” behavior.
>>
>> I think the 'prio' is one of the under-explained parameters to HTB.
>>
>>>I was trying to give an absolute priority to real-time traffic, but
>>>concurrent TCP traffic with lower priority was still sometimes
>>>taking over (even though its rate was limited to minimum so was
>>>getting almost nothing in the green mode since I know that queues
>>>in green mode are always sent before those in yellow).
>>
>> It's quite good that you have set the 'rate' on the concurrent TCP
>> traffic class to be quite low, because as you point out, green
>> classes always get to meet their Assured Rate, the HTB 'rate'
>> parameter.
>>
>> [
>>
>> For others who may be reading this message, here's a review of our
>> references to class colors:
>>
>> * green: the current dequeue rate of an HTB class is below its
>> 'rate' (in the theory, this is called 'Assured Rate' [1])
>>
>> * yellow: the current dequeue rate of an HTB class is between
>> its 'rate' and its 'ceil'
>>
>> * red: the current dequeue rate of an HTB class equals or exceeds
>> the 'ceil' (it is possible to go slightly over 'ceil' when
>> borrowing and dequeuing 'quantum' bytes)
>> ]
>>
>> My main (somewhat rhetorical) question in reply to you is:
>>
>> How large is the quantum?
>>
>> In the rest of my reply, I will try to describe the interaction
>> between the 'quantum' and 'prio' parameters in HTB operation.
>>
>>>Thus I would like to ask how exactly the priority is treated in
>>>HTB.
>>>
>> >From the documentation, I understand that in case of the same
>>>priority, the quantum and DRR is used. So no problem here.
>>
>> Here's are some framing rules, that may help (it's quite likely to
>> be review for you).
>>
>> * each class is either a leaf class or an ancestor class (a.k.a.
>> inner class)
>>
>> * leaf classes enqueue, hold and dequeue packets (they do all of
>> the actual work)
>>
>> * ancestor classes only offer control over borrowing; they hold no
>> packets, but rather control the distribution of shared resources
>>
>> * the 'prio' and 'quantum' parameters are only used in leaf
>> classes ('prio' and 'quantum' have no real meaning in parent
>> classes, even if they accept the parameters)
>>
>> * lower numbered values for the 'prio' of a class represent higher
>> priority
>>
>> * borrowing does not kick in until a leaf class has hit its
>> 'rate' (when the class becomes yellow)
>>
>> * for ancestor classes, the 'rate' and 'ceil' parameters are only
>> used to determine when that ancestor can lend to children (I
>> think that only 'ceil' is used, maybe others who know can
>> comment authoritatively)
>>
>> * the HTB qdisc will only service the leaf classes looking for
>> packets to dequeue if at least one leaf is green or if there are
>> ancestors are capable of lending tokens
>>
>> At any point, either there are tokens available, in which case, HTB
>> can send, or there are no tokens available, in which case HTB waits
>> until tokens are available again. Here are a couple of (the many)
>> possible states for an HTB tree:
>>
>> * all leaves in an HTB tree may be completely 'red'; every leaf
>> class is at its 'ceil'; no packets will be dequeued
>>
>> * all (or some) leaves in an HTB tree are 'yellow', but all
>> ancestor classes are 'red'; no packets dequeued
>>
>> * all (or some) leaves in an HTB tree are 'yellow', and some
>> ancestor classes are 'green' or 'yellow'; HTB will allow leaf
>> classes to borrow from ancestors and dequeue packets up to
>> 'quantum' bytes in 'prio' order
>>
>> * all (or some) leaves in an HTB tree are 'green' (ancestor
>> classes don't matter here); allow leaf classes to dequeue
>> packets up to 'quantum' bytes in 'prio' order
>>
>> Here's another way to look at it. This is what HTB does in an
>> infinite loop (of course, this is an inexact approximation in
>> pseudo-code):
>>
>> while exist(green_classes)
>> for leaf in ordered_by_prio(green_classes)
>> dequeue_and_charge_parents(leaf, quantum_bytes)
>>
>> while exist(yellow_classes)
>> for leaf in ordered_by_prio(yellow_classes)
>> if tokens_available(ancestors(leaf))
>> dequeue_and_charge_parents(leaf, quantum_bytes)
>>
>> So, as you can see, any leaf class that has not spent its 'ceil' yet
>> gets an opportunity to send 'quantum' bytes, during each time it is
>> possible to send. The above is, of course, an approximation, since
>> packets can arrive at any time to any class....but, this is roughly
>> how HTB behaves.
>>
>> And, now to try to answer your questions directly.
>>
>>>However, in case of queues with different priorities (but on the
>>>same level), how is it exactly divided?
>>
>> All leaf classes are visited in order (by 'prio'). Each leaf class
>> gets to send up to 'quantum' bytes. Then, the next leaf class gets
>> an opportunity. And, this is the magic of 'quantum'.
>>
>> The 'quantum' parameter has significant possible effects. By
>> setting the 'quantum' parameter really large on one class, you can
>> starve other classes. But, by setting the 'quantum' very low, you
>> can cause more work, because HTB has to visit each class more times
>> to dequeue the same number of bytes (packets).
>>
>> You must balance the possibility of sibling starvation against the
>> amount of work incurred.
>>
>> For reference, HTB clamps the 'quantum' values at the following
>> minima and maxima (from 1489ff. in sch_htb.c [0]):
>>
>> cl->quantum = 1000; # -- minimum quantum
>> cl->quantum = 200000; # -- maximum quantum
>>
>> If you have not set the 'quantum' on any of your classes, then it is
>> calculated sanely for you based on the 'rate'.
>>
>>>Does the highest priority send packets as long as its queue is full
>>>or also only for a given quantum?
>>
>> Hopefully the above explanation answers your question. The highest
>> priority leaf class does not get to send as long as its queue is
>> full. It gets to send 'quantum' bytes for each round of service,
>> but that highest priority leaf class will always get the first
>> opportunity to send and to borrow from ancestors.
>>
>>>If it sends as long as its queue is full, how come the traffic with
>>>lower priority does not get stalled?
>>
>> And, this is precisely why the 'quantum' parameter exists. It's not
>> really that important when classes are green, but as soon as classes
>> turn yellow, it is the interaction of 'prio' and 'quantum' which
>> offers us control over resource sharing among siblings.
>>
>> With 'quantum', you get to determine how much each yellow sibling
>> class gets to transmit at a single service. With 'prio', you
>> determine which sibling transmits first.
>>
>> Analogy: If your higher priority sister has a prodigious appetite
>> (large 'quantum' value), then she may eat most of the available
>> stew! THe horrible beast! But, maybe in a different world, your
>> higher priority sister eats only enough to keep a bird alive (small
>> 'quantum' value). Considerate sister! In that case, there's plenty
>> of stew left for you and even your lower priority siblings.
>>
>>>Additionally, is this behavior the same when both queues are in
>>>yellow and in green mode (thus at parent and leaf level)?
>>
>> Also, with the reference to the yellow and green, I see that you
>> have read Martin Devera's theory page [1] on HTB! Wonderfully
>> helpful and dense stuff!
>>
>> Have a wonderful day,
>>
>> -Martin
>>
>> [0] http://lxr.free-electrons.com/source/net/sched/sch_htb.c#L1489
>> [1] http://luxik.cdi.cz/~devik/qos/htb/manual/theory.htm
>>
>> --
>> Martin A. Brown
>> http://linux-ip.net/
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
