Re: details about prioritization in HTB

Linux Advanced Routing and Traffic Control

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I did so privately, but couldn't help thanking Martin publicly for the
detailed reply.

I'm sure not only Ewa, but also many other 'spectators' like me,
learned a awful lot from this thread.

Kudos !

G.
Gonçalo Luiz


On 6 April 2016 at 10:23, Ewa Janczukowicz <janczukowicz.ewa@xxxxxxxxx> wrote:
> Hello Martin,
>
> Thank you so much for taking time to answer my questions in such a detailed way.
>
> You have provided the information I was missing to understand how the
> priorities work, hence now I understand the importance of the quantum
> parameter, but also the behavior that I was observing during my HTB
> tests.
>
> For me quantum was mainly applicable in case of the same priorities,
> thus I have left it to default, i.e. calculated according to
> configured rates. But given the information you have provided, I
> definitely have a better understanding and I will see how I can adapt
> this parameter.
>
> Thank you again for your help.
>
> Have a great day,
> Ewa
>
>
>
>
>
>
>
> On Tue, Apr 5, 2016 at 7:41 PM, Martin A. Brown <martin@xxxxxxxxxxxx> wrote:
>>
>> Greetings Ewa,
>>
>> You seem to be exercising HTB quite well.
>>
>>>I have questions about how exactly priorities work in HTB. I have
>>>read the documentation and different websites, but it still appears
>>>not really clear to me, especially that I do not observe the
>>>“right” behavior.
>>
>> I think the 'prio' is one of the under-explained parameters to HTB.
>>
>>>I was trying to give an absolute priority to real-time traffic, but
>>>concurrent TCP traffic with lower priority was still sometimes
>>>taking over (even though its rate was limited to minimum so was
>>>getting almost nothing in the green mode since I know that queues
>>>in green mode are always sent before those in yellow).
>>
>> It's quite good that you have set the 'rate' on the concurrent TCP
>> traffic class to be quite low, because as you point out, green
>> classes always get to meet their Assured Rate, the HTB 'rate'
>> parameter.
>>
>> [
>>
>> For others who may be reading this message, here's a review of our
>> references to class colors:
>>
>>   * green:  the current dequeue rate of an HTB class is below its
>>     'rate' (in the theory, this is called 'Assured Rate' [1])
>>
>>   * yellow: the current dequeue rate of an HTB class is between
>>     its 'rate' and its 'ceil'
>>
>>   * red:  the current dequeue rate of an HTB class equals or exceeds
>>     the 'ceil' (it is possible to go slightly over 'ceil' when
>>     borrowing and dequeuing 'quantum' bytes)
>> ]
>>
>> My main (somewhat rhetorical) question in reply to you is:
>>
>>   How large is the quantum?
>>
>> In the rest of my reply, I will try to describe the interaction
>> between the 'quantum' and 'prio' parameters in HTB operation.
>>
>>>Thus I would like to ask how exactly the priority is treated in
>>>HTB.
>>>
>> >From the documentation, I understand that in case of the same
>>>priority, the quantum and DRR is used. So no problem here.
>>
>> Here's are some framing rules, that may help (it's quite likely to
>> be review for you).
>>
>>   * each class is either a leaf class or an ancestor class (a.k.a.
>>     inner class)
>>
>>   * leaf classes enqueue, hold and dequeue packets (they do all of
>>     the actual work)
>>
>>   * ancestor classes only offer control over borrowing; they hold no
>>     packets, but rather control the distribution of shared resources
>>
>>   * the 'prio' and 'quantum' parameters are only used in leaf
>>     classes ('prio' and 'quantum' have no real meaning in parent
>>     classes, even if they accept the parameters)
>>
>>   * lower numbered values for the 'prio' of a class represent higher
>>     priority
>>
>>   * borrowing does not kick in until a leaf class has hit its
>>     'rate' (when the class becomes yellow)
>>
>>   * for ancestor classes, the 'rate' and 'ceil' parameters are only
>>     used to determine when that ancestor can lend to children (I
>>     think that only 'ceil' is used, maybe others who know can
>>     comment authoritatively)
>>
>>   * the HTB qdisc will only service the leaf classes looking for
>>     packets to dequeue if at least one leaf is green or if there are
>>     ancestors are capable of lending tokens
>>
>> At any point, either there are tokens available, in which case, HTB
>> can send, or there are no tokens available, in which case HTB waits
>> until tokens are available again.  Here are a couple of (the many)
>> possible states for an HTB tree:
>>
>>   * all leaves in an HTB tree may be completely 'red'; every leaf
>>     class is at its 'ceil'; no packets will be dequeued
>>
>>   * all (or some) leaves in an HTB tree are 'yellow', but all
>>     ancestor classes are 'red'; no packets dequeued
>>
>>   * all (or some) leaves in an HTB tree are 'yellow', and some
>>     ancestor classes are 'green' or 'yellow'; HTB will allow leaf
>>     classes to borrow from ancestors and dequeue packets up to
>>     'quantum' bytes in 'prio' order
>>
>>   * all (or some) leaves in an HTB tree are 'green' (ancestor
>>     classes don't matter here); allow leaf classes to dequeue
>>     packets up to 'quantum' bytes in 'prio' order
>>
>> Here's another way to look at it.  This is what HTB does in an
>> infinite loop (of course, this is an inexact approximation in
>> pseudo-code):
>>
>>   while exist(green_classes)
>>       for leaf in ordered_by_prio(green_classes)
>>           dequeue_and_charge_parents(leaf, quantum_bytes)
>>
>>   while exist(yellow_classes)
>>       for leaf in ordered_by_prio(yellow_classes)
>>           if tokens_available(ancestors(leaf))
>>               dequeue_and_charge_parents(leaf, quantum_bytes)
>>
>> So, as you can see, any leaf class that has not spent its 'ceil' yet
>> gets an opportunity to send 'quantum' bytes, during each time it is
>> possible to send.  The above is, of course, an approximation, since
>> packets can arrive at any time to any class....but, this is roughly
>> how HTB behaves.
>>
>> And, now to try to answer your questions directly.
>>
>>>However, in case of queues with different priorities (but on the
>>>same level), how is it exactly divided?
>>
>> All leaf classes are visited in order (by 'prio').  Each leaf class
>> gets to send up to 'quantum' bytes.  Then, the next leaf class gets
>> an opportunity.  And, this is the magic of 'quantum'.
>>
>> The 'quantum' parameter has significant possible effects.  By
>> setting the 'quantum' parameter really large on one class, you can
>> starve other classes.  But, by setting the 'quantum' very low, you
>> can cause more work, because HTB has to visit each class more times
>> to dequeue the same number of bytes (packets).
>>
>> You must balance the possibility of sibling starvation against the
>> amount of work incurred.
>>
>> For reference, HTB clamps the 'quantum' values at the following
>> minima and maxima (from 1489ff. in sch_htb.c [0]):
>>
>>   cl->quantum = 1000;      # -- minimum quantum
>>   cl->quantum = 200000;    # -- maximum quantum
>>
>> If you have not set the 'quantum' on any of your classes, then it is
>> calculated sanely for you based on the 'rate'.
>>
>>>Does the highest priority send packets as long as its queue is full
>>>or also only for a given quantum?
>>
>> Hopefully the above explanation answers your question.  The highest
>> priority leaf class does not get to send as long as its queue is
>> full.  It gets to send 'quantum' bytes for each round of service,
>> but that highest priority leaf class will always get the first
>> opportunity to send and to borrow from ancestors.
>>
>>>If it sends as long as its queue is full, how come the traffic with
>>>lower priority does not get stalled?
>>
>> And, this is precisely why the 'quantum' parameter exists.  It's not
>> really that important when classes are green, but as soon as classes
>> turn yellow, it is the interaction of 'prio' and 'quantum' which
>> offers us control over resource sharing among siblings.
>>
>> With 'quantum', you get to determine how much each yellow sibling
>> class gets to transmit at a single service.  With 'prio', you
>> determine which sibling transmits first.
>>
>> Analogy:  If your higher priority sister has a prodigious appetite
>> (large 'quantum' value), then she may eat most of the available
>> stew!  THe horrible beast!  But, maybe in a different world, your
>> higher priority sister eats only enough to keep a bird alive (small
>> 'quantum' value).  Considerate sister!  In that case, there's plenty
>> of stew left for you and even your lower priority siblings.
>>
>>>Additionally, is this behavior the same when both queues are in
>>>yellow and in green mode (thus at parent and leaf level)?
>>
>> Also, with the reference to the yellow and green, I see that you
>> have read Martin Devera's theory page [1] on HTB!  Wonderfully
>> helpful and dense stuff!
>>
>> Have a wonderful day,
>>
>> -Martin
>>
>>  [0] http://lxr.free-electrons.com/source/net/sched/sch_htb.c#L1489
>>  [1] http://luxik.cdi.cz/~devik/qos/htb/manual/theory.htm
>>
>> --
>> Martin A. Brown
>> http://linux-ip.net/
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [LARTC Home Page]     [Netfilter]     [Netfilter Development]     [Network Development]     [Bugtraq]     [GCC Help]     [Yosemite News]     [Linux Kernel]     [Fedora Users]
  Powered by Linux