Re: details about prioritization in HTB

Linux Advanced Routing and Traffic Control

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello Martin,

Thank you so much for taking time to answer my questions in such a detailed way.

You have provided the information I was missing to understand how the
priorities work, hence now I understand the importance of the quantum
parameter, but also the behavior that I was observing during my HTB
tests.

For me quantum was mainly applicable in case of the same priorities,
thus I have left it to default, i.e. calculated according to
configured rates. But given the information you have provided, I
definitely have a better understanding and I will see how I can adapt
this parameter.

Thank you again for your help.

Have a great day,
Ewa







On Tue, Apr 5, 2016 at 7:41 PM, Martin A. Brown <martin@xxxxxxxxxxxx> wrote:
>
> Greetings Ewa,
>
> You seem to be exercising HTB quite well.
>
>>I have questions about how exactly priorities work in HTB. I have
>>read the documentation and different websites, but it still appears
>>not really clear to me, especially that I do not observe the
>>“right” behavior.
>
> I think the 'prio' is one of the under-explained parameters to HTB.
>
>>I was trying to give an absolute priority to real-time traffic, but
>>concurrent TCP traffic with lower priority was still sometimes
>>taking over (even though its rate was limited to minimum so was
>>getting almost nothing in the green mode since I know that queues
>>in green mode are always sent before those in yellow).
>
> It's quite good that you have set the 'rate' on the concurrent TCP
> traffic class to be quite low, because as you point out, green
> classes always get to meet their Assured Rate, the HTB 'rate'
> parameter.
>
> [
>
> For others who may be reading this message, here's a review of our
> references to class colors:
>
>   * green:  the current dequeue rate of an HTB class is below its
>     'rate' (in the theory, this is called 'Assured Rate' [1])
>
>   * yellow: the current dequeue rate of an HTB class is between
>     its 'rate' and its 'ceil'
>
>   * red:  the current dequeue rate of an HTB class equals or exceeds
>     the 'ceil' (it is possible to go slightly over 'ceil' when
>     borrowing and dequeuing 'quantum' bytes)
> ]
>
> My main (somewhat rhetorical) question in reply to you is:
>
>   How large is the quantum?
>
> In the rest of my reply, I will try to describe the interaction
> between the 'quantum' and 'prio' parameters in HTB operation.
>
>>Thus I would like to ask how exactly the priority is treated in
>>HTB.
>>
> >From the documentation, I understand that in case of the same
>>priority, the quantum and DRR is used. So no problem here.
>
> Here's are some framing rules, that may help (it's quite likely to
> be review for you).
>
>   * each class is either a leaf class or an ancestor class (a.k.a.
>     inner class)
>
>   * leaf classes enqueue, hold and dequeue packets (they do all of
>     the actual work)
>
>   * ancestor classes only offer control over borrowing; they hold no
>     packets, but rather control the distribution of shared resources
>
>   * the 'prio' and 'quantum' parameters are only used in leaf
>     classes ('prio' and 'quantum' have no real meaning in parent
>     classes, even if they accept the parameters)
>
>   * lower numbered values for the 'prio' of a class represent higher
>     priority
>
>   * borrowing does not kick in until a leaf class has hit its
>     'rate' (when the class becomes yellow)
>
>   * for ancestor classes, the 'rate' and 'ceil' parameters are only
>     used to determine when that ancestor can lend to children (I
>     think that only 'ceil' is used, maybe others who know can
>     comment authoritatively)
>
>   * the HTB qdisc will only service the leaf classes looking for
>     packets to dequeue if at least one leaf is green or if there are
>     ancestors are capable of lending tokens
>
> At any point, either there are tokens available, in which case, HTB
> can send, or there are no tokens available, in which case HTB waits
> until tokens are available again.  Here are a couple of (the many)
> possible states for an HTB tree:
>
>   * all leaves in an HTB tree may be completely 'red'; every leaf
>     class is at its 'ceil'; no packets will be dequeued
>
>   * all (or some) leaves in an HTB tree are 'yellow', but all
>     ancestor classes are 'red'; no packets dequeued
>
>   * all (or some) leaves in an HTB tree are 'yellow', and some
>     ancestor classes are 'green' or 'yellow'; HTB will allow leaf
>     classes to borrow from ancestors and dequeue packets up to
>     'quantum' bytes in 'prio' order
>
>   * all (or some) leaves in an HTB tree are 'green' (ancestor
>     classes don't matter here); allow leaf classes to dequeue
>     packets up to 'quantum' bytes in 'prio' order
>
> Here's another way to look at it.  This is what HTB does in an
> infinite loop (of course, this is an inexact approximation in
> pseudo-code):
>
>   while exist(green_classes)
>       for leaf in ordered_by_prio(green_classes)
>           dequeue_and_charge_parents(leaf, quantum_bytes)
>
>   while exist(yellow_classes)
>       for leaf in ordered_by_prio(yellow_classes)
>           if tokens_available(ancestors(leaf))
>               dequeue_and_charge_parents(leaf, quantum_bytes)
>
> So, as you can see, any leaf class that has not spent its 'ceil' yet
> gets an opportunity to send 'quantum' bytes, during each time it is
> possible to send.  The above is, of course, an approximation, since
> packets can arrive at any time to any class....but, this is roughly
> how HTB behaves.
>
> And, now to try to answer your questions directly.
>
>>However, in case of queues with different priorities (but on the
>>same level), how is it exactly divided?
>
> All leaf classes are visited in order (by 'prio').  Each leaf class
> gets to send up to 'quantum' bytes.  Then, the next leaf class gets
> an opportunity.  And, this is the magic of 'quantum'.
>
> The 'quantum' parameter has significant possible effects.  By
> setting the 'quantum' parameter really large on one class, you can
> starve other classes.  But, by setting the 'quantum' very low, you
> can cause more work, because HTB has to visit each class more times
> to dequeue the same number of bytes (packets).
>
> You must balance the possibility of sibling starvation against the
> amount of work incurred.
>
> For reference, HTB clamps the 'quantum' values at the following
> minima and maxima (from 1489ff. in sch_htb.c [0]):
>
>   cl->quantum = 1000;      # -- minimum quantum
>   cl->quantum = 200000;    # -- maximum quantum
>
> If you have not set the 'quantum' on any of your classes, then it is
> calculated sanely for you based on the 'rate'.
>
>>Does the highest priority send packets as long as its queue is full
>>or also only for a given quantum?
>
> Hopefully the above explanation answers your question.  The highest
> priority leaf class does not get to send as long as its queue is
> full.  It gets to send 'quantum' bytes for each round of service,
> but that highest priority leaf class will always get the first
> opportunity to send and to borrow from ancestors.
>
>>If it sends as long as its queue is full, how come the traffic with
>>lower priority does not get stalled?
>
> And, this is precisely why the 'quantum' parameter exists.  It's not
> really that important when classes are green, but as soon as classes
> turn yellow, it is the interaction of 'prio' and 'quantum' which
> offers us control over resource sharing among siblings.
>
> With 'quantum', you get to determine how much each yellow sibling
> class gets to transmit at a single service.  With 'prio', you
> determine which sibling transmits first.
>
> Analogy:  If your higher priority sister has a prodigious appetite
> (large 'quantum' value), then she may eat most of the available
> stew!  THe horrible beast!  But, maybe in a different world, your
> higher priority sister eats only enough to keep a bird alive (small
> 'quantum' value).  Considerate sister!  In that case, there's plenty
> of stew left for you and even your lower priority siblings.
>
>>Additionally, is this behavior the same when both queues are in
>>yellow and in green mode (thus at parent and leaf level)?
>
> Also, with the reference to the yellow and green, I see that you
> have read Martin Devera's theory page [1] on HTB!  Wonderfully
> helpful and dense stuff!
>
> Have a wonderful day,
>
> -Martin
>
>  [0] http://lxr.free-electrons.com/source/net/sched/sch_htb.c#L1489
>  [1] http://luxik.cdi.cz/~devik/qos/htb/manual/theory.htm
>
> --
> Martin A. Brown
> http://linux-ip.net/
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [LARTC Home Page]     [Netfilter]     [Netfilter Development]     [Network Development]     [Bugtraq]     [GCC Help]     [Yosemite News]     [Linux Kernel]     [Fedora Users]
  Powered by Linux