Hello Martin,
Thank you so much for taking time to answer my questions in such a detailed way.
You have provided the information I was missing to understand how the
priorities work, hence now I understand the importance of the quantum
parameter, but also the behavior that I was observing during my HTB
tests.
For me quantum was mainly applicable in case of the same priorities,
thus I have left it to default, i.e. calculated according to
configured rates. But given the information you have provided, I
definitely have a better understanding and I will see how I can adapt
this parameter.
Thank you again for your help.
Have a great day,
Ewa
On Tue, Apr 5, 2016 at 7:41 PM, Martin A. Brown <martin@xxxxxxxxxxxx> wrote:
>
> Greetings Ewa,
>
> You seem to be exercising HTB quite well.
>
>>I have questions about how exactly priorities work in HTB. I have
>>read the documentation and different websites, but it still appears
>>not really clear to me, especially that I do not observe the
>>“right” behavior.
>
> I think the 'prio' is one of the under-explained parameters to HTB.
>
>>I was trying to give an absolute priority to real-time traffic, but
>>concurrent TCP traffic with lower priority was still sometimes
>>taking over (even though its rate was limited to minimum so was
>>getting almost nothing in the green mode since I know that queues
>>in green mode are always sent before those in yellow).
>
> It's quite good that you have set the 'rate' on the concurrent TCP
> traffic class to be quite low, because as you point out, green
> classes always get to meet their Assured Rate, the HTB 'rate'
> parameter.
>
> [
>
> For others who may be reading this message, here's a review of our
> references to class colors:
>
> * green: the current dequeue rate of an HTB class is below its
> 'rate' (in the theory, this is called 'Assured Rate' [1])
>
> * yellow: the current dequeue rate of an HTB class is between
> its 'rate' and its 'ceil'
>
> * red: the current dequeue rate of an HTB class equals or exceeds
> the 'ceil' (it is possible to go slightly over 'ceil' when
> borrowing and dequeuing 'quantum' bytes)
> ]
>
> My main (somewhat rhetorical) question in reply to you is:
>
> How large is the quantum?
>
> In the rest of my reply, I will try to describe the interaction
> between the 'quantum' and 'prio' parameters in HTB operation.
>
>>Thus I would like to ask how exactly the priority is treated in
>>HTB.
>>
> >From the documentation, I understand that in case of the same
>>priority, the quantum and DRR is used. So no problem here.
>
> Here's are some framing rules, that may help (it's quite likely to
> be review for you).
>
> * each class is either a leaf class or an ancestor class (a.k.a.
> inner class)
>
> * leaf classes enqueue, hold and dequeue packets (they do all of
> the actual work)
>
> * ancestor classes only offer control over borrowing; they hold no
> packets, but rather control the distribution of shared resources
>
> * the 'prio' and 'quantum' parameters are only used in leaf
> classes ('prio' and 'quantum' have no real meaning in parent
> classes, even if they accept the parameters)
>
> * lower numbered values for the 'prio' of a class represent higher
> priority
>
> * borrowing does not kick in until a leaf class has hit its
> 'rate' (when the class becomes yellow)
>
> * for ancestor classes, the 'rate' and 'ceil' parameters are only
> used to determine when that ancestor can lend to children (I
> think that only 'ceil' is used, maybe others who know can
> comment authoritatively)
>
> * the HTB qdisc will only service the leaf classes looking for
> packets to dequeue if at least one leaf is green or if there are
> ancestors are capable of lending tokens
>
> At any point, either there are tokens available, in which case, HTB
> can send, or there are no tokens available, in which case HTB waits
> until tokens are available again. Here are a couple of (the many)
> possible states for an HTB tree:
>
> * all leaves in an HTB tree may be completely 'red'; every leaf
> class is at its 'ceil'; no packets will be dequeued
>
> * all (or some) leaves in an HTB tree are 'yellow', but all
> ancestor classes are 'red'; no packets dequeued
>
> * all (or some) leaves in an HTB tree are 'yellow', and some
> ancestor classes are 'green' or 'yellow'; HTB will allow leaf
> classes to borrow from ancestors and dequeue packets up to
> 'quantum' bytes in 'prio' order
>
> * all (or some) leaves in an HTB tree are 'green' (ancestor
> classes don't matter here); allow leaf classes to dequeue
> packets up to 'quantum' bytes in 'prio' order
>
> Here's another way to look at it. This is what HTB does in an
> infinite loop (of course, this is an inexact approximation in
> pseudo-code):
>
> while exist(green_classes)
> for leaf in ordered_by_prio(green_classes)
> dequeue_and_charge_parents(leaf, quantum_bytes)
>
> while exist(yellow_classes)
> for leaf in ordered_by_prio(yellow_classes)
> if tokens_available(ancestors(leaf))
> dequeue_and_charge_parents(leaf, quantum_bytes)
>
> So, as you can see, any leaf class that has not spent its 'ceil' yet
> gets an opportunity to send 'quantum' bytes, during each time it is
> possible to send. The above is, of course, an approximation, since
> packets can arrive at any time to any class....but, this is roughly
> how HTB behaves.
>
> And, now to try to answer your questions directly.
>
>>However, in case of queues with different priorities (but on the
>>same level), how is it exactly divided?
>
> All leaf classes are visited in order (by 'prio'). Each leaf class
> gets to send up to 'quantum' bytes. Then, the next leaf class gets
> an opportunity. And, this is the magic of 'quantum'.
>
> The 'quantum' parameter has significant possible effects. By
> setting the 'quantum' parameter really large on one class, you can
> starve other classes. But, by setting the 'quantum' very low, you
> can cause more work, because HTB has to visit each class more times
> to dequeue the same number of bytes (packets).
>
> You must balance the possibility of sibling starvation against the
> amount of work incurred.
>
> For reference, HTB clamps the 'quantum' values at the following
> minima and maxima (from 1489ff. in sch_htb.c [0]):
>
> cl->quantum = 1000; # -- minimum quantum
> cl->quantum = 200000; # -- maximum quantum
>
> If you have not set the 'quantum' on any of your classes, then it is
> calculated sanely for you based on the 'rate'.
>
>>Does the highest priority send packets as long as its queue is full
>>or also only for a given quantum?
>
> Hopefully the above explanation answers your question. The highest
> priority leaf class does not get to send as long as its queue is
> full. It gets to send 'quantum' bytes for each round of service,
> but that highest priority leaf class will always get the first
> opportunity to send and to borrow from ancestors.
>
>>If it sends as long as its queue is full, how come the traffic with
>>lower priority does not get stalled?
>
> And, this is precisely why the 'quantum' parameter exists. It's not
> really that important when classes are green, but as soon as classes
> turn yellow, it is the interaction of 'prio' and 'quantum' which
> offers us control over resource sharing among siblings.
>
> With 'quantum', you get to determine how much each yellow sibling
> class gets to transmit at a single service. With 'prio', you
> determine which sibling transmits first.
>
> Analogy: If your higher priority sister has a prodigious appetite
> (large 'quantum' value), then she may eat most of the available
> stew! THe horrible beast! But, maybe in a different world, your
> higher priority sister eats only enough to keep a bird alive (small
> 'quantum' value). Considerate sister! In that case, there's plenty
> of stew left for you and even your lower priority siblings.
>
>>Additionally, is this behavior the same when both queues are in
>>yellow and in green mode (thus at parent and leaf level)?
>
> Also, with the reference to the yellow and green, I see that you
> have read Martin Devera's theory page [1] on HTB! Wonderfully
> helpful and dense stuff!
>
> Have a wonderful day,
>
> -Martin
>
> [0] http://lxr.free-electrons.com/source/net/sched/sch_htb.c#L1489
> [1] http://luxik.cdi.cz/~devik/qos/htb/manual/theory.htm
>
> --
> Martin A. Brown
> http://linux-ip.net/
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
