Greetings Ewa,
You seem to be exercising HTB quite well.
>I have questions about how exactly priorities work in HTB. I have
>read the documentation and different websites, but it still appears
>not really clear to me, especially that I do not observe the
>“right” behavior.
I think the 'prio' is one of the under-explained parameters to HTB.
>I was trying to give an absolute priority to real-time traffic, but
>concurrent TCP traffic with lower priority was still sometimes
>taking over (even though its rate was limited to minimum so was
>getting almost nothing in the green mode since I know that queues
>in green mode are always sent before those in yellow).
It's quite good that you have set the 'rate' on the concurrent TCP
traffic class to be quite low, because as you point out, green
classes always get to meet their Assured Rate, the HTB 'rate'
parameter.
[
For others who may be reading this message, here's a review of our
references to class colors:
* green: the current dequeue rate of an HTB class is below its
'rate' (in the theory, this is called 'Assured Rate' [1])
* yellow: the current dequeue rate of an HTB class is between
its 'rate' and its 'ceil'
* red: the current dequeue rate of an HTB class equals or exceeds
the 'ceil' (it is possible to go slightly over 'ceil' when
borrowing and dequeuing 'quantum' bytes)
]
My main (somewhat rhetorical) question in reply to you is:
How large is the quantum?
In the rest of my reply, I will try to describe the interaction
between the 'quantum' and 'prio' parameters in HTB operation.
>Thus I would like to ask how exactly the priority is treated in
>HTB.
>
>From the documentation, I understand that in case of the same
>priority, the quantum and DRR is used. So no problem here.
Here's are some framing rules, that may help (it's quite likely to
be review for you).
* each class is either a leaf class or an ancestor class (a.k.a.
inner class)
* leaf classes enqueue, hold and dequeue packets (they do all of
the actual work)
* ancestor classes only offer control over borrowing; they hold no
packets, but rather control the distribution of shared resources
* the 'prio' and 'quantum' parameters are only used in leaf
classes ('prio' and 'quantum' have no real meaning in parent
classes, even if they accept the parameters)
* lower numbered values for the 'prio' of a class represent higher
priority
* borrowing does not kick in until a leaf class has hit its
'rate' (when the class becomes yellow)
* for ancestor classes, the 'rate' and 'ceil' parameters are only
used to determine when that ancestor can lend to children (I
think that only 'ceil' is used, maybe others who know can
comment authoritatively)
* the HTB qdisc will only service the leaf classes looking for
packets to dequeue if at least one leaf is green or if there are
ancestors are capable of lending tokens
At any point, either there are tokens available, in which case, HTB
can send, or there are no tokens available, in which case HTB waits
until tokens are available again. Here are a couple of (the many)
possible states for an HTB tree:
* all leaves in an HTB tree may be completely 'red'; every leaf
class is at its 'ceil'; no packets will be dequeued
* all (or some) leaves in an HTB tree are 'yellow', but all
ancestor classes are 'red'; no packets dequeued
* all (or some) leaves in an HTB tree are 'yellow', and some
ancestor classes are 'green' or 'yellow'; HTB will allow leaf
classes to borrow from ancestors and dequeue packets up to
'quantum' bytes in 'prio' order
* all (or some) leaves in an HTB tree are 'green' (ancestor
classes don't matter here); allow leaf classes to dequeue
packets up to 'quantum' bytes in 'prio' order
Here's another way to look at it. This is what HTB does in an
infinite loop (of course, this is an inexact approximation in
pseudo-code):
while exist(green_classes)
for leaf in ordered_by_prio(green_classes)
dequeue_and_charge_parents(leaf, quantum_bytes)
while exist(yellow_classes)
for leaf in ordered_by_prio(yellow_classes)
if tokens_available(ancestors(leaf))
dequeue_and_charge_parents(leaf, quantum_bytes)
So, as you can see, any leaf class that has not spent its 'ceil' yet
gets an opportunity to send 'quantum' bytes, during each time it is
possible to send. The above is, of course, an approximation, since
packets can arrive at any time to any class....but, this is roughly
how HTB behaves.
And, now to try to answer your questions directly.
>However, in case of queues with different priorities (but on the
>same level), how is it exactly divided?
All leaf classes are visited in order (by 'prio'). Each leaf class
gets to send up to 'quantum' bytes. Then, the next leaf class gets
an opportunity. And, this is the magic of 'quantum'.
The 'quantum' parameter has significant possible effects. By
setting the 'quantum' parameter really large on one class, you can
starve other classes. But, by setting the 'quantum' very low, you
can cause more work, because HTB has to visit each class more times
to dequeue the same number of bytes (packets).
You must balance the possibility of sibling starvation against the
amount of work incurred.
For reference, HTB clamps the 'quantum' values at the following
minima and maxima (from 1489ff. in sch_htb.c [0]):
cl->quantum = 1000; # -- minimum quantum
cl->quantum = 200000; # -- maximum quantum
If you have not set the 'quantum' on any of your classes, then it is
calculated sanely for you based on the 'rate'.
>Does the highest priority send packets as long as its queue is full
>or also only for a given quantum?
Hopefully the above explanation answers your question. The highest
priority leaf class does not get to send as long as its queue is
full. It gets to send 'quantum' bytes for each round of service,
but that highest priority leaf class will always get the first
opportunity to send and to borrow from ancestors.
>If it sends as long as its queue is full, how come the traffic with
>lower priority does not get stalled?
And, this is precisely why the 'quantum' parameter exists. It's not
really that important when classes are green, but as soon as classes
turn yellow, it is the interaction of 'prio' and 'quantum' which
offers us control over resource sharing among siblings.
With 'quantum', you get to determine how much each yellow sibling
class gets to transmit at a single service. With 'prio', you
determine which sibling transmits first.
Analogy: If your higher priority sister has a prodigious appetite
(large 'quantum' value), then she may eat most of the available
stew! THe horrible beast! But, maybe in a different world, your
higher priority sister eats only enough to keep a bird alive (small
'quantum' value). Considerate sister! In that case, there's plenty
of stew left for you and even your lower priority siblings.
>Additionally, is this behavior the same when both queues are in
>yellow and in green mode (thus at parent and leaf level)?
Also, with the reference to the yellow and green, I see that you
have read Martin Devera's theory page [1] on HTB! Wonderfully
helpful and dense stuff!
Have a wonderful day,
-Martin
[0] http://lxr.free-electrons.com/source/net/sched/sch_htb.c#L1489
[1] http://luxik.cdi.cz/~devik/qos/htb/manual/theory.htm
--
Martin A. Brown
http://linux-ip.net/
