Ok, this is where I do some thinking out loud. After I think, I'll
continue my research. If and when I find the answer, I'll write it up
nice and tidy with a mind toward submitting it to the HOWTO. If anyone
out there wants to contribute knowledge or wisdom, then things will go
that much faster. :)
The situation is vanilla home-LAN-to-Internet, where I have no influence
over my ISP's method of traffic control on the interface I connect to.
The majority of the traffic on the link is inbound.
The problem is that when the link is saturated with incoming traffic,
the scheduling of things does not appear to be even remotely fair. The
usual culprit is a single long-lived TCP (HTTP) connection monopolizing
all the bandwidth. Subsequent HTTP connections are then *very* slow to
establish and proceed, sometimes taking so long that the kernel's
connection tracker forgets they exist and discards replies as
'un-related'. Interactive traffic also lags severly.
So the task is to regulate traffic on the link so it gets handled in a
responsive and fair manner. Retransmission cost for packets is
relatively expensive, so we would like to accomplish this without
randomly dropping packets[1]. The task is complicated by the fact that
the link is being saturated by incoming traffic -- traffic which we
cannot control directly. Our saviour (I hope) is in the fact that it's
an established TCP connection saturating the link, which we should be
able to control by regulating ACKs.
One way to obtain control over incoming traffic might be to place a
virtual entity between the incoming interface and the local network.
Now our "real" host (that is, the one which runs the services which
generate and request the traffic) gets its traffic not from the
uncontrolled modem interface, but from the virtual interface to the
virtual host. This virtual interface is under our control, so we may
discipline incoming traffic here, and we control the traffic on the
outbound modem interface as well.
[ Sanity check: Will our traffic policies for the virtual interface ]
[ be reflected back through the modem interface in a useful way? Or ]
[ will excess traffic just end up piling up in the virtual host's ]
[ outgoing queue, not affecting upstream's behaviour at all? ]
[ There's a 'FIXME: explain congestion windows' in the HOWTO, ]
[ is that the section I'm looking for? ;) ]
(Note: Perhaps by the time I figure out the solution, the generalization
provided by this virtual host will not be necessary, but for now it does
make things cleaner conceptually. Never mind the fact that I'm not sure
how to implement it yet -- I'm confident that there is a way to do it
*somewhere* in this heap of routing power and complexity.)
Now, the problem we have here is one I have not seen addressed in
the how-to: The outbound queue and the inbound queue are intimately
related to each other. If the outbound queue is empty, the inbound
queue is free to use as much bandwidth as it likes. However, if the
outbound queue contains low-latency traffic, or packets to establish
new connections, the inbound queue needs to back off enough to let this
traffic out (and to let the low-latency responses back in, though we'll
need the ISP's cooperation if we truly want low-latency responses at the
head of our incoming queue).
Put another way:
Outbound and inbound traffic are really classes of a common queue[2],
and the outbound class has priority.
I say this is a problem, because my current understanding is that queues
are a per-device thing. So how do we get these queues on different
devices to play together?
That's about as far as my thoughts run for the time being.
Other helpful things to let me know might be:
1) Am I misdiagnosing the problem? Is this not really a job for TC?
2) Is there some flaw in my reasoning, borne from my ignorance of how
modems schedule incoming and outgoing traffic?
3) Am I completely on crack?
Thanks for any light you can shed,
- Kevin Turner
Endnotes:
[1] ...without randomly dropping packets:
I think this makes the current ingress policer a poor choice.
However, there is a patch, in iptables's patch-o-matic
distribution, which adds a DROPPED table to netfilter. Perhaps
that could be used to rescue these packets?
(This relates to section 10.5 "FIXME: instead of dropping, can we
assign it to a real queue?")
Working on a patch for ingress might be another option.
[2] outbound and inbound...a common queue:
At least, I believe this is true for modem transports. One can
easily imagine this might not be true for all transports.
--
Kevin Turner <acapnotic@xxxxxxxxxxxxxxxxxxxxx> | OpenPGP encryption welcome here
"Uhh, it's called LART-C. Is this a bad sign?"
