On Wed, Feb 4, 2015 at 2:22 PM, Phillip Hallam-Baker <phill@xxxxxxxxxxxxxxx> wrote:
> Today most Web browsers attempt to optimize download of images etc. by
> opening multiple TCP/IP streams at the same time. This is actually done for
> two reasons, first to reduce load times and second to allow the browser to
> optimize page layout by getting image sizes etc up front.
>
> This approach first appeared round about 1994. I am not sure whether anyone
> actually did a study to see if multiple TCP/IP streams are faster than one
> but the approach has certainly stuck.
They did study it; there was/is a point of diminishing returns... In that era, the diminishing returns were in part due to insufficient buffering at dialup modem banks.
>
> But looking at the problem from the perspective of the network it is really
> hard to see why setting up five TCP/IP streams between the same endpoints
> should provide any more bandwidth than one. If the narrow waist is observed,
> then the only parts of the Internet that are taking note of the TCP part of
> the packet are the end points. So having five streams should not provide any
> more bandwidth than one unless the bandwidth bottleneck was at one or other
> endpoint.
Most web objects are small. In fact, given even IW4, much less IW10, most objects will fit in the initial window of packets for that connection.
Note, this means that the increase in IW means that there is increasing head of line blocking occurring at (most) bottleneck links.
Also note that the web is *not* a bulk data transfer system; it is very latency sensitive.
If you increase the effective RTT, you kill performance.
On (almost all) TCP implementations today, those packets
from all of those objects
will be transmitted at *line rate* in the data center, just to arrive effectively simultaneously at the bottleneck near the user (typically the broadband last mile, or in the customer's home router just before the WiFi hop), where they cause head of line blocking on those devices, as almost all
lack any sort of flow queuing.So you get a burst of possibly up to hundreds of packets (due to web site sharding) transmitted at the data center arriving *splat* at the bottleneck link(s).
I've measured transients of up to hundreds of milliseconds on 50Mbps links.
So TCP congestion control/avoidance has effectively been defeated entirely.
It just isn't in play at all!
I call this "transient bufferbloat". It is the cause of a lot of the VOIP/teleconferencing drop outs you hear if you surf the web.
>
> Now there are some parts of the deployed Internet that do actually perform
> statefull inspection. But I would expect increasing the number of channels
> to degrade performance at a firewall or any other middle boxen.
>
> So we have a set of behavior that seems at odd with the theory. Has anyone
> done any experiments recently that would show which is right?
>
>
> The reason it makes a difference is that it is becoming clear that modern
> applications are not best served by an application API that is limited to
> one bi-directional stream. There are two possible ways to fix this
> situation. The first is to build something on top of TCP/IP the second is to
> replace single stream TCP with multi-stream.
You generally find the size information needed for page layout in the first *packet* of data. Might be nice if those packets from different flows got scheduled before the later packets, might it not? Might be nice if other traffic, such as other TCP opens, or DNS lookups, or your voip packets got scheduled for transmission before those additional packets from that bunch of flows started when you visited that crappy web page with tons of little objects on it (median is
of order
10 objects, btw).Oh, wait, you say that fq_codel does that? Well, you might consider using it.
On
any
current Linux system, that is "one line" of configuration. And you might want to replace your home router too, and use firmware on it that implements fq_codel.... Life gets much better, indeed. And you might want to beat up your ISP, as the bottleneck may be your broadband head-end and not under your control. And ask Microsoft and Apple when their OS's will do something sensible too....
https://tools.ietf.org/html/draft-hoeiland-joergensen-aqm-fq-codel-00
- Jim