On Mon, Jun 1, 2015 at 11:37 AM, jsullivan@xxxxxxxxxxxxxxxxxxx <jsullivan@xxxxxxxxxxxxxxxxxxx> wrote: > >> On June 1, 2015 at 2:12 PM Dave Taht <dave.taht@xxxxxxxxx> wrote: >> >> >> On Mon, Jun 1, 2015 at 10:06 AM, jsullivan@xxxxxxxxxxxxxxxxxxx >> <jsullivan@xxxxxxxxxxxxxxxxxxx> wrote: >> > >> >> On June 1, 2015 at 12:36 PM Andy Furniss <adf.lists@xxxxxxxxx> wrote: >> >> >> >> >> >> Robert LeBlanc wrote: >> >> > -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 >> >> > >> >> > Any ideas on this? >> >> >> >> 40 gig nics are way beyond anything I've ever done with tc and I guess >> >> involve some offload = huge "packets". >> >> >> >> It could be that aas sfq has a default qlen of 128 and as you are not >> >> actually rate limiting (and to do that may be "interesting" at 80 gig) >> >> the prio relies on some downstream buffer being full. Perhaps it's just >> >> that at these rates prio can not dequeue anything for periods of time so >> >> the 128 limit of sfq is overrun even for the highest prio. >> >> >> >> This is pure guesswork. >> >> -- >> >> To unsubscribe from this list: send the line "unsubscribe lartc" in >> >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > >> > Alas, I haven't yet read the details of the original post but I know we just >> > replaced all our sfq leaves with fq_codel because of concerns about the per >> > flow >> > packet depth on high speed, high latency networks. >> >> Yes, older versions of SFQ had a hard limit on queue depth. This was >> "improved" in linux 3.6 and later but that work ultimately pointed at >> a need to actively manage the queue depth, which begat sfqred, and >> ultimately fq_codel. >> >> I note that, these days, the best results we get for >> tcp-heavy-*servers and hosts* (not routers, not udp heavy services, >> and the bare metal under a vm is a "router" in this context), in the >> data center, at these speeds, now come from sch_fq (from the pacing, >> fq, and tso fixes), and a low setting for tcp_limit_output_bytes. >> >> example: >> >> https://fasterdata.es.net/host-tuning/linux/fair-queuing-scheduler/ >> >> fq_codel remains a great all-around choice, but what pacing is doing >> is really remarkable for servers in sch_fq. >> > <snip> > Interesting. We do a lot of video work and hence UDP. For heavily mixed > environments with high bandwidth and sometimes high latency, is fq_codel still > the preferred leaf qdisc or should we be investigating sch_fq? Thanks - John The answer is too nuanced to fit into the margins of this email. :) I believe FAR more research and development work is needed on low latency udp or current-ly using udp - applications. Most of the industry focus is (and has always been) on making tcp perform well - and me, well, it is obvious that we are going to lose the existing phone system in under 2 decades - and we need to replace it with a (hopefully better) internet, add videoconferencing, and make other highly interactive apps like games co-exist with netflix. But: What exactly do you mean by leaf qdisc in this question? Latency happens at the bottleneck links to the largest extent, but there is also latency that accrues within a machine at various layers in the stack. you need to find and improve the bottlenecks, which is something of a wackamole exercise. My principal concern (today) with the potential wide deployment of sch_fq (over fq_codel) is that sch_fq generates a unique internal id per new flow which is not sufficiently resistant to DDOS attacks for my taste. sch_fq has "improved" lately with some protection for that, but it disables the distinct per-flow-fq-advantage sch_fq brought to the party in the first place, but, (on the gripping hand) may well be the closest thing to the right thing for your application mix! Try it! report back! :) fq_codel is faster and safer in a wider range of environments than anything else we got. Safety is an important feature... Other ideas at the protocol layer for "things with cookies" - would be nice - like, (the now impossible to implement) rfc6013 - and Quic is quite promising - these ideas strike me as a long term answer to the bcp38 and DDOS problem, but those ideas need to move into voip transactions. -- Dave Täht What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast -- To unsubscribe from this list: send the line "unsubscribe lartc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
