> On Aug 18, 2016, at 2:58 AM, David Woodhouse <dwmw2 at infradead.org> wrote: > > On Wed, 2016-08-17 at 09:26 -0700, Tom Rodriguez wrote: >>> I?ve been using open connect on Mac OS X and noticed that I?m >> getting much worse throughput than the same setup running on linux on >> a local virtual machine. Almost an order of magnitude worse. I?ve >> googled for help on the issue but without any luck, so here I am. > > Under Linux I'd be pointing you at the profiling tools so you can see > precisely where the *system* is spending its time. > > For OS X I am unfamiliar with any such tools. One thing which comes to > mind is the different tun devices. We'll be using utun on OS X these > days by default... does it get better if you install the Tun/Tap driver > and use that instead? There's plenty of scope for screwing performance > up in the user/kernel interface... > > >> I?ve tried a few different command line options and adjusting the MTU >> but without luck. Are there any known issues concerning this? I do >> get a lot of "Failed to write incoming packet: No buffer space >> available? on the mac but it didn?t seem like enough to explain the >> low throughput. > > That's on receiving a packet from the VPN, and feeding it to the kernel > through (presumably) the utun interface. Which tends to support my > first supposition above? I?m actually using ocproxy to handle the traffic so I don?t think I?m going through the tunnel device. So does that mean the packets are being lost on the datagram socket connecting me to ocproxy? My actual command line is this: openconnect -q --compression=none --script-tun --script "ocproxy -D 9998? <host> Maybe buffering in the unix socket talking to ocproxy is to blame? In the output below fd 9 is the UDP connection to the VPN and 8 is one half of the unix socket talking to ocproxy. select(0xA, 0x7FFF50ABFDE0, 0x7FFF50ABFD60, 0x7FFF50ABFCE0, 0x7FFF50ABFE60) = 1 0 recvfrom(0x9, 0x7FC8BD800053, 0x495C) = 1117 0 recvfrom(0x9, 0x7FC8BD804A53, 0x495C) = 1117 0 recvfrom(0x9, 0x7FC8BD800053, 0x495C) = 1117 0 recvfrom(0x9, 0x7FC8BD804A53, 0x495C) = 1117 0 recvfrom(0x9, 0x7FC8BD800053, 0x495C) = 1117 0 recvfrom(0x9, 0x7FC8BD804A53, 0x495C) = -1 Err#35 recvfrom(0x6, 0x7FC8BD804A5B, 0x5) = -1 Err#35 read(0x8, "\0", 0x514) = -1 Err#35 write(0x8, "E\0", 0x428) = 1064 0 write(0x8, "E\0", 0x428) = 1064 0 write(0x8, "E\0", 0x428) = 1064 0 write(0x8, "E\0", 0x428) = 1064 0 write(0x8, "\0", 0x428) = -1 Err#55 write_nocancel(0x2, "Failed to write incoming packet: No buffer space available\n\0", 0x3B) = 59 0 Some googling led me to net.local.dgram.recvspace to tune the buffer space for unix domain sockets. Running this: sudo sysctl -w net.local.dgram.recvspace=82320 brought my mac performance back up. The default value of that tunable is 8192 which I guess is too low. Maybe setsockopt should be used to increase it on the Mac? Presumably linux has a higher default which is why it works ok there. tom > > Note that we *drop* the packet when that happens. See the code at > http://git.infradead.org/users/dwmw2/openconnect.git/blob/HEAD:/tun.c#l551 > http://git.infradead.org/users/dwmw2/openconnect.git/blob/HEAD:/mainloop.c#l95 > > So... the tun interface's queue is full and it stops accepting packets > from us. We *drop* them, and that's going to screw your TCP connections > up quite a lot. > > In an ideal world, the poll() system call would let us know when the > tun is writeable again, and we'd start shovelling packets then. In > Linux that doesn't work so as the comments note, we don't bother with > that. But perhaps we should, for OS X? You'd make os_write_tun() > actually return -EAGAIN, and make mainlop.c react accordingly by > calling monitor_write_fd() on the tun fd and only ever trying to write > to it when it's... well, writeable. > > And we'd still need a hack for Linux so that we don't just busy-wait > trying to write the same packet over and over again until it's > accepted. But first check if the utun interface really *does* support > poll() for writeability at all, perhaps... > > -- > dwmw2
