CC'ing Cong. On Fri, Apr 14, 2023 at 12:25:56AM +0000, Bobby Eshleman wrote: > Hey all! > > This series introduces support for datagrams to virtio/vsock. > > It is a spin-off (and smaller version) of this series from the summer: > https://lore.kernel.org/all/cover.1660362668.git.bobby.eshleman@xxxxxxxxxxxxx/ > > Please note that this is an RFC and should not be merged until > associated changes are made to the virtio specification, which will > follow after discussion from this series. > > This series first supports datagrams in a basic form for virtio, and > then optimizes the sendpath for all transports. > > The result is a very fast datagram communication protocol that > outperforms even UDP on multi-queue virtio-net w/ vhost on a variety > of multi-threaded workload samples. > > For those that are curious, some summary data comparing UDP and VSOCK > DGRAM (N=5): > > vCPUS: 16 > virtio-net queues: 16 > payload size: 4KB > Setup: bare metal + vm (non-nested) > > UDP: 287.59 MB/s > VSOCK DGRAM: 509.2 MB/s > > Some notes about the implementation... > > This datagram implementation forces datagrams to self-throttle according > to the threshold set by sk_sndbuf. It behaves similar to the credits > used by streams in its effect on throughput and memory consumption, but > it is not influenced by the receiving socket as credits are. > > The device drops packets silently. There is room for improvement by > building into the device and driver some intelligence around how to > reduce frequency of kicking the virtqueue when packet loss is high. I > think there is a good discussion to be had on this. > > In this series I am also proposing that fairness be reexamined as an > issue separate from datagrams, which differs from my previous series > that coupled these issues. After further testing and reflection on the > design, I do not believe that these need to be coupled and I do not > believe this implementation introduces additional unfairness or > exacerbates pre-existing unfairness. > > I attempted to characterize vsock fairness by using a pool of processes > to stress test the shared resources while measuring the performance of a > lone stream socket. Given unfair preference for datagrams, we would > assume that a lone stream socket would degrade much more when a pool of > datagram sockets was stressing the system than when a pool of stream > sockets are stressing the system. The result, however, showed no > significant difference between the degradation of throughput of the lone > stream socket when using a pool of datagrams to stress the queue over > using a pool of streams. The absolute difference in throughput actually > favored datagrams as interfering least as the mean difference was +16% > compared to using streams to stress test (N=7), but it was not > statistically significant. Workloads were matched for payload size and > buffer size (to approximate memory consumption) and process count, and > stress workloads were configured to start before and last long after the > lifetime of the "lone" stream socket flow to ensure that competing flows > were continuously hot. > > Given the above data, I propose that vsock fairness be addressed > independent of datagrams and to defer its implementation to a future > series. > > Signed-off-by: Bobby Eshleman <bobby.eshleman@xxxxxxxxxxxxx> > --- > Bobby Eshleman (3): > virtio/vsock: support dgram > virtio/vsock: add VIRTIO_VSOCK_F_DGRAM feature bit > vsock: Add lockless sendmsg() support > > Jiang Wang (1): > tests: add vsock dgram tests > > drivers/vhost/vsock.c | 17 +- > include/net/af_vsock.h | 20 ++- > include/uapi/linux/virtio_vsock.h | 2 + > net/vmw_vsock/af_vsock.c | 287 ++++++++++++++++++++++++++++---- > net/vmw_vsock/diag.c | 10 +- > net/vmw_vsock/hyperv_transport.c | 15 +- > net/vmw_vsock/virtio_transport.c | 10 +- > net/vmw_vsock/virtio_transport_common.c | 221 ++++++++++++++++++++---- > net/vmw_vsock/vmci_transport.c | 70 ++++++-- > tools/testing/vsock/util.c | 105 ++++++++++++ > tools/testing/vsock/util.h | 4 + > tools/testing/vsock/vsock_test.c | 193 +++++++++++++++++++++ > 12 files changed, 859 insertions(+), 95 deletions(-) > --- > base-commit: ed72bd5a6790a0c3747cb32b0427f921bd03bb71 > change-id: 20230413-b4-vsock-dgram-3b6eba6a64e5 > > Best regards, > -- > Bobby Eshleman <bobby.eshleman@xxxxxxxxxxxxx> >
