Hi,
On Mon, Aug 15, 2022 at 10:56:03AM -0700, Bobby Eshleman wrote:
Hey everybody,
This series introduces datagrams, packet scheduling, and sk_buff usage
to virtio vsock.
Just a reminder for those who are interested, tomorrow Sep 27 @ 16:00
UTC we will discuss more about the next steps for this series in this
room: https://meet.google.com/fxi-vuzr-jjb
(I'll try to record it and take notes that we will share)
Bobby, thank you so much for working on this! It would be great to solve
the fairness issue and support datagram!
I took a look at the series, left some comments in the individual
patches, and add some advice here that we could pick up tomorrow:
- it would be nice to run benchmarks (e.g., iperf-vsock, uperf, etc.) to
see how much the changes cost (e.g. sk_buff use)
- we should take care also of other transports (i.e. vmci, hyperv), the
uAPI should be as close as possible regardless of the transport
About the use of netdev, it seems the most controversial point and I
understand Jakub and Michael's concerns. Tomorrow would be great if you
can update us if you have found any way to avoid it, just reusing a
packet scheduler somehow.
It would be great if we could make it available for all transports (I'm
not asking you to implement it for all, but to have a generic api that
others can use).
But we can talk about that tomorrow!
Thanks,
Stefano
The usage of struct sk_buff benefits users by a) preparing vsock to use
other related systems that require sk_buff, such as sockmap and qdisc,
b) supporting basic congestion control via sock_alloc_send_skb, and c)
reducing copying when delivering packets to TAP.
The socket layer no longer forces errors to be -ENOMEM, as typically
userspace expects -EAGAIN when the sk_sndbuf threshold )s reached and
messages are being sent with option MSG_DONTWAIT.
The datagram work is based off previous patches by Jiang Wang[1].
The introduction of datagrams creates a transport layer fairness issue
where datagrams may freely starve streams of queue access. This happens
because, unlike streams, datagrams lack the transactions necessary for
calculating credits and throttling.
Previous proposals introduce changes to the spec to add an additional
virtqueue pair for datagrams[1]. Although this solution works, using
Linux's qdisc for packet scheduling leverages already existing systems,
avoids the need to change the virtio specification, and gives additional
capabilities. The usage of SFQ or fq_codel, for example, may solve the
transport layer starvation problem. It is easy to imagine other use
cases as well. For example, services of varying importance may be
assigned different priorities, and qdisc will apply appropriate
priority-based scheduling. By default, the system default pfifo qdisc is
used. The qdisc may be bypassed and legacy queuing is resumed by simply
setting the virtio-vsock%d network device to state DOWN. This technique
still allows vsock to work with zero-configuration.
In summary, this series introduces these major changes to vsock:
- virtio vsock supports datagrams
- virtio vsock uses struct sk_buff instead of virtio_vsock_pkt
- Because virtio vsock uses sk_buff, it also uses sock_alloc_send_skb,
which applies the throttling threshold sk_sndbuf.
- The vsock socket layer supports returning errors other than -ENOMEM.
- This is used to return -EAGAIN when the sk_sndbuf threshold is
reached.
- virtio vsock uses a net_device, through which qdisc may be used.
- qdisc allows scheduling policies to be applied to vsock flows.
- Some qdiscs, like SFQ, may allow vsock to avoid transport layer congestion. That is,
it may avoid datagrams from flooding out stream flows. The benefit
to this is that additional virtqueues are not needed for datagrams.
- The net_device and qdisc is bypassed by simply setting the
net_device state to DOWN.
[1]: https://lore.kernel.org/all/20210914055440.3121004-1-jiang.wang@xxxxxxxxxxxxx/
Bobby Eshleman (5):
vsock: replace virtio_vsock_pkt with sk_buff
vsock: return errors other than -ENOMEM to socket
vsock: add netdev to vhost/virtio vsock
virtio/vsock: add VIRTIO_VSOCK_F_DGRAM feature bit
virtio/vsock: add support for dgram
Jiang Wang (1):
vsock_test: add tests for vsock dgram
drivers/vhost/vsock.c | 238 ++++----
include/linux/virtio_vsock.h | 73 ++-
include/net/af_vsock.h | 2 +
include/uapi/linux/virtio_vsock.h | 2 +
net/vmw_vsock/af_vsock.c | 30 +-
net/vmw_vsock/hyperv_transport.c | 2 +-
net/vmw_vsock/virtio_transport.c | 237 +++++---
net/vmw_vsock/virtio_transport_common.c | 771 ++++++++++++++++--------
net/vmw_vsock/vmci_transport.c | 9 +-
net/vmw_vsock/vsock_loopback.c | 51 +-
tools/testing/vsock/util.c | 105 ++++
tools/testing/vsock/util.h | 4 +
tools/testing/vsock/vsock_test.c | 195 ++++++
13 files changed, 1176 insertions(+), 543 deletions(-)
--
2.35.1
