On Sat, Jun 10, 2023 at 12:58:34AM +0000, Bobby Eshleman wrote: > Because the dgram sendmsg() path for AF_VSOCK acquires the socket lock > it does not scale when many senders share a socket. > > Prior to this patch the socket lock is used to protect both reads and > writes to the local_addr, remote_addr, transport, and buffer size > variables of a vsock socket. What follows are the new protection schemes > for these fields that ensure a race-free and usually lock-free > multi-sender sendmsg() path for vsock dgrams. > > - local_addr > local_addr changes as a result of binding a socket. The write path > for local_addr is bind() and various vsock_auto_bind() call sites. > After a socket has been bound via vsock_auto_bind() or bind(), subsequent > calls to bind()/vsock_auto_bind() do not write to local_addr again. bind() > rejects the user request and vsock_auto_bind() early exits. > Therefore, the local addr can not change while a parallel thread is > in sendmsg() and lock-free reads of local addr in sendmsg() are safe. > Change: only acquire lock for auto-binding as-needed in sendmsg(). > > - buffer size variables > Not used by dgram, so they do not need protection. No change. > > - remote_addr and transport > Because a remote_addr update may result in a changed transport, but we > would like to be able to read these two fields lock-free but coherently > in the vsock send path, this patch packages these two fields into a new > struct vsock_remote_info that is referenced by an RCU-protected pointer. > > Writes are synchronized as usual by the socket lock. Reads only take > place in RCU read-side critical sections. When remote_addr or transport > is updated, a new remote info is allocated. Old readers still see the > old coherent remote_addr/transport pair, and new readers will refer to > the new coherent. The coherency between remote_addr and transport > previously provided by the socket lock alone is now also preserved by > RCU, except with the highly-scalable lock-free read-side. > > Helpers are introduced for accessing and updating the new pointer. > > The new structure is contains an rcu_head so that kfree_rcu() can be > used. This removes the need of writers to use synchronize_rcu() after > freeing old structures which is simply more efficient and reduces code > churn where remote_addr/transport are already being updated inside RCU > read-side sections. > > Only virtio has been tested, but updates were necessary to the VMCI and > hyperv code. Unfortunately the author does not have access to > VMCI/hyperv systems so those changes are untested. > > Perf Tests (results from patch v2) > vCPUS: 16 > Threads: 16 > Payload: 4KB > Test Runs: 5 > Type: SOCK_DGRAM > > Before: 245.2 MB/s > After: 509.2 MB/s (+107%) > > Notably, on the same test system, vsock dgram even outperforms > multi-threaded UDP over virtio-net with vhost and MQ support enabled. > > Throughput metrics for single-threaded SOCK_DGRAM and > single/multi-threaded SOCK_STREAM showed no statistically signficant Hi Bobby, a minor nit from checkpatch --codespell: signficant -> significant > throughput changes (lowest p-value reaching 0.27), with the range of the > mean difference ranging between -5% to +1%. > > Signed-off-by: Bobby Eshleman <bobby.eshleman@xxxxxxxxxxxxx> ...
