Hi Michael and Stefan, I thought about this and discussed it with my colleague Cong Wang. One idea is to make current asynchronous send_pkt flow to be synchronous, then if the virtqueue is full, the function can return ENOMEM all the way back to the caller and the caller can check the return value of sendmsg and slow down when necessary. In the spec, we can put something like, if the virtqueue is full, the caller should be notified with an error etc. In terms of implementation, that means we will remove the current send_pkt_work for both stream and dgram sockets. Currently, the code path uses RCU and a work queue, then grab a mutex in the work queue function. Since we cannot grab mutex when in rcu critical section, we have to change RCU to a normal reference counting mechanism. I think this is doable. The drawback is that the reference counting in general spends a little more cycles than the RCU, so there is a small price to pay. Another option is to use Sleepable RCU and remove the work queue. What do you guys think? btw, I will also add some SENDBUF restrictions for the dgram sockets, but I don't think it needs to be in the spec. Regards, Jiang On Tue, Mar 23, 2021 at 1:53 AM Stefan Hajnoczi <stefanha@xxxxxxxxxx> wrote: > > On Mon, Mar 22, 2021 at 07:23:14PM -0700, Jiang Wang . wrote: > > Got it. Will do. > > You could look at udp_sendmsg() to see how sockets compete when > transmitting to the same net device. > > I'm not very familiar with this but I guess that the qdisc (like > fq_codel) decides which packets to place into the device's tx queue. I > guess sk_buffs waiting to be put onto the device's tx queue are > accounted for against the socket's sndbuf. Further sendmsg calls will > fail with -ENOBUFS when the sndbuf limit is reached. > > It's not clear to me how much of the existing code can be reused since > vsock does not use sk_buff or netdev :(. > > Stefan _______________________________________________ Virtualization mailing list Virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx https://lists.linuxfoundation.org/mailman/listinfo/virtualization
