On 05.12.2023 13:54, Stefano Garzarella wrote:
> On Tue, Dec 05, 2023 at 09:48:05AM +0300, Arseniy Krasnov wrote:
>> Add one more condition for sending credit update during dequeue from
>> stream socket: when number of bytes in the rx queue is smaller than
>> SO_RCVLOWAT value of the socket. This is actual for non-default value
>> of SO_RCVLOWAT (e.g. not 1) - idea is to "kick" peer to continue data
>> transmission, because we need at least SO_RCVLOWAT bytes in our rx
>> queue to wake up user for reading data (in corner case it is also
>> possible to stuck both tx and rx sides, this is why 'Fixes' is used).
>>
>> Fixes: b89d882dc9fc ("vsock/virtio: reduce credit update messages")
>> Signed-off-by: Arseniy Krasnov <avkrasnov@xxxxxxxxxxxxxxxxx>
>> ---
>> net/vmw_vsock/virtio_transport_common.c | 9 +++++++--
>> 1 file changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
>> index e137d740804e..461c89882142 100644
>> --- a/net/vmw_vsock/virtio_transport_common.c
>> +++ b/net/vmw_vsock/virtio_transport_common.c
>> @@ -558,6 +558,7 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk,
>> struct virtio_vsock_sock *vvs = vsk->trans;
>> size_t bytes, total = 0;
>> struct sk_buff *skb;
>> + bool low_rx_bytes;
>> int err = -EFAULT;
>> u32 free_space;
>>
>> @@ -602,6 +603,8 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk,
>> }
>>
>> free_space = vvs->buf_alloc - (vvs->fwd_cnt - vvs->last_fwd_cnt);
>> + low_rx_bytes = (vvs->rx_bytes <
>> + sock_rcvlowat(sk_vsock(vsk), 0, INT_MAX));
>
> As in the previous patch, should we avoid the update it if `fwd_cnt` and `last_fwd_cnt` are the same?
>
> Now I'm thinking if it is better to add that check directly in virtio_transport_send_credit_update().
Good point, but I think, that it is better to keep this check here, because access to 'fwd_cnt' and 'last_fwd_cnt'
requires taking rx_lock - so I guess it is better to avoid taking this lock every time in 'virtio_transport_send_credit_update()'.
So may be we can do something like:
fwd_cnt_delta = vvs->fwd_cnt - vvs->last_fwd_cnt;
free_space = vvs->buf_alloc - fwd_cnt_delta;
and then, after lock is released:
if (fwd_cnt_delta && (free_space < VIRTIO_VSOCK_MAX_PKT_BUF_SIZE ||
low_rx_bytes))
virtio_transport_send_credit_update(vsk);
WDYT?
Also, I guess that next idea to update this optimization(in next patchset), is to make
threshold depends on vvs->buf_alloc. Because if someone changes minimum buffer size to
for example 32KB, and then sets buffer size to 32KB, then free_space will be always
non-zero, thus optimization is off now and credit update is sent on every read.
Thanks, Arseniy
>
> Stefano
>
>>
>> spin_unlock_bh(&vvs->rx_lock);
>>
>> @@ -611,9 +614,11 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk,
>> * too high causes extra messages. Too low causes transmitter
>> * stalls. As stalls are in theory more expensive than extra
>> * messages, we set the limit to a high value. TODO: experiment
>> - * with different values.
>> + * with different values. Also send credit update message when
>> + * number of bytes in rx queue is not enough to wake up reader.
>> */
>> - if (free_space < VIRTIO_VSOCK_MAX_PKT_BUF_SIZE)
>> + if (free_space < VIRTIO_VSOCK_MAX_PKT_BUF_SIZE ||
>> + low_rx_bytes)
>> virtio_transport_send_credit_update(vsk);
>>
>> return total;
>> --
>> 2.25.1
>>
>
