There is noticeable slowdown with this patch. Seems some iterations (even successful) are taking several seconds of usleep(). Eventually hit a new issue: fio: client host1, timeout on cmd SEND_ETA fio: client host1 timed out But host1 still logged the iops/bw/lat logs, but did not send summary to output. Regards, Jeff -----Original Message----- From: Jens Axboe [mailto:axboe@xxxxxxxxx] Sent: Wednesday, March 21, 2018 12:10 PM To: Jeff Furlong <jeff.furlong@xxxxxxx>; fio@xxxxxxxxxxxxxxx Subject: Re: fio server/client disconnect bug On 3/21/18 1:03 PM, Jens Axboe wrote: > Shoot, I think I see what it is. Some commands we punt to out-of-line > sending, others we do inline. STOP/QUIT are inline, whereas the > ts/gs/du are not. So we could potentially have a situation where we > queue up the ts/gs/du, but then process the STOP/QUIT first. If we do > that, then we don't get any output... > > Let me think about this for a bit, and see what I can come up with. Can you try with this? Can be improved to have the thread signal us, but it should be a rare occurrence and a 1ms stall from this context isn't a concern. diff --git a/server.c b/server.c index 65d4484edae5..52bcf7bf44c6 100644 --- a/server.c +++ b/server.c @@ -551,9 +551,15 @@ static void fio_net_queue_entry(struct sk_entry *entry) { struct sk_out *sk_out = pthread_getspecific(sk_out_key); - if (entry->flags & SK_F_INLINE) + if (entry->flags & SK_F_INLINE) { + /* + * Honor ordering - if the out-of-line queue isn't empty, + * wait for it to drain before proceeding. + */ + while (!flist_empty(&sk_out->list)) + usleep(1000); handle_sk_entry(sk_out, entry); - else { + } else { sk_lock(sk_out); flist_add_tail(&entry->list, &sk_out->list); sk_unlock(sk_out); -- Jens Axboe -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
