I started trying to use the rate_min and rate_iops_min options. I was trying to use them with io_submit_mode=offload. There appear to be some significant issues with it though. The first is that it will intermittently assert and generally ignore any minimum specified rate. Why? Because in do_io in backend.c, in the IO_OFFLOAD case, comp_time is never set. This means that later when check_min_rate is passed comp_time, it is checking using a time for "now" that is an uninitialized variable. Sometimes that will trip the assert in rel_time_since, sometimes not. And when not, any rate checking is not going to be right. So I added a call to fio_gettime in the IO_OFFLOAD case under the if(should_check_rate(td)...) Now the job will catch when rate_min is not met, but something goes wrong in the shutdown. Fio ends up in a state where IO appears to stop, but fio doesn't exit, and ctrl-C will not cause an exit either. This is because td->error is (rightly) set, so the code after the main loop in do_io will enter the "else cleanup_pending_aio(td)" and hang in its second call to io_u_queued_complete. I think the right fix is to call workqueue_flush before cleanup_pending_aio (in the error, offload case), so that there is nothing left in the work queues and cleanup can proceed as it does in the non-offload case. Two questions before I submit a PR for this: 1) Does adding the fio_gettime() call in the offload case seem right? 2) Does adding the workqueue_flush in the error case before cleanup seem right? Thanks, Nick
