On 2012-10-23 03:29, Yufei Ren wrote:
> From: Yufei Ren <renyufei83@xxxxxxxxx>
>
> 1) Use fio shipped reentrant and thread-safe rand to replace buggy rand().
> 2) Add a pause time period before client start committing IOs.
> In SEND/RECV test, it's a good practice to setup the iodepth of
> of the RECV side deeper than that of the SEND side to
> avoid RNR (receiver not ready) error. The
> SEND side may send so many unsolicited message before
> RECV side commits sufficient recv buffers into recv queue.
> This may lead to RNR error. Here, SEND side pauses for a while
> during which RECV side commits sufficient recv buffers.
> 3) Fix server thread hanging bug.
> For RDMA WRITE/READ test, No IO bytes are accumulated in server side
> during test. Server thread indicates its task completion by changing
> its state into `done' as an alternitive. Add `td->done' checking in
> do_io().
> 4) Some comments revision.
> export EXTFLAGS and EXTLIBS with '+=".
> explanation on exchanging steps of RDMA ioengine control messages.
I applied the below on top. We can't reuse td->__random_state, as
that'll impact the generation of random numbers used for IO unit
offsets.
diff --git a/engines/rdma.c b/engines/rdma.c
index 2633432..87b061a 100644
--- a/engines/rdma.c
+++ b/engines/rdma.c
@@ -119,6 +119,8 @@ struct rdmaio_data {
int io_u_flight_nr;
struct io_u **io_us_completed;
int io_u_completed_nr;
+
+ struct frand_state rand_state;
};
static int client_recv(struct thread_data *td, struct ibv_wc *wc)
@@ -617,7 +619,7 @@ static int fio_rdmaio_send(struct thread_data *td, struct io_u **io_us,
if (td->o.use_os_rand)
index = os_random_long(&td->random_state) % rd->rmt_nr;
else
- index = __rand(&td->__random_state) % rd->rmt_nr;
+ index = __rand(&rd->rand_state) % rd->rmt_nr;
r_io_u_d->sq_wr.opcode = IBV_WR_RDMA_WRITE;
r_io_u_d->sq_wr.wr.rdma.rkey = rd->rmt_us[index].rkey;
r_io_u_d->sq_wr.wr.rdma.remote_addr = \
@@ -630,7 +632,7 @@ static int fio_rdmaio_send(struct thread_data *td, struct io_u **io_us,
if (td->o.use_os_rand)
index = os_random_long(&td->random_state) % rd->rmt_nr;
else
- index = __rand(&td->__random_state) % rd->rmt_nr;
+ index = __rand(&rd->rand_state) % rd->rmt_nr;
r_io_u_d->sq_wr.opcode = IBV_WR_RDMA_READ;
r_io_u_d->sq_wr.wr.rdma.rkey = rd->rmt_us[index].rkey;
r_io_u_d->sq_wr.wr.rdma.remote_addr = \
@@ -1210,6 +1212,7 @@ static int fio_rdmaio_setup(struct thread_data *td)
rd = malloc(sizeof(*rd));;
memset(rd, 0, sizeof(*rd));
+ init_rand_seed(&rd->rand_state, GOLDEN_RATIO_PRIME);
td->io_ops->data = rd;
}
--
Jens Axboe
--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
