Hi Jens, I will post job files at the bottom, I am committing client / server mode tests. one launcher to four servers writing IO to RBD Pool using RBD engine, the tests seem to run normally and then when runtime completes it seems to be waiting for something and then fails with this console message - [root@tme-sj-s16 4krd]# /usr/local/bin/fio --client=tme-sj-s27.lab 4kread_s27.job --client=tme-sj-s28.lab 4kread_s28.job --client=tme-sj-s29.lab 4kread_s29.job --client=tme-sj-s30.lab 4kread_s30.job --output=./4krd_qd6.txt fio: client: unable to find matching tag (13cca90)s] [0/0/0 iops] [eta 00m:00s] fio: client.c:1241: handle_eta: Assertion `client->eta_in_flight == eta' failed. Aborted Job files:(4) 4kread_s27.job [global] direct=1 rw=randread randrepeat=0 ioengine=rbd pool=pool001 clientname=admin bs=4k iodepth=6 numjobs=10 runtime=600 group_reporting write_iops_log=./log/4kreadiops_27_qd6.res write_bw_log=./log/4kreadbw_27_qd6.res [IF_4krd_27_qd6] rbdname=rbdimg001 4kread_s28.job [global] direct=1 rw=randread randrepeat=0 ioengine=rbd pool=pool001 clientname=admin bs=4k iodepth=6 numjobs=10 runtime=600 group_reporting write_iops_log=./log/4kreadiops_28_qd6.res write_bw_log=./log/4kreadbw_28_qd6.res [IF_4krd_28_qd6] rbdname=rbdimg002 4kread_s29.job [global] direct=1 rw=randread randrepeat=0 ioengine=rbd pool=pool002 clientname=admin bs=4k iodepth=6 numjobs=10 runtime=600 group_reporting write_iops_log=./log/4kreadiops_29_qd6.res write_bw_log=./log/4kreadbw_29_qd6.res [IF_4krd_29_qd6] rbdname=rbdimg001 4kread_s30.job [global] direct=1 rw=randread randrepeat=0 ioengine=rbd pool=pool002 clientname=admin bs=4k iodepth=6 numjobs=10 runtime=600 group_reporting write_iops_log=./log/4kreadiops_30_qd6.res write_bw_log=./log/4kreadbw_30_qd6.res [IF_4krd_30_qd6] rbdname=rbdimg002 Thank you Jens! very much! -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html