[ISSUE] The time cost of IOSQE_IO_LINK

[Carter Li 李通洲 <carter.li@xxxxxxxxxx>]

Hi everyone,

IOSQE_IO_LINK seems to have very high cost, even greater then io_uring_enter syscall.

Test code attached below. The program completes after getting 100000000 cqes.

$ gcc test.c -luring -o test0 -g -O3 -DUSE_LINK=0
$ time ./test0
USE_LINK: 0, count: 100000000, submit_count: 1562500
0.99user 9.99system 0:11.02elapsed 99%CPU (0avgtext+0avgdata 1608maxresident)k
0inputs+0outputs (0major+72minor)pagefaults 0swaps

$ gcc test.c -luring -o test1 -g -O3 -DUSE_LINK=1
$ time ./test1
USE_LINK: 1, count: 100000110, submit_count: 799584
0.83user 19.21system 0:20.90elapsed 95%CPU (0avgtext+0avgdata 1632maxresident)k
0inputs+0outputs (0major+72minor)pagefaults 0swaps

As you can see, the `-DUSE_LINK=1` version emits only about half io_uring_submit calls
of the other version, but takes twice as long. That makes IOSQE_IO_LINK almost useless,
please have a check.

Thanks,
Carter Li

---

#include <liburing.h>
#include <stdio.h>

#define ENTRY_SIZE 128
#define MAX_COUNT 100000000

#ifndef USE_LINK
#	define USE_LINK 0
#endif

int main(void) {
	struct io_uring ring;
	unsigned i, head;
	unsigned count = 0, cqe_count = 0, submit_count = 0;
	struct io_uring_sqe *sqe;
	struct io_uring_cqe *cqe;

	if (io_uring_queue_init(ENTRY_SIZE, &ring, 0) < 0) {
		return -1;
	}

	for (i = 0; i < ENTRY_SIZE / 2; ++i) {
		sqe = io_uring_get_sqe(&ring);
		io_uring_prep_nop(sqe);
#if USE_LINK
		io_uring_sqe_set_flags(sqe, IOSQE_IO_LINK);
		sqe = io_uring_get_sqe(&ring);
		io_uring_prep_nop(sqe);
		sqe->user_data = 1;
#endif
	}

	while (count < MAX_COUNT) {
		io_uring_submit_and_wait(&ring, 1);
		++submit_count;

		io_uring_for_each_cqe(&ring, head, cqe) {
			++cqe_count;
			++count;
#if USE_LINK
			if (cqe->user_data) {
				sqe = io_uring_get_sqe(&ring);
				io_uring_prep_nop(sqe);
				io_uring_sqe_set_flags(sqe, IOSQE_IO_LINK);
				sqe = io_uring_get_sqe(&ring);
				io_uring_prep_nop(sqe);
				sqe->user_data = 1;
			}
#else
			sqe = io_uring_get_sqe(&ring);
			io_uring_prep_nop(sqe);
			sqe->user_data = !cqe->user_data;
#endif
		}
		io_uring_cq_advance(&ring, cqe_count);
		cqe_count = 0;
	}

	printf("USE_LINK: %d, count: %u, submit_count: %u\n\n",
		USE_LINK, count, submit_count);
}