Using an ITER_UBUF is more efficient than an ITER_IOV, and for the single
segment case, there's no reason to use an ITER_IOV when an ITER_UBUF will
do. Experimental data collected shows that ~2/3rds of iovec imports are
single segments, from applications using readv/writev or recvmsg/sendmsg
that are iovec based.
Explicitly check for nr_segs == 1 and import those as ubuf rather than
iovec based iterators.
Signed-off-by: Jens Axboe <axboe@xxxxxxxxx>
---
Lightly tested - boots and works just fine, and I ran this through the
liburing test suite which does plenty of single segment readv/writev
as well.
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 6b2bf6f6f374..0973c622d3c0 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -1813,6 +1813,21 @@ ssize_t __import_iovec(int type, const struct iovec __user *uvec,
return PTR_ERR(iov);
}
+ /*
+ * Fast path - single segment import. Use UBUF for these, rather
+ * than setup an ITER_IOV.
+ */
+ if (nr_segs == 1) {
+ ssize_t ret;
+
+ total_len = iovp[0]->iov_len;
+ ret = import_ubuf(type, iovp[0]->iov_base, total_len, i);
+ *iovp = NULL;
+ if (unlikely(ret < 0))
+ return ret;
+ return total_len;
+ }
+
/*
* According to the Single Unix Specification we should return EINVAL if
* an element length is < 0 when cast to ssize_t or if the total length
--
Jens Axboe
