On 2023-02-16 9:59 p.m., John David Anglin wrote:
As mentioned previously, this one and the other -233 I suspect are due
to the same coloring issue as was fixed by Helge's patch for the ring
mmaps, as the provided buffer rings work kinda the same way. The
application allocates some aligned memory, and registers it and the
kernel then maps it.
I wonder if these would work properly if the address was aligned to
0x400000? Should be easy to verify, just modify the alignment for the
posix_memalign() calls in test/buf-ring.c.
Doesn't help. Same error. Can you point to where the kernel maps it?
Yep, it goes io_uring.c:io_uring_register() ->
kbuf.c:io_register_pbuf_ring() -> rsrc.c:io_pin_pages() which ultimately
calls pin_user_pages() to map the memory.
Followup - a few of the provided buffer ring cases failed to properly
initialize the ring, poll-mshot-race was one of them... I've pushed out
fixes for this. Not sure if it fixes your particular issue, but worth
giving it another run.
Results are still the same:
Running test file-verify.t Found 163840, wanted 688128
Buffered novec reg test failed
Test file-verify.t failed with ret 1
Tests timed out (2): <a4c0b3decb33.t> <send-zerocopy.t>
Tests failed (4): <buf-ring.t> <file-verify.t> <ringbuf-read.t> <send_recvmsg.t>
poll-mshot-race still causes HPMC.
The timeouts are not a problem. The following change fixed <a4c0b3decb33.t> <send-zerocopy.t>:
diff --git a/test/a4c0b3decb33.c b/test/a4c0b3decb33.c
index f282d1b..6be73b6 100644
--- a/test/a4c0b3decb33.c
+++ b/test/a4c0b3decb33.c
@@ -124,7 +124,7 @@ static void loop(void)
if (waitpid(-1, &status, WNOHANG | WAIT_FLAGS) == pid)
break;
sleep_ms(1);
- if (current_time_ms() - start < 5 * 1000)
+ if (current_time_ms() - start < 100 * 1000)
continue;
kill_and_wait(pid, &status);
break;
diff --git a/test/runtests.sh b/test/runtests.sh
index 924fdce..8c3a4bf 100755
--- a/test/runtests.sh
+++ b/test/runtests.sh
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
TESTS=("$@")
-TIMEOUT=60
+TIMEOUT=300
DMESG_FILTER="cat"
TEST_DIR=$(dirname "$0")
FAILED=()
I believe you are correct about the colouring issue being the problem with the other tests.
I've been playing with the send_recvmsg.t test as it seems the simplest.
On parisc, caches are required to detect that the same physical memory is being accessed by
two virtual addresses if offset bits 42 through 63 are the same in both virtual addresses (i.e.,
the addresses must be equal modulo 0x400000). There is also a constraint on space bits but
space register hashing is disabled, so it doesn't come into play.
We have a linear offset between kernel and physical addresses in linux. So, the user virtual
address must be equivalent to the physical address of a page for user and kernel accesses to
be detected by the caches.
For io_uring to work, I believe the user and kernel addresses used to access the buffers must
be equivalent. However, as far as I can see, we only setup equivalent aliases for file backed
mappings with MAP_SHARED. There doesn't appear to be any connection between the kernel
page addresses allocated for a mapping and the assigned user virtual addresses. Thus, it doesn't
help to align the user virtual address to 0x400000. The kernel virtual address still has the wrong
colour.
Maybe something could by done with anonymous MAP_SHARED mappings to make them equivalent?
The mmap man page says "Support for MAP_ANONYMOUS in conjunction with MAP_SHARED was added
in Linux 2.4."
I tried to use a file based mapping in the send_recvmsg.t (tried both ring.ring_fd and a temporary
file). At first, I thought this worked. But it turns out that pin_user_pages fails and returns -EFAULT.
reg.ring_addr = (unsigned long) ptr;
reg.ring_entries = 1;
reg.bgid = BUF_BGID;
if (io_uring_register_buf_ring(&ring, ®, 0)) {
no_pbuf_ring = 1;
goto out;
}
So, the io_uring_register_buf_ring call fails and the code bails with no error message.
I'm not sure why pin_user_pages fails. Today, I've been wondering if a mlock call would
lock the mmap'd buffer into RAM and fix pin_user_pages?
Dave
--
John David Anglin dave.anglin@xxxxxxxx
