On Wed, Jul 10, 2024 at 8:08 AM Lei Liu <liulei.rjpt@xxxxxxxx> wrote:
on 2024/7/10 22:48, Christian König wrote:
Am 10.07.24 um 16:35 schrieb Lei Liu:
on 2024/7/10 22:14, Christian König wrote:
Am 10.07.24 um 15:57 schrieb Lei Liu:
Use vm_insert_page to establish a mapping for the memory allocated
by dmabuf, thus supporting direct I/O read and write; and fix the
issue of incorrect memory statistics after mapping dmabuf memory.
Well big NAK to that! Direct I/O is intentionally disabled on
DMA-bufs.
Hello! Could you explain why direct_io is disabled on DMABUF? Is
there any historical reason for this?
It's basically one of the most fundamental design decision of DMA-Buf.
The attachment/map/fence model DMA-buf uses is not really compatible
with direct I/O on the underlying pages.
Thank you! Is there any related documentation on this? I would like to
understand and learn more about the fundamental reasons for the lack of
support.
Hi Lei and Christian,
This is now the third request I've seen from three different companies
who are interested in this,
but the others are not for reasons of read
performance that you mention in the commit message on your first
patch. Someone else at Google ran a comparison between a normal read()
and a direct I/O read() into a preallocated user buffer and found that
with large readahead (16 MB) the throughput can actually be slightly
higher than direct I/O. If you have concerns about read performance,
have you tried increasing the readahead size?
The other motivation is to load a gajillion byte file from disk into a
dmabuf without evicting the entire contents of pagecache while doing
so. Something like this (which does not currently work because read()
tries to GUP on the dmabuf memory as you mention):
static int dmabuf_heap_alloc(int heap_fd, size_t len)
{
struct dma_heap_allocation_data data = {
.len = len,
.fd = 0,
.fd_flags = O_RDWR | O_CLOEXEC,
.heap_flags = 0,
};
int ret = ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &data);
if (ret < 0)
return ret;
return data.fd;
}
int main(int, char **argv)
{
const char *file_path = argv[1];
printf("File: %s\n", file_path);
int file_fd = open(file_path, O_RDONLY | O_DIRECT);
struct stat st;
stat(file_path, &st);
ssize_t file_size = st.st_size;
ssize_t aligned_size = (file_size + 4095) & ~4095;
printf("File size: %zd Aligned size: %zd\n", file_size,
aligned_size);
int heap_fd = open("/dev/dma_heap/system", O_RDONLY);
int dmabuf_fd = dmabuf_heap_alloc(heap_fd, aligned_size);
void *vm = mmap(nullptr, aligned_size, PROT_READ | PROT_WRITE,
MAP_SHARED, dmabuf_fd, 0);
printf("VM at 0x%lx\n", (unsigned long)vm);
dma_buf_sync sync_flags { DMA_BUF_SYNC_START |
DMA_BUF_SYNC_READ | DMA_BUF_SYNC_WRITE };
ioctl(dmabuf_fd, DMA_BUF_IOCTL_SYNC, &sync_flags);
ssize_t rc = read(file_fd, vm, file_size);
printf("Read: %zd %s\n", rc, rc < 0 ? strerror(errno) : "");
sync_flags.flags = DMA_BUF_SYNC_END | DMA_BUF_SYNC_READ |
DMA_BUF_SYNC_WRITE;
ioctl(dmabuf_fd, DMA_BUF_IOCTL_SYNC, &sync_flags);
}
Or replace the mmap() + read() with sendfile().
