On Sat, Jan 04, 2025 at 02:30:42PM -0800, Bart Van Assche wrote: > On 1/3/25 8:04 PM, Luis Chamberlain wrote: > > Try aiming high for a single 2 MiB for a single IO on x86_64 on NVMe, that is > > currently not possible. At the max 128 NVMe number of DMA segments, and we have > > 4 KiB per DMA segment, for a 512 KiB IO limit. Should multi-page bvec > > enable to lift this? > > 4 KiB per DMA segment for NVMe? I think that the DMA segment size limit for > PRP and SGL modes is much larger than 4 KiB. Of course in terms of device capability, I was referring to what we support in software. > See also the > description of the CC.MPS parameter and PRP Lists in the NVMe base > specification. From a system with an NVMe controller: > > $ cat /sys/block/nvme0n1/queue/max_segment_size > 4294967295 To enable such things you need either huge pages or large folios, and for a deterministic large dma segment you can use min-order, my experimentation shows we have a bit of work to lever really large dma segments above 64k. Luis
