On Tue, Jun 18, 2024 at 08:41:09AM +0900, Shunsuke Mie wrote: > Let's clarify the situation. > > The Virtio device and driver are not working properly due to a > combination of the following reasons: > > 1. Regarding VIRTIO_F_ACCESS_PLATFORM: > - The modern spec includes VIRTIO_F_ACCESS_PLATFORM, which allows > Physical DMAC to be used. > - This feature is not available in the legacy spec. > > 2. Regarding Virtio PCIe Capability: > - The modern spec requires Virtio PCIe Capability. > - In some environments, Virtio PCIe Capability cannot be provided. > > Ideas to solve this problem: > 1. Introduce an ACCESS_PLATFORM-like flag in the legacy spec: > There are some unused bits, but it may be difficult to make changes to > the legacy spec at this stage. > For sure this is a no-go based on the response from Michael. But if Linux Virtio maintainers were willing to add some flexibility for legacy devices to use DMA APIs as like xen-domain (I haven't checked in detail) you previously mentioned, then it can be helpful. > 2. Mani's Idea: > I think it is best to add support for modern virtio PCI device to make > use of IOMMU. Legacy devices can continue to use physical address. > It is same as option 3. Legacy devices will continue to use physical address and once we add support for modern virtio device, it can use IOMMU translation. So the conclusion is, if the platform has IOMMU and doesn't allow configuring the vendor specific capability, then it cannot be supported as is in upstream Linux Endpoint framework (unless some changes happens in the Linux virtio stack). - Mani > The meaning of "Legacy devices can continue to use physical address" > is not fully understood. @mani Could you explain more? > > 3. Wait until the HW supports the modern spec: > This depends on the chip vendor. > > Option 3 is essentially doing nothing, so it would be preferable to > consider other ideas. > > Best, > Shunsuke > > 2024年6月14日(金) 18:50 Manivannan Sadhasivam <mani@xxxxxxxxxx>: > > > > On Mon, May 20, 2024 at 09:22:54AM -0400, Michael S. Tsirkin wrote: > > > On Thu, May 16, 2024 at 02:59:13PM +0200, Manivannan Sadhasivam wrote: > > > > On Thu, May 16, 2024 at 01:38:40PM +0900, Shunsuke Mie wrote: > > > > > Hi virtio folks, > > > > > > > > > > > > > You forgot to CC the actual Virtio folks. I've CCed them now. > > > > > > > > > I'm writing to discuss finding a workaround with Virtio drivers and legacy > > > > > devices with limited memory access. > > > > > > > > > > # Background > > > > > The Virtio specification defines a feature (VIRTIO_F_ACCESS_PLATFORM) to > > > > > indicate devices requiring restricted memory access or IOMMU translation. This > > > > > feature bit resides at position 33 in the 64-bit Features register on modern > > > > > interfaces. When the linux virtio driver finds the flag, the driver uses DMA > > > > > API that handles to use of appropriate memory. > > > > > > > > > > # Problem > > > > > However, legacy devices only have a 32-bit register for the features bits. > > > > > Consequently, these devices cannot represent the ACCESS_PLATFORM bit. As a > > > > > result, legacy devices with restricted memory access cannot function > > > > > properly[1]. This is a legacy spec issue, but I'd like to find a workaround. > > > > > > > > > > # Proposed Solutions > > > > > I know these are not ideal, but I propose the following solution. > > > > > Driver-side: > > > > > - Implement special handling similar to xen_domain. > > > > > In xen_domain, linux virtio driver enables to use the DMA API. > > > > > - Introduce a CONFIG option to adjust the DMA API behavior. > > > > > Device-side: > > > > > Due to indistinguishability from the guest's perspective, a device-side > > > > > solution might be difficult. > > > > > > > > > > I'm open to any comments or suggestions you may have on this issue or > > > > > alternative approaches. > > > > > > > > > > [1] virtio-net PCI endpoint function using PCIe Endpoint Framework, > > > > > https://lore.kernel.org/lkml/54ee46c3-c845-3df3-8ba0-0ee79a2acab1@xxxxxxxxxx/t/ > > > > > The Linux PCIe endpoint framework is used to implement the virtio-net device on > > > > > a legacy interface. This is necessary because of the framework and hardware > > > > > limitation. > > > > > > > > > > > > > We can fix the endpoint framework limitation, but the problem lies with some > > > > platforms where we cannot write to vendor capability registers and still have > > > > IOMMU. > > > > > > > > - Mani > > > > > > What are vendor capability registers and what do they have to do > > > with the IOMMU? > > > > > > > Virtio spec v1.2, sec 4.1.4 says, > > > > "Each structure can be mapped by a Base Address register (BAR) belonging to the > > function, or accessed via the special VIRTIO_PCI_CAP_PCI_CFG field in the PCI > > configuration space. > > > > The location of each structure is specified using a vendor-specific PCI > > capability located on the capability list in PCI configuration space of the > > device." > > > > So this means the device has to expose the virtio structures through vendor > > specific capability isn't it? > > > > And only in that case, it can expose VIRTIO_F_ACCESS_PLATFORM bit for making > > use of IOMMU translation. > > > > - Mani > > > > -- > > மணிவண்ணன் சதாசிவம் -- மணிவண்ணன் சதாசிவம்
