On Mon, Jun 10, 2024 at 3:20 PM Jason Gunthorpe <jgg@xxxxxxxx> wrote: > > On Mon, Jun 10, 2024 at 11:48:23AM -0700, Tomasz Jeznach wrote: > > > Right, this is why I asked about negative caching. > > > > > > The VMMs are a prime example of negative caching, in something like > > > the SMMU implementation the VMM will cache the V=0 STE until they see > > > an invalidation. > > > > > > Driving the VMM shadowing/caching entirely off of the standard > > > invalidation mechanism is so much better than any other option. > > > > > > IMHO you should have the RISCV spec revised to allow negative caching > > > in any invalidated data structure to permit the typical VMM design > > > driven off of shadowing triggered by invalidation commands. > > > > > > Once the spec permits negative caching then the software would have to > > > invalidate after going V=0 -> V=1. > > > > > > Jason > > > > Allowing negative cacheing by the spec (e.g. for VMM use cases) and > > documenting required invalidation sequences would definitely help > > here. > > Yes, you probably should really do that. > > > I'm hesitating adding IODIR.INVAL that is not required by the > > spec [1], > > If you expect to rapidly revise the spec then you should add it right > now so that all SW implementations that exist are > conforming. Otherwise you'll have compatability problems when you come > to implement nesting. > > Obviously the VMM can't rely on a negative caching technique unless > the spec says it can. > > > but this is something that can be controlled by a > > capabilities/feature bit once added to the specification or based on > > VID:DID of the emulated Risc-V IOMMU. > > I'm not sure it really can. Once you start shipping SW people will run > it in a VM and the VMM will have to forever work without negative > caching. > > My strong advice is to not expect the VMM trap random pages in guest > memory, that is a huge mess to implement and will delay your VMM side. > > > Another option to consider for VMM is to utilize the WARL property of > > DDTP, and provide fixed location of the single level DDTP, pointing to > > MMIO region, where DDTE updates will result in vmm exit / fault > > handler. This will likely not be as efficient as IODIR.INVAL issued > > for any DDTE updates. > > I don't know what all those things mean, but if you mean to have the > VMM supply faulting MMIO space that the VM is forced to put the DDTE > table into, then that would be better. It is still quite abnormal from > the VMM side.. > > My strong advice is to fix this. It is trivial to add the negative > caching language to the spec and will cause insignificant extra work > in this driver. > > The gains on at least the ease of VMM implementation and architectural > similarity to other arches are well worth the effort. Otherwise I fear > it will be a struggle to get nesting support completed :( > > Jason > Thanks for the comments, it definitely makes sense. We will revise the wording in the RISC-V IOMMU spec to note that certain implementations may require invalidating commands to be issued when making DDT entries valid, and device tree / ACPI may be used to report such requirement. For now, I'll change the implementation to assume negative caching for DDTE and will follow up with device tree / driver updates to make the invalidation optional when revised specifications will be available. Best, - Tomasz > -- > You received this message because you are subscribed to the Google Groups "linux" group. > To unsubscribe from this group and stop receiving emails from it, send an email to linux+unsubscribe@xxxxxxxxxxxx. > To view this discussion on the web visit https://groups.google.com/a/rivosinc.com/d/msgid/linux/20240610222051.GO791043%40ziepe.ca. > For more options, visit https://groups.google.com/a/rivosinc.com/d/optout.
