Hi Marc, > From: Marc Zyngier <maz@xxxxxxxxxx> > Hi Vikram, > > The problem I see is that we have VM and userspace being written in terms > of Write-Combine, which is: > > - loosely defined even on x86 > > - subject to interpretations in the way it maps to PCI > > - has no direct equivalent in the ARMv8 collection of memory > attributes (and Normal_NC comes with speculation capabilities which > strikes me as extremely undesirable on arbitrary devices) If speculation with Normal NC to prefetchable BARs in devices was a problem, those devices would already be broken in baremetal with ioremap_wc on arm64, and we would need quirks there to not do Normal NC for them but Device GRE, and if such a quirk was needed on baremetal, it could be picked up by vfio/KVM as well. But we haven't seen any broken devices doing wc on baremetal on ARM64, have we? I know we have tested NICs write combining on arm64 in baremetal, as well as GPU and NVMe CMB without issues. Further, I don't see why speculation to non cacheble would be an issue if prefetch without side effects is allowed by the device, which is what a prefetchable BAR is. If it is an issue for a device I would consider that a bug already needing a quirk in Baremetal/host kernel already. >From PCI spec " A prefetchable address range may have write side effects, but it may not have read side effects." > > How do we translate this into something consistent? I'd like to see an actual > description of what we *really* expect from WC on prefetchable PCI regions, > turn that into a documented definition agreed across architectures, and then > we can look at implementing it with one memory type or another on arm64. > > Because once we expose that memory type at S2 for KVM guests, it > becomes ABI and there is no turning back. So I want to get it right once and > for all. > I agree that we need a precise definition for the Linux ioremap_wc API wrt what drivers (kernel and userspace) can expect and whether memset/memcpy is expected to work or not and whether aligned accesses are a requirement. To the extent ABI is set, I would think that the ABI is also already set in the host kernel for arm64 WC = Normal NC, so why should that not also be the ABI for same driver in VMs. > Thanks, > > M. > > -- > Without deviation from the norm, progress is not possible.
