On 10/19/20 10:25 AM, Jason Gunthorpe wrote: > Hi Tom, Hi Jason, > > We've found a bug where systems that have the AMD SME turned on are > not able to run RDMA work loads. It seems the kernel is automatically > encrypting VMA's pointing at PCI BAR memory created by > io_remap_pfn_range() - adding a prot_decrypted() causes things to > start working. > > To me this is surprising, before I go adding random prot_decrypted() > into the RDMA subsystem can you confirm this is actually how things > are expected to work? Yes, currently, the idea is that anything being done in user space is mapped encrypted. > > Is RDMA missing something? I don't see anything special in VFIO for > instance and the two are very similar - does VFIO work with SME, eg > DPDK or something unrelated to virtualization? If user space is mapping un-encrypted memory, then, yes, it would seem that there is a gap in the support where the pgprot_decrypted() would be needed in order to override the protection map. > > Is there a reason not to just add prot_decrypted() to > io_remap_pfn_range()? Is there use cases where a caller actually wants > encrypted io memory? As long as you never have physical memory / ram being mapped in this path, it seems that applying pgprot_decrypted() would be ok. > > I saw your original patch series edited a few drivers this way, but > not nearly enough. So I feel like I'm missing something.. Does vfio > work with SME? I couldn't find any sign of it calling prot_decrypted() > either? I haven't tested SME with VFIO/DPDK. > > (BTW, I don't have any AMD SME systems to test on here, I'm getting > this bug report from deployed system, running a distro kernel) As a work around, if the system has support for TSME (transparent SME), then that can be enabled (it is a BIOS option that the BIOS vendor would have had to expose) to encrypt all of the system memory without requiring SME support. Thanks, Tom > > Thanks, > Jason >
