On Sun, Oct 01, 2017 at 09:42:03AM +0200, Christoph Hellwig wrote:
> This looks very convoluted, mostly because the existing code is
> doing weird things. For one thing what is sq_dma_addr currently
> is not a DMA adddress - we either need the resource address
> for the ioremap, but we don't need to stash that away, and second
> the one programmed into the controller should be a pci_bus_addr_t.
>
> Second we already have a nice PCI-layer helper called pci_bus_address
> to get the bus address for us and we should use it.
>
> Something like the patch below should solve the issue:
Yah, calling this a DMA address was a misnomer and confusing.
> ---
> From b78f4164881125c4fecfdb87878d0120b2177c53 Mon Sep 17 00:00:00 2001
> From: Christoph Hellwig <hch@xxxxxx>
> Date: Sun, 1 Oct 2017 09:37:35 +0200
> Subject: nvme-pci: Use PCI bus address for data/queues in CMB
>
> Currently, NVMe PCI host driver is programming CMB dma address as
> I/O SQs addresses. This results in failures on systems where 1:1
> outbound mapping is not used (example Broadcom iProc SOCs) because
> CMB BAR will be progammed with PCI bus address but NVMe PCI EP will
> try to access CMB using dma address.
>
> To have CMB working on systems without 1:1 outbound mapping, we
> program PCI bus address for I/O SQs instead of dma address. This
> approach will work on systems with/without 1:1 outbound mapping.
>
> Based on a report and previous patch from Abhishek Shah.
>
> Fixes: 8ffaadf7 ("NVMe: Use CMB for the IO SQes if available")
> Cc: stable@xxxxxxxxxxxxxxx
> Reported-by: Abhishek Shah <abhishek.shah@xxxxxxxxxxxx>
> Signed-off-by: Christoph Hellwig <hch@xxxxxx>
This looks good.
Reviewed-by: Keith Busch <keith.busch@xxxxxxxxx>