On Thu, Oct 13, 2016 at 04:16:03PM +0200, Alexander Gordeev wrote:
> On Thu, Oct 13, 2016 at 02:40:35PM +0800, Peter Xu wrote:
> > > > > + return pci_translate_addr(dev, addr);
> > > >
> > > > Raw question: do we need to translate bar addresses as well?
> > >
> > > I believe, yes.
> > >
> > > Unless we always have identity mapping between PCI address space and
> > > CPU physical address space I can not realize how could it be done
> > > otherwise. But even if we were, I would leave the translation routine
> > > for clarity.
> >
> > Sorry I didn't quite catch your point. Are we talking about IOMMU
> > address remapping here? IMHO BAR addresses are from CPU's point of
> > view. It's only used by CPU, not device. In that case, BAR address
> > should not be translated at least by IOMMU (no matter for x86/arm or
> > whatever).
> >
> > Take Linux as example: pci_ioremap_bar() is responsible to be called
> > for any PCI drivers to map device memory bars into kernel virtual
> > address space. Basically it does:
> >
> > void __iomem *pci_ioremap_bar(struct pci_dev *pdev, int bar)
> > {
> > struct resource *res = &pdev->resource[bar];
> > return ioremap_nocache(res->start, resource_size(res));
> > }
> >
> > So as it is written: I believe we don't translate the bar address
> > (which should be res->start). We use it as physical address.
> >
> > Or, do you mean other kinds of translation that I don't aware of?
>
> Yes, I mean translation from PCI bus address space to CPU physical
> address space. These two busses are different and hence need a
> translation. I assume Linux pci_dev::resource[] have translated
> address, but it is not what PCI devices see. Unless I do not terribly
> missing somethig, BAR addresses is what a device sees on its AD[0..31]
> pins.
I believe pci_dev::resource[] should be assigned by BIOS or something
before Linux. At that time, IOMMU is possibly even not inited. So no
chance for a translation at all.
If you see PCI Local Bus Spec Rev 3.0, chap 6.2.5.1:
Power-up software needs to build a consistent address map before
booting the machine to an operating system. This means it has to
determine how much memory is in the system, and how much address
space the I/O controllers in the system require. After determining
this information, power-up software can map the I/O controllers
into reasonable locations and proceed with system boot. In order
to do this mapping in a device independent manner, the base
registers for this mapping are placed in the predefined header
portion of Configuration Space.
BARs for each PCI devices should be pre-allocated during power-up
software, and a consistent map is built with the knowledge of existing
RAM in the system.
If you boot a VM with/without IOMMU, you'll see that BAR addresses
won't change before/after enabling IOMMU.
Thanks,
-- peterx
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
