On Mon, Aug 1, 2011 at 9:47 PM, Nikolaev, Maxim <maxim.nikolaev@xxxxxxxxxxx> wrote: > Hello Folks, > > I work on passing-thought nVidia PCI video card from Linux host to > Windows guest and was able to do it for FX-3800 but I faced the problem > with Quadro 2000. The problem is one of Quadro 2000 BAR's conflicts with > memory reported by BIOS-e820 as reserved. Basically they are overlapped > and kvm does not want to go further. dmesg, /proc/iomem and lspci are at > the end of this email. > The memory reserved by BIOS-e820 > [ 0.000000] BIOS-e820: 00000000fa000000 - 00000000fc000000 > (reserved) > Quadro 2000 is 05:00.0 and its conflicting BAR is > Region 3: Memory at f8000000 (64-bit, prefetchable) [disabled] > [size=64M] > and its upstream bridge 00:1c.00 > > I have a number of questions and will appreciate you a lot for > clarification > 1. Is this conflict a really problem of kernel PCI? (I just assume that > maybe kvm does not handle this case properly) > 2. If it is a kernel problem/feature then how I can avoid it? Where BAR > allocation is located in kernel source code? I enabled pci=earlydump and > see that PCI config space is the same at early boot stage and at the > entire kernel boot up that means at least in my case kernel does nothing > with BAR's allocated by BIOS. I've tried different kernel PCI options: > nocrs, use_crs, noacpi with the same results: BAR's are always located > at the same addresses. What I need is to make kernel to allocate BAR's > at different addresses that does not overlap with BIOS-e820 region. How > to make kernel to rearrange BAR's the other way then it is done in BIOS? Linux is quite strict about the E820 table. For instance, it puts E820 reserved areas in the iomem_resource tree, as you can see from /proc/iomem. We apparently don't move a BAR that conflicts with an E820 reserved area, but as you can see, a driver has trouble trying to claim the BAR. My experience is that Windows handles E820 reservations differently than Linux does. As far as I can tell, E820 has no connection at all to PCI resources (one example is here: https://bugzilla.kernel.org/show_bug.cgi?id=16228#c45). I expect that the Quadro 2000 probably works fine at [mem 0xf8000000-0xfbffffff] under Windows. Does it work under Linux as a host device? I suspect that might be broken too, because the nvidia driver would probably hit the same reservation issue that pci-stub hits. I think we might be better off if Linux kept track of E820 reservations off to the side, in such a way that they don't show up in iomem_resource, so they don't cause reservation problems like this, but at the same time, we don't allocate E820-reserved space when we're assigning a BAR. Sorry that this is really a philosophical answer, not a practical one, so it doesn't help solve your immediate problem. It would be nice to have a generic "pci=" argument to force reassignment of a BAR. Maybe that would be enough to limp along for a while. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
