On Tue, Aug 2, 2011 at 9:24 AM, Jan Kiszka <jan.kiszka@xxxxxxxxxxx> wrote: > On 2011-08-02 17:14, Bjorn Helgaas wrote: >> On Mon, Aug 1, 2011 at 9:47 PM, Nikolaev, Maxim >> <maxim.nikolaev@xxxxxxxxxxx> wrote: >>> Hello Folks, >>> >>> I work on passing-thought nVidia PCI video card from Linux host to >>> Windows guest and was able to do it for FX-3800 but I faced the problem >>> with Quadro 2000. The problem is one of Quadro 2000 BAR's conflicts with >>> memory reported by BIOS-e820 as reserved. Basically they are overlapped >>> and kvm does not want to go further. dmesg, /proc/iomem and lspci are at >>> the end of this email. >>> The memory reserved by BIOS-e820 >>> [ 0.000000] BIOS-e820: 00000000fa000000 - 00000000fc000000 >>> (reserved) >>> Quadro 2000 is 05:00.0 and its conflicting BAR is >>> Region 3: Memory at f8000000 (64-bit, prefetchable) [disabled] >>> [size=64M] >>> and its upstream bridge 00:1c.00 >>> >>> I have a number of questions and will appreciate you a lot for >>> clarification >>> 1. Is this conflict a really problem of kernel PCI? (I just assume that >>> maybe kvm does not handle this case properly) >>> 2. If it is a kernel problem/feature then how I can avoid it? Where BAR >>> allocation is located in kernel source code? I enabled pci=earlydump and >>> see that PCI config space is the same at early boot stage and at the >>> entire kernel boot up that means at least in my case kernel does nothing >>> with BAR's allocated by BIOS. I've tried different kernel PCI options: >>> nocrs, use_crs, noacpi with the same results: BAR's are always located >>> at the same addresses. What I need is to make kernel to allocate BAR's >>> at different addresses that does not overlap with BIOS-e820 region. How >>> to make kernel to rearrange BAR's the other way then it is done in BIOS? >> >> Linux is quite strict about the E820 table. For instance, it puts >> E820 reserved areas in the iomem_resource tree, as you can see from >> /proc/iomem. We apparently don't move a BAR that conflicts with an >> E820 reserved area, but as you can see, a driver has trouble trying to >> claim the BAR. >> >> My experience is that Windows handles E820 reservations differently >> than Linux does. As far as I can tell, E820 has no connection at all >> to PCI resources (one example is here: >> https://bugzilla.kernel.org/show_bug.cgi?id=16228#c45). >> >> I expect that the Quadro 2000 probably works fine at [mem >> 0xf8000000-0xfbffffff] under Windows. Does it work under Linux as a >> host device? I suspect that might be broken too, because the nvidia >> driver would probably hit the same reservation issue that pci-stub >> hits. >> >> I think we might be better off if Linux kept track of E820 >> reservations off to the side, in such a way that they don't show up in >> iomem_resource, so they don't cause reservation problems like this, >> but at the same time, we don't allocate E820-reserved space when we're >> assigning a BAR. >> >> Sorry that this is really a philosophical answer, not a practical one, >> so it doesn't help solve your immediate problem. It would be nice to >> have a generic "pci=" argument to force reassignment of a BAR. Maybe >> that would be enough to limp along for a while. > > Why would it take an additional kernel parameter? Can't the PCI > subsystem check for such conflicts during boot (instead of doing this on > resource requests) and run a reassignment automatically? I presume that > would happen as well if you hot-plug that device. My point is that I think we want to do *less* with E820 rather than more. We could certainly discover this "conflict" during boot and reassign then. But that would make Linux behave less like Windows, and generally we're safer if we behave more like Windows. I think if we start moving every PCI BAR that conflicts with an E820 reserved area, we're likely to uncover more problems than we fix. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
