On 2011-08-02 18:22, Bjorn Helgaas wrote: > On Tue, Aug 2, 2011 at 9:24 AM, Jan Kiszka <jan.kiszka@xxxxxxxxxxx> wrote: >> On 2011-08-02 17:14, Bjorn Helgaas wrote: >>> On Mon, Aug 1, 2011 at 9:47 PM, Nikolaev, Maxim >>> <maxim.nikolaev@xxxxxxxxxxx> wrote: >>>> Hello Folks, >>>> >>>> I work on passing-thought nVidia PCI video card from Linux host to >>>> Windows guest and was able to do it for FX-3800 but I faced the problem >>>> with Quadro 2000. The problem is one of Quadro 2000 BAR's conflicts with >>>> memory reported by BIOS-e820 as reserved. Basically they are overlapped >>>> and kvm does not want to go further. dmesg, /proc/iomem and lspci are at >>>> the end of this email. >>>> The memory reserved by BIOS-e820 >>>> [ 0.000000] BIOS-e820: 00000000fa000000 - 00000000fc000000 >>>> (reserved) >>>> Quadro 2000 is 05:00.0 and its conflicting BAR is >>>> Region 3: Memory at f8000000 (64-bit, prefetchable) [disabled] >>>> [size=64M] >>>> and its upstream bridge 00:1c.00 >>>> >>>> I have a number of questions and will appreciate you a lot for >>>> clarification >>>> 1. Is this conflict a really problem of kernel PCI? (I just assume that >>>> maybe kvm does not handle this case properly) >>>> 2. If it is a kernel problem/feature then how I can avoid it? Where BAR >>>> allocation is located in kernel source code? I enabled pci=earlydump and >>>> see that PCI config space is the same at early boot stage and at the >>>> entire kernel boot up that means at least in my case kernel does nothing >>>> with BAR's allocated by BIOS. I've tried different kernel PCI options: >>>> nocrs, use_crs, noacpi with the same results: BAR's are always located >>>> at the same addresses. What I need is to make kernel to allocate BAR's >>>> at different addresses that does not overlap with BIOS-e820 region. How >>>> to make kernel to rearrange BAR's the other way then it is done in BIOS? >>> >>> Linux is quite strict about the E820 table. For instance, it puts >>> E820 reserved areas in the iomem_resource tree, as you can see from >>> /proc/iomem. We apparently don't move a BAR that conflicts with an >>> E820 reserved area, but as you can see, a driver has trouble trying to >>> claim the BAR. >>> >>> My experience is that Windows handles E820 reservations differently >>> than Linux does. As far as I can tell, E820 has no connection at all >>> to PCI resources (one example is here: >>> https://bugzilla.kernel.org/show_bug.cgi?id=16228#c45). >>> >>> I expect that the Quadro 2000 probably works fine at [mem >>> 0xf8000000-0xfbffffff] under Windows. Does it work under Linux as a >>> host device? I suspect that might be broken too, because the nvidia >>> driver would probably hit the same reservation issue that pci-stub >>> hits. >>> >>> I think we might be better off if Linux kept track of E820 >>> reservations off to the side, in such a way that they don't show up in >>> iomem_resource, so they don't cause reservation problems like this, >>> but at the same time, we don't allocate E820-reserved space when we're >>> assigning a BAR. >>> >>> Sorry that this is really a philosophical answer, not a practical one, >>> so it doesn't help solve your immediate problem. It would be nice to >>> have a generic "pci=" argument to force reassignment of a BAR. Maybe >>> that would be enough to limp along for a while. >> >> Why would it take an additional kernel parameter? Can't the PCI >> subsystem check for such conflicts during boot (instead of doing this on >> resource requests) and run a reassignment automatically? I presume that >> would happen as well if you hot-plug that device. > > My point is that I think we want to do *less* with E820 rather than > more. We could certainly discover this "conflict" during boot and > reassign then. But that would make Linux behave less like Windows, > and generally we're safer if we behave more like Windows. I think if > we start moving every PCI BAR that conflicts with an E820 reserved > area, we're likely to uncover more problems than we fix. OK, if it's safe to ignore for PCI allocations, then ignore the reservation. But I think that should be done automatically if somehow possible, not by the user via a kernel parameter. Is there a simple way to test such a policy without significant refactoring of the e820 code? Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
