Re: pcie: kernel log - BAR 15: no space for... BAR 15: failed to assign..

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Feb 12, 2020 at 1:46 AM Bjorn Helgaas <helgaas@xxxxxxxxxx> wrote:
>
> On Tue, Feb 11, 2020 at 10:02:13PM +0530, Muni Sekhar wrote:
> > On Tue, Feb 11, 2020 at 8:10 PM Bjorn Helgaas <helgaas@xxxxxxxxxx> wrote:
> > > On Tue, Feb 11, 2020 at 07:36:00PM +0530, Muni Sekhar wrote:
> > > > On Tue, Feb 11, 2020 at 3:58 AM Bjorn Helgaas <helgaas@xxxxxxxxxx> wrote:
> > > > > On Sun, Feb 09, 2020 at 07:59:41AM +0530, Muni Sekhar wrote:
> > > > > > Hi all,
> > > > > >
> > > > > > After rebooting the system following messages are seen in dmesg.
> > > > > > Not sure if these indicate a problem. Can some one look at these and
> > > > > > confirm if this is problem or can be ignored ?
> > > > > >
> > > > > > Also any suggestions as to what would cause this?
> > > > > >
> > > > > > [    1.084728] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.084813] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.084890] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > > > > > [    1.084949] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > > > > > [    1.085037] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085108] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085199] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085270] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085343] pci 0000:00:1c.0: BAR 13: assigned [io  0x1000-0x1fff]
> > > > > > [    1.085403] pci 0000:00:1c.2: BAR 13: assigned [io  0x2000-0x2fff]
> > > > > > [    1.085470] pci 0000:00:1c.3: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085540] pci 0000:00:1c.3: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085613] pci 0000:00:1c.2: BAR 14: no space for [mem size 0x00200000]
> > > > > > [    1.085672] pci 0000:00:1c.2: BAR 14: failed to assign [mem size 0x00200000]
> > > > > > [    1.085738] pci 0000:00:1c.2: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085808] pci 0000:00:1c.2: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085884] pci 0000:00:1c.0: BAR 15: no space for [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.085954] pci 0000:00:1c.0: BAR 15: failed to assign [mem size
> > > > > > 0x00200000 64bit pref]
> > > > > > [    1.086026] pci 0000:00:1c.0: PCI bridge to [bus 01]
> > > > > > [    1.086083] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
> > > > > > [    1.086144] pci 0000:00:1c.0:   bridge window [mem 0xd0400000-0xd07fffff]
> > > > >
> > > > > The "no space" and "failed to assign" messages are all for bridge
> > > > > windows (13 is the I/O window, 14 is the MMIO window, 15 is the MMIO
> > > > > pref window).  I can't tell if you have any devices below these
> > > > > bridges (lspci would show them).  If you don't have any devices below
> > > > > these bridges, you can ignore the messages.
> > > >
> > > > I have the devices below these bridges. FPGA endpoint is connected to
> > > > '00:1c.0 PCI bridge' and Ethernet controller is connected to '00:1c.3
> > > > PCI bridge'.
> > > > Does these messages impact the functionality of the connected devices?
> > >
> > > Yes.  We tried to allocate 0x00200000 of prefetchable MMIO to 00:1c.0
> > > for use by the FPGA endpoint.  But this failed, so there is no
> > > prefetchable MMIO available for the FPGA.  The 0xd0400000-0xd07fffff
> > > non-prefetachable MMIO space *is* available for it.
> > >
> > > Similarly, we were unable to allocate 0x00200000 of prefetchable MMIO
> > > for the 00:1c.3 bridge for use by the ethernet controller.  I don't
> > > know what non-prefetchable MMIO space was allocated or what the NIC
> > > needs.
> > >
> > > "lspci -v" will show you what the FPGA and the NIC need.
> > >
> > > > If so what kind of impact and is there any solution for this?
> > >
> > > > Also, I'd like to know why "no space" and "failed to assign"
> > > > messages displayed?
> > >
> > > These messages mean we tried to allocate space for the bridges but we
> > > unable to do so.  This is because either the host bridge didn't have
> > > big enough apertures (on x86, these usually come from ACPI _CRS
> > > methods), or there's a bug in the Linux allocation algorithms.
> > >
> > > I can't tell anything more without seeing the complete dmesg log,
> > > which contains the host bridge aperture information and the BAR sizes
> > > of the FPGA and NIC.
> > Thank you for the clarification. dmesg log is attached.
> > If PCI bridge aperture is small, What is the most sensible way to proceed?
>
> Here's the relevant info from your dmesg log:
>
>   PCI host bridge to bus 0000:00
>   pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
>   pci_bus 0000:00: root bus resource [mem 0xc0000000-0xd0c18ffe window]
>   pci_bus 0000:00: root bus resource [bus 00-ff]
>   pci 0000:00:1c.0: PCI bridge to [bus 01]
>   pci 0000:00:1c.0:   bridge window [mem 0xd0400000-0xd07fffff]
>   pci 0000:01:00.0: [1556:5555] type 00 class 0x050000 # FPGA
>   pci 0000:01:00.0: reg 0x10: [mem 0xd0400000-0xd07fffff]
>   pci 0000:00:1c.3: PCI bridge to [bus 03]
>   pci 0000:00:1c.3:   bridge window [io  0xd000-0xdfff]
>   pci 0000:00:1c.3:   bridge window [mem 0xd0a00000-0xd0bfffff]
>   pci 0000:03:00.0: [8086:1533] type 00 class 0x020000 # I210 NIC
>   pci 0000:03:00.0: reg 0x10: [mem 0xd0a00000-0xd0afffff]
>   pci 0000:03:00.0: reg 0x18: [io  0xd000-0xd01f]
>   pci 0000:03:00.0: reg 0x1c: [mem 0xd0b00000-0xd0b03fff]
>
> The FPGA at 01:00.0 has one non-prefetchable memory BAR and is
> assigned [mem 0xd0400000-0xd07fffff].  That range is inside the host
> bridge aperture ([mem 0xc0000000-0xd0c18ffe window]) and is routed
> through the 00:1c.0 bridge (bridge window [mem 0xd0400000-0xd07fffff]).
> So the FPGA resources are fine.
>
> The fact that Linux tried and failed to allocate prefetchable memory
> space for the bridge is immaterial because the FPGA can't use
> prefetchable memory anyway.
>
> The NIC at 03:00.0 is similar: it has two non-prefetchable memory BARs
> and one I/O BAR.  They're all assigned valid space, and the fact that
> Linux tried and failed to allocate prefetchable memory space for the
> bridge makes no difference because the NIC can't use it.
>
> So these messages do not indicate a problem.  Maybe there's something
> we can do to make this more clear to the user.
Thank you for the detailed explanation.
>
> Bjorn



-- 
Thanks,
Sekhar



[Index of Archives]     [DMA Engine]     [Linux Coverity]     [Linux USB]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [Greybus]

  Powered by Linux