On Thu, Jun 02, 2011 at 11:19:44AM -0700, Ram Pai wrote: > On Wed, Jun 01, 2011 at 01:50:28PM -0700, Yinghai Lu wrote: > > On 06/01/2011 01:32 PM, Ben Hutchings wrote: > > > On Tue, 2011-05-31 at 23:17 -0700, Yinghai Lu wrote: > > >> On 05/31/2011 06:10 PM, Ben Hutchings wrote: > > >>> > > >>> In this configuration it can be ignored completely, but I don't think > > >>> there's any generic way to determine that. > > >> > > >> please checking two patches > > > > > > I applied these on top of v3.0-rc1: > > > > > >> [PATCH] PCI: honor child buses add_size in hot plug configuration > > > [...] > > >> [PATCH] PCI: Make assign resource to unassigned SRIOV BAR to be optional > > > [...] > > > > > > After this, PCI resource allocation was successful and the driver and > > > devices passed a basic self-test. > > > > good. > > > > > > > > The changes in output of 'lspci -vn' relative to the previous working > > > version (f8fcfd775523347afe460dc3a0f45d0479e784a2) were: > > > > > > --- /home/bwh/tmp/lspci-good-init.log 2011-05-31 23:30:39.496353000 +0100 > > > +++ /home/bwh/tmp/lspci-fixed-init.log 2011-06-01 21:28:30.547420000 +0100 > > > @@ -270,18 +270,18 @@ > > > Subsystem: 1924:6102 > > > Flags: bus master, fast devsel, latency 0, IRQ 11 > > > I/O ports at b800 [size=256] > > > - Memory at ed000000 (64-bit, non-prefetchable) [size=16M] > > > - Memory at ecfe0000 (64-bit, non-prefetchable) [size=64K] > > > - Expansion ROM at ec000000 [disabled] [size=128K] > > > + Memory at ec000000 (64-bit, non-prefetchable) [size=16M] > > > + Memory at ee040000 (64-bit, non-prefetchable) [size=64K] > > > + Expansion ROM at ee000000 [disabled] [size=128K] > > > Capabilities: <access denied> > > > > > > 0c:00.1 0200: 1924:0813 > > > Subsystem: 1924:6102 > > > Flags: bus master, fast devsel, latency 0, IRQ 11 > > > I/O ports at bc00 [size=256] > > > - Memory at ee000000 (64-bit, non-prefetchable) [size=16M] > > > - Memory at ecff0000 (64-bit, non-prefetchable) [size=64K] > > > - Expansion ROM at ec020000 [disabled] [size=128K] > > > + Memory at ed000000 (64-bit, non-prefetchable) [size=16M] > > > + Memory at ee050000 (64-bit, non-prefetchable) [size=64K] > > > + Expansion ROM at ee020000 [disabled] [size=128K] > > > Capabilities: <access denied> > > > > > > 20:08.0 0604: 1166:0140 (rev a2) > > > @@ -315,17 +315,17 @@ > > > Subsystem: 1924:6205 > > > Flags: bus master, fast devsel, latency 0, IRQ 5 > > > I/O ports at 9800 [size=256] > > > - Memory at d6000000 (64-bit, non-prefetchable) [size=16M] > > > - Memory at d5fe0000 (64-bit, non-prefetchable) [size=64K] > > > - Expansion ROM at d5000000 [disabled] [size=128K] > > > + Memory at d5000000 (64-bit, non-prefetchable) [size=16M] > > > + Memory at d7040000 (64-bit, non-prefetchable) [size=64K] > > > + Expansion ROM at d7000000 [disabled] [size=128K] > > > Capabilities: <access denied> > > > > > > 21:00.1 0200: 1924:0803 > > > Subsystem: 1924:6205 > > > Flags: bus master, fast devsel, latency 0, IRQ 5 > > > I/O ports at 9c00 [size=256] > > > - Memory at d7000000 (64-bit, non-prefetchable) [size=16M] > > > - Memory at d5ff0000 (64-bit, non-prefetchable) [size=64K] > > > - Expansion ROM at d5020000 [disabled] [size=128K] > > > + Memory at d6000000 (64-bit, non-prefetchable) [size=16M] > > > + Memory at d7050000 (64-bit, non-prefetchable) [size=64K] > > > + Expansion ROM at d7020000 [disabled] [size=128K] > > > Capabilities: <access denied> > > > > > > --- END --- > > > > some shuffle around... to make some space for SRIOV registers... > > > > > > > > The boot log messages are below. > > > > > > Ben. > > > > > > Linux version 3.0.0-rc1+ (bwh@bwh-desktop) (gcc version 4.5.1 20100924 (Red Hat 4.5.1-4) (GCC) ) #47 SMP Wed Jun 1 19:40:53 BST 2011 > > > > Linus changed his mind finally. > > > > ... > > > PCI: No. 3 try to assign unassigned res > > > release child resource [mem 0xd5000000-0xd5ffffff 64bit] > > > release child resource [mem 0xd6000000-0xd6ffffff 64bit] > > > release child resource [mem 0xd7000000-0xd701ffff pref] > > > release child resource [mem 0xd7020000-0xd703ffff pref] > > > release child resource [mem 0xd7040000-0xd704ffff 64bit] > > > release child resource [mem 0xd7050000-0xd705ffff 64bit] > > > release child resource [mem 0xd7060000-0xd784ffff 64bit] > > > release child resource [mem 0xd7850000-0xd794dfff 64bit] > > > release child resource [mem 0xd794e000-0xd7a4bfff 64bit] > > > pci 0000:20:08.0: resource 14 [mem 0xd5000000-0xd7ffffff] released > > > pci 0000:20:08.0: PCI bridge to [bus 21-21] > > > pci 0000:20:08.0: bridge window [mem disabled] > > > release child resource [mem 0xec000000-0xecffffff 64bit] > > > release child resource [mem 0xed000000-0xedffffff 64bit] > > > release child resource [mem 0xee000000-0xee01ffff pref] > > > release child resource [mem 0xee020000-0xee03ffff pref] > > > release child resource [mem 0xee040000-0xee04ffff 64bit] > > > release child resource [mem 0xee050000-0xee05ffff 64bit] > > > release child resource [mem 0xee060000-0xee84ffff 64bit] > > > release child resource [mem 0xee850000-0xee94dfff 64bit] > > > release child resource [mem 0xee94e000-0xeea4bfff 64bit] > > > pci 0000:00:08.0: resource 14 [mem 0xec000000-0xeeffffff] released > > > pci 0000:00:08.0: PCI bridge to [bus 0c-0c] > > > pci 0000:00:08.0: bridge window [mem disabled] > > > pci 0000:0c:00.0: reg 184: [mem 0xee94e000-0xee94ffff 64bit] > > > pci 0000:0c:00.0: reg 18c: [mem 0xec040000-0xec04ffff 64bit] > > > pci 0000:0c:00.1: reg 184: [mem 0xee850000-0xee851fff 64bit] > > > pci 0000:0c:00.1: reg 18c: [mem 0xee060000-0xee06ffff 64bit] > > > pci 0000:21:00.0: reg 184: [mem 0xd794e000-0xd794ffff 64bit] > > > pci 0000:21:00.0: reg 18c: [mem 0xd5040000-0xd504ffff 64bit] > > > pci 0000:21:00.1: reg 184: [mem 0xd7850000-0xd7851fff 64bit] > > > pci 0000:21:00.1: reg 18c: [mem 0xd7060000-0xd706ffff 64bit] > > > pci 0000:00:08.0: BAR 14: assigned [mem 0xec000000-0xeeffffff] > > > pci 0000:00:08.0: BAR 15: can't assign mem pref (size 0x100000) > > ... > > > pci 0000:0c:00.0: reg 184: [mem 0xee94e000-0xee94ffff 64bit] > > > pci 0000:0c:00.0: reg 18c: [mem 0xec040000-0xec04ffff 64bit] > > > pci 0000:0c:00.0: reg 184: [mem 0xee94e000-0xee94ffff 64bit] > > > pci 0000:0c:00.0: reg 18c: [mem 0xec040000-0xec04ffff 64bit] > > > pci 0000:0c:00.0: reg 184: [mem 0xee94e000-0xee94ffff 64bit] > > > pci 0000:0c:00.1: reg 184: [mem 0xee850000-0xee851fff 64bit] > > > pci 0000:0c:00.0: reg 18c: [mem 0xec040000-0xec04ffff 64bit] > > > pci 0000:0c:00.0: reg 184: [mem 0xee94e000-0xee94ffff 64bit] > > > pci 0000:0c:00.1: reg 18c: [mem 0xee060000-0xee06ffff 64bit] > > > pci 0000:0c:00.0: reg 18c: [mem 0xec040000-0xec04ffff 64bit] > > > pci 0000:0c:00.0: reg 184: [mem 0xee94e000-0xee94ffff 64bit] > > > pci 0000:0c:00.0: BAR 2: assigned [mem 0xec000000-0xecffffff 64bit] > > > pci 0000:0c:00.0: BAR 2: set to [mem 0xec000000-0xecffffff 64bit] (PCI address [0xec000000-0xecffffff]) > > > pci 0000:0c:00.1: BAR 2: assigned [mem 0xed000000-0xedffffff 64bit] > > > pci 0000:0c:00.1: BAR 2: set to [mem 0xed000000-0xedffffff 64bit] (PCI address [0xed000000-0xedffffff]) > > > pci 0000:0c:00.0: BAR 6: assigned [mem 0xee000000-0xee01ffff pref] > > > pci 0000:0c:00.1: BAR 6: assigned [mem 0xee020000-0xee03ffff pref] > > > pci 0000:0c:00.0: BAR 4: assigned [mem 0xee040000-0xee04ffff 64bit] > > > pci 0000:0c:00.0: BAR 4: set to [mem 0xee040000-0xee04ffff 64bit] (PCI address [0xee040000-0xee04ffff]) > > > pci 0000:0c:00.1: BAR 4: assigned [mem 0xee050000-0xee05ffff 64bit] > > > pci 0000:0c:00.1: BAR 4: set to [mem 0xee050000-0xee05ffff 64bit] (PCI address [0xee050000-0xee05ffff]) > > > pci 0000:0c:00.1: reg 18c: [mem 0xee060000-0xee06ffff 64bit] > > > pci 0000:0c:00.1: reg 18c: [mem 0xee060000-0xee06ffff 64bit] > > > pci 0000:0c:00.1: BAR 9: assigned [mem 0xee060000-0xee84ffff 64bit] > > > pci 0000:0c:00.1: BAR 9: set to [mem 0xee060000-0xee84ffff 64bit] (PCI address [0xee060000-0xee84ffff]) > > > pci 0000:0c:00.1: reg 184: [mem 0xee850000-0xee851fff 64bit] > > > pci 0000:0c:00.1: reg 184: [mem 0xee850000-0xee851fff 64bit] > > > pci 0000:0c:00.1: BAR 7: assigned [mem 0xee850000-0xee94dfff 64bit] > > > pci 0000:0c:00.1: BAR 7: set to [mem 0xee850000-0xee94dfff 64bit] (PCI address [0xee850000-0xee94dfff]) > > > pci 0000:0c:00.0: reg 18c: [mem 0xec040000-0xec04ffff 64bit] > > > pci 0000:0c:00.0: reg 18c: [mem 0xec040000-0xec04ffff 64bit] > > > pci 0000:0c:00.0: BAR 9: can't assign mem (size 0x7f0000) > > > > 18c is not assigned... for 0c:00.0 > > > > > pci 0000:0c:00.0: reg 184: [mem 0xee94e000-0xee94ffff 64bit] > > > pci 0000:0c:00.0: reg 184: [mem 0xee94e000-0xee94ffff 64bit] > > > pci 0000:0c:00.0: BAR 7: assigned [mem 0xee94e000-0xeea4bfff 64bit] > > > pci 0000:0c:00.0: BAR 7: set to [mem 0xee94e000-0xeea4bfff 64bit] (PCI address [0xee94e000-0xeea4bfff]) > > > pci 0000:00:08.0: PCI bridge to [bus 0c-0c] > > > pci 0000:00:08.0: bridge window [io 0xb000-0xbfff] > > > pci 0000:00:08.0: bridge window [mem 0xec000000-0xeeffffff] > > > pci 0000:00:08.0: bridge window [mem pref disabled] > > ... > > > pci 0000:20:08.0: BAR 14: assigned [mem 0xd5000000-0xd7ffffff] > > > pci 0000:20:08.0: BAR 15: can't assign mem pref (size 0x100000) > > > pci 0000:21:00.0: reg 184: [mem 0xd794e000-0xd794ffff 64bit] > > > pci 0000:21:00.0: reg 18c: [mem 0xd5040000-0xd504ffff 64bit] > > > pci 0000:21:00.0: reg 184: [mem 0xd794e000-0xd794ffff 64bit] > > > pci 0000:21:00.0: reg 18c: [mem 0xd5040000-0xd504ffff 64bit] > > > pci 0000:21:00.0: reg 184: [mem 0xd794e000-0xd794ffff 64bit] > > > pci 0000:21:00.1: reg 184: [mem 0xd7850000-0xd7851fff 64bit] > > > pci 0000:21:00.0: reg 18c: [mem 0xd5040000-0xd504ffff 64bit] > > > pci 0000:21:00.0: reg 184: [mem 0xd794e000-0xd794ffff 64bit] > > > pci 0000:21:00.1: reg 18c: [mem 0xd7060000-0xd706ffff 64bit] > > > pci 0000:21:00.0: reg 18c: [mem 0xd5040000-0xd504ffff 64bit] > > > pci 0000:21:00.0: reg 184: [mem 0xd794e000-0xd794ffff 64bit] > > > pci 0000:21:00.0: BAR 2: assigned [mem 0xd5000000-0xd5ffffff 64bit] > > > pci 0000:21:00.0: BAR 2: set to [mem 0xd5000000-0xd5ffffff 64bit] (PCI address [0xd5000000-0xd5ffffff]) > > > pci 0000:21:00.1: BAR 2: assigned [mem 0xd6000000-0xd6ffffff 64bit] > > > pci 0000:21:00.1: BAR 2: set to [mem 0xd6000000-0xd6ffffff 64bit] (PCI address [0xd6000000-0xd6ffffff]) > > > pci 0000:21:00.0: BAR 6: assigned [mem 0xd7000000-0xd701ffff pref] > > > pci 0000:21:00.1: BAR 6: assigned [mem 0xd7020000-0xd703ffff pref] > > > pci 0000:21:00.0: BAR 4: assigned [mem 0xd7040000-0xd704ffff 64bit] > > > pci 0000:21:00.0: BAR 4: set to [mem 0xd7040000-0xd704ffff 64bit] (PCI address [0xd7040000-0xd704ffff]) > > > pci 0000:21:00.1: BAR 4: assigned [mem 0xd7050000-0xd705ffff 64bit] > > > pci 0000:21:00.1: BAR 4: set to [mem 0xd7050000-0xd705ffff 64bit] (PCI address [0xd7050000-0xd705ffff]) > > > pci 0000:21:00.1: reg 18c: [mem 0xd7060000-0xd706ffff 64bit] > > > pci 0000:21:00.1: reg 18c: [mem 0xd7060000-0xd706ffff 64bit] > > > pci 0000:21:00.1: BAR 9: assigned [mem 0xd7060000-0xd784ffff 64bit] > > > pci 0000:21:00.1: BAR 9: set to [mem 0xd7060000-0xd784ffff 64bit] (PCI address [0xd7060000-0xd784ffff]) > > > pci 0000:21:00.1: reg 184: [mem 0xd7850000-0xd7851fff 64bit] > > > pci 0000:21:00.1: reg 184: [mem 0xd7850000-0xd7851fff 64bit] > > > pci 0000:21:00.1: BAR 7: assigned [mem 0xd7850000-0xd794dfff 64bit] > > > pci 0000:21:00.1: BAR 7: set to [mem 0xd7850000-0xd794dfff 64bit] (PCI address [0xd7850000-0xd794dfff]) > > > pci 0000:21:00.0: reg 18c: [mem 0xd5040000-0xd504ffff 64bit] > > > pci 0000:21:00.0: reg 18c: [mem 0xd5040000-0xd504ffff 64bit] > > > pci 0000:21:00.0: BAR 9: can't assign mem (size 0x7f0000) > > > > 21:00.0 18c is not assigned > > > > > pci 0000:21:00.0: reg 184: [mem 0xd794e000-0xd794ffff 64bit] > > > pci 0000:21:00.0: reg 184: [mem 0xd794e000-0xd794ffff 64bit] > > > pci 0000:21:00.0: BAR 7: assigned [mem 0xd794e000-0xd7a4bfff 64bit] > > > pci 0000:21:00.0: BAR 7: set to [mem 0xd794e000-0xd7a4bfff 64bit] (PCI address [0xd794e000-0xd7a4bfff]) > > > pci 0000:20:08.0: PCI bridge to [bus 21-21] > > > pci 0000:20:08.0: bridge window [io 0x9000-0x9fff] > > > pci 0000:20:08.0: bridge window [mem 0xd5000000-0xd7ffffff] > > > pci 0000:20:08.0: bridge window [mem pref disabled] > > > > so 6 SRIOV BARs get allocated, other 2 can not be allocated. > > > > that could be best result. > > > > Ram, can you check those two patches to see if it has any problem with your setup? > > No. the patch fails on my platform to enable SRIOV. The SRIOV BARs fail to allocate. debugging.. It fails to adjust-in a large window size, 4 times more than the one already assigned, to the bridge below which the SRIOV device resides. Adjusting in a smaller window size, like that for hotplug, more than often succeed. But adjusting in manytimes larger size cannot be guaranteed to work. I dont think we should treat SRIOV BARs as nice-to-have resources. Should we? RP -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
