On Mon, Oct 31, 2016 at 03:36:52PM -0400, Joshua Kinard wrote: [...] > > By not claiming resources, as you can see in the log below > > /proc/iomem and /proc/ioports do not report the PCI devices resource > > usages and the respective resources are not reserved in the kernel, > > which is wrong and that's exactly why Bjorn patched the kernel. > > > > Please let me know. > > > > Thanks, > > Lorenzo > > Yup, I can see that on my SGI Octane machine (IP30 platform), which is > architecturally-similar to IP27, but uses a more "normal" addressing > layout. Previously, it was also setting the PCI_PROBE_ONLY flag and > thus /proc/iomem looked very similar to IP27's output. Now, however, > Octane's /proc/iomem looks more like this: > > # cat /proc/iomem > 00000000-00003fff : reserved > 1d200000-1d9fffff : Bridge MEM > d080000000-d080003fff : 0001:00:03.0 > 1d204000-1d204fff : 0001:00:02.0 > 1d205000-1d205fff : 0001:00:02.1 > 1d206000-1d206fff : 0001:00:02.2 > 1d207000-1d2070ff : 0001:00:02.3 > 1d207100-1d20717f : 0001:00:01.0 > 1f200000-1f9fffff : Bridge MEM > f080100000-f0801fffff : 0000:00:02.0 > f080010000-f08001ffff : 0000:00:00.0 > f080030000-f08003ffff : 0000:00:01.0 > f080000000-f080000fff : 0000:00:00.0 > f080020000-f080020fff : 0000:00:01.0 Is this the resource hierarchy you get with the patch reverted ? This resource hierarchy is weird (and IIUC it should be the end result of the kernel assigning the whole resource space). I need kernel log with mainline and mainline with this patch reverted please on IP30. Still, without digging into the platform code that sets-up the PCI resources I doubt I can get to the bottom of this so if you can give me a pointer to the source code files that set-up the PCI bridges I am keen to help. Thanks a lot, Lorenzo > 20004000-20a27fff : reserved > 20004000-207180b7 : Kernel code > 207180b8-2090ffff : Kernel data > 20a28000-20efffff : System RAM > 20f00000-20ffffff : System RAM > 21000000-9fffffff : System RAM > f080100000-f0801fffff : ioc3 > > And /proc/ioports: > # cat /proc/ioports > 1da00000-1dbfffff : Bridge IO > 1da00000-1da000ff : 0001:00:01.0 > 1da00400-1da0047f : 0001:00:01.0 > 1fa00000-1fbfffff : Bridge IO > f100000000-f1000000ff : 0000:00:00.0 > f100000100-f1000001ff : 0000:00:01.0 > > PCI support on IP27 has been a bit weird for a long time, mostly due to it > using a lot of "IRIX'isms" in the code to get it working initially. Some of > these are actually short-circuits for things that would instead use normal > kernel mechanisms, and no one's really had the time or interest to try and > clean it up. I've done a significant amount of refactoring of the IP27's > machine-specific code so far the last few months, but haven't spent much > time understanding PCI and XIO some more to make further progress. Lack of > time has made it difficult to break those code changes down into bite-sized > patches to send back upstream (I should've planned things a bit better...). > > --J > > > >> First, is the boot output from my serial console: > >> [ 0.000000] Linux version 4.8.2-mipsgit-20161016 (root@helcaraxe) (gcc version 5.4.0 (Gentoo Hardened 5.4.0 p1.0, pie-0.6.5) ) #4 SMP Mon Oct 17 19:48:30 EDT 2016 > >> [ 0.000000] ARCH: SGI-IP27 > >> [ 0.000000] PROMLIB: ARC firmware Version 64 Revision 0 > >> [ 0.000000] SMP: Discovered 4 cpus on 2 nodes > >> [ 0.000000] ************** Topology ******************** > >> [ 0.000000] 00 01 > >> [ 0.000000] 00 255 255 > >> [ 0.000000] 01 255 255 > >> [ 0.000000] bootconsole [early0] enabled > >> [ 0.000000] CPU0 revision is: 00000f14 (R14000) > >> [ 0.000000] FPU revision is: 00000900 > >> [ 0.000000] Checking for the multiply/shift bug... no. > >> [ 0.000000] Checking for the daddiu bug... no. > >> [ 0.000000] IP27: Running on node 0. > >> [ 0.000000] IP27: Node 0 has a primary CPU, CPU is running. > >> [ 0.000000] IP27: Node 0 has a secondary CPU, CPU is running. > >> [ 0.000000] IP27: Machine is in M mode. > >> [ 0.000000] IP27: CPU 1A (CPU0): 500 MHz CPU detected. > >> [ 0.000000] Determined physical RAM map: > >> [ 0.000000] memory: 0000000000838000 @ 000000000001c000 (usable) > >> [ 0.000000] memory: 000000000057c000 @ 0000000000854000 (usable after init) > >> [ 0.000000] Initrd not found or empty - disabling initrd > >> [ 0.000000] cma: Reserved 32 MiB at 0x00000001fe000000 > >> [ 0.000000] REPLICATION: ON nasid 0, ktext from nasid 0, kdata from nasid 0 > >> [ 0.000000] REPLICATION: ON nasid 1, ktext from nasid 0, kdata from nasid 0 > >> [ 0.000000] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes. > >> [ 0.000000] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes > >> [ 0.000000] Unified secondary cache 8192kB 2-way, linesize 128 bytes. > >> [ 0.000000] Zone ranges: > >> [ 0.000000] Normal [mem 0x0000000000000000-0x00000001ffffffff] > >> [ 0.000000] Movable zone start for each node > >> [ 0.000000] Early memory node ranges > >> [ 0.000000] node 0: [mem 0x0000000000000000-0x00000000ffffffff] > >> [ 0.000000] node 1: [mem 0x0000000100000000-0x00000001ffffffff] > >> [ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000ffffffff] > >> [ 0.000000] Initmem setup node 1 [mem 0x0000000100000000-0x00000001ffffffff] > >> [ 0.000000] percpu: Embedded 4 pages/cpu @a800000002010000 s36096 r0 d29440 u65536 > >> [ 0.000000] Built 2 zonelists in Node order, mobility grouping on. Total pages: 522240 > >> [ 0.000000] Policy zone: Normal > >> [ 0.000000] Kernel command line: root=dksc(0,1,0) console=ttyS0,9600 > >> [ 0.000000] PID hash table entries: 4096 (order: 1, 32768 bytes) > >> [ 0.000000] Memory: 8307776K/8388608K available (6498K kernel code, 454K rwdata, 1436K rodata, 5616K init, 543K bss, 48064K reserved, 32768K cma-reserved) > >> [ 0.000000] Hierarchical RCU implementation. > >> [ 0.000000] Build-time adjustment of leaf fanout to 64. > >> [ 0.000000] RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4. > >> [ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4 > >> [ 0.000000] NR_IRQS:256 > >> [ 0.000000] clocksource: HUB: mask: 0xfffffffffffff max_cycles: 0x127350b88, max_idle_ns: 1763180808480 ns > >> [ 0.000019] sched_clock: 52 bits at 1250kHz, resolution 800ns, wraps every 4398046510800ns > >> [ 0.102422] Console: colour dummy device 80x25 > >> [ 0.154975] Calibrating delay loop... 749.56 BogoMIPS (lpj=3747840) > >> [ 0.317670] pid_max: default: 32768 minimum: 301 > >> [ 0.377938] Dentry cache hash table entries: 1048576 (order: 9, 8388608 bytes) > >> [ 0.592303] Inode-cache hash table entries: 524288 (order: 8, 4194304 bytes) > >> [ 0.736316] Mount-cache hash table entries: 16384 (order: 3, 131072 bytes) > >> [ 0.817310] Mountpoint-cache hash table entries: 16384 (order: 3, 131072 bytes) > >> [ 0.916094] Checking for the daddi bug... no. > >> [ 0.976495] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes. > >> [ 0.976520] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes > >> [ 0.976532] Unified secondary cache 8192kB 2-way, linesize 128 bytes. > >> [ 0.979792] IP27: CPU 1B (CPU1): 500 MHz CPU detected. > >> [ 0.979824] CPU1 revision is: 00000f14 (R14000) > >> [ 0.979831] FPU revision is: 00000900 > >> [ 1.014463] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes. > >> [ 1.014493] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes > >> [ 1.014506] Unified secondary cache 8192kB 2-way, linesize 128 bytes. > >> [ 1.017888] IP27: CPU 2A (CPU2): 500 MHz CPU detected. > >> [ 1.017931] CPU2 revision is: 00000f14 (R14000) > >> [ 1.017938] FPU revision is: 00000900 > >> [ 1.055420] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes. > >> [ 1.055452] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes > >> [ 1.055467] Unified secondary cache 8192kB 2-way, linesize 128 bytes. > >> [ 1.058810] IP27: CPU 2B (CPU3): 500 MHz CPU detected. > >> [ 1.058851] CPU3 revision is: 00000f14 (R14000) > >> [ 1.058858] FPU revision is: 00000900 > >> [ 1.092672] Brought up 4 CPUs > >> [ 2.484801] devtmpfs: initialized > >> [ 2.603700] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns > >> [ 2.892716] random: fast init done > >> [ 2.990752] xor: measuring software checksum speed > >> [ 3.146730] 8regs : 486.400 MB/sec > >> [ 3.295350] 8regs_prefetch: 486.400 MB/sec > >> [ 3.448144] 32regs : 486.400 MB/sec > >> [ 3.596768] 32regs_prefetch: 486.400 MB/sec > >> [ 3.650415] xor: using function: 32regs_prefetch (486.400 MB/sec) > >> [ 3.834852] NET: Registered protocol family 16 > >> [ 3.946535] cpuidle: using governor ladder > >> [ 4.043955] cpuidle: using governor menu > >> [ 4.089355] xtalk:n0/0 xbow widget (rev 1.3) > >> [ 4.140796] xtalk:n0/8 kona widget (rev unknown) registered as a platform device. > >> [ 4.230796] xtalk:n0/b bridge widget (rev D) registered as a platform device. > >> [ 4.316712] xtalk:n0/c bridge widget (rev D) registered as a platform device. > >> [ 4.402590] xtalk:n0/d impact widget (rev B) registered as a platform device. > >> [ 4.488444] xtalk:n0/f bridge widget (rev C) registered as a platform device. > >> [ 4.574178] xtalk:n1/0 xbow widget (rev 1.3) > >> [ 5.247396] raid6: int64x1 gen() 84 MB/s > >> [ 5.467370] raid6: int64x1 xor() 80 MB/s > >> [ 5.687324] raid6: int64x2 gen() 151 MB/s > >> [ 5.907243] raid6: int64x2 xor() 116 MB/s > >> [ 6.127125] raid6: int64x4 gen() 235 MB/s > >> [ 6.347155] raid6: int64x4 xor() 142 MB/s > >> [ 6.567117] raid6: int64x8 gen() 251 MB/s > >> [ 6.787018] raid6: int64x8 xor() 175 MB/s > >> [ 6.836420] raid6: using algorithm int64x8 gen() 251 MB/s > >> [ 6.901352] raid6: .... xor() 175 MB/s, rmw enabled > >> [ 6.960002] raid6: using intx1 recovery algorithm > >> [ 7.017544] SCSI subsystem initialized > >> [ 7.063853] clocksource: Switched to clocksource HUB > >> [ 7.363862] FS-Cache: Loaded > >> [ 31.643872] NET: Registered protocol family 2 > >> [ 31.696149] TCP established hash table entries: 65536 (order: 5, 524288 bytes) > >> [ 31.787294] TCP bind hash table entries: 65536 (order: 6, 1048576 bytes) > >> [ 31.874832] TCP: Hash tables configured (established 65536 bind 65536) > >> [ 31.952084] UDP hash table entries: 4096 (order: 3, 131072 bytes) > >> [ 32.026304] UDP-Lite hash table entries: 4096 (order: 3, 131072 bytes) > >> [ 32.403836] NET: Registered protocol family 1 > >> [ 32.583876] RPC: Registered named UNIX socket transport module. > >> [ 32.653282] RPC: Registered udp transport module. > >> [ 32.709785] RPC: Registered tcp transport module. > >> [ 32.766346] RPC: Registered tcp NFSv4.1 backchannel transport module. > >> [ 34.045622] futex hash table entries: 1024 (order: 3, 131072 bytes) > >> [ 34.275238] workingset: timestamp_bits=40 max_order=19 bucket_order=0 > >> [ 34.350961] zbud: loaded > >> [ 37.383916] NFS: Registering the id_resolver key type > >> [ 37.442917] Key type id_resolver registered > >> [ 37.493101] Key type id_legacy registered > >> [ 37.541388] efs: 1.0a - http://aeschi.ch.eu.org/efs/ > >> [ 37.873853] SGI XFS with ACLs, security attributes, realtime, no debug enabled > >> [ 39.453936] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253) > >> [ 39.541066] io scheduler noop registered > >> [ 39.588223] io scheduler deadline registered > >> [ 39.639639] io scheduler cfq registered (default) > >> [ 39.973936] pci_hotplug: PCI Hot Plug PCI Core version: 0.5 > >> [ 40.148640] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled > >> [ 40.245916] loop: module loaded > >> [ 40.283288] mousedev: PS/2 mouse device common for all mice > >> [ 40.349460] md: linear personality registered for level -1 > >> [ 40.414802] md: raid0 personality registered for level 0 > >> [ 40.478677] md: raid1 personality registered for level 1 > >> [ 40.542549] md: raid10 personality registered for level 10 > >> [ 40.609068] md: raid6 personality registered for level 6 > >> [ 40.672411] md: raid5 personality registered for level 5 > >> [ 40.736315] md: raid4 personality registered for level 4 > >> [ 40.800808] PCI host bridge to bus 0000:00 > >> [ 40.849424] pci_bus 0000:00: root bus resource [mem 0x920000000b200000-0x920000000b9fffff] > >> [ 40.948920] pci_bus 0000:00: root bus resource [io 0x920000000ba00000-0x920000000bbfffff] > >> [ 41.048408] pci_bus 0000:00: root bus resource [bus 00-ff] > >> [ 41.114880] PCI host bridge to bus 0001:00 > >> [ 41.163629] pci_bus 0001:00: root bus resource [mem 0x920000000c200000-0x920000000c9fffff] > >> [ 41.263152] pci_bus 0001:00: root bus resource [io 0x920000000ca00000-0x920000000cbfffff] > >> [ 41.362645] pci_bus 0001:00: root bus resource [bus 01-ff] > >> [ 41.430847] PCI host bridge to bus 0002:00 > >> [ 41.478263] pci_bus 0002:00: root bus resource [mem 0x920000000f200000-0x920000000f9fffff] > >> [ 41.577771] pci_bus 0002:00: root bus resource [io 0x920000000fa00000-0x920000000fbfffff] > >> [ 41.677250] pci_bus 0002:00: root bus resource [bus 02-ff] > >> [ 41.746742] qla1280: QLA1040 found on PCI bus 0, dev 0 > >> [ 41.806687] PCI: Enabling device 0002:00:00.0 (0006 -> 0007) > >> [ 41.875796] scsi(0): Enabling vchannel on BRIDGE for SGI/MIPS > >> [ 42.324520] random: crng init done > >> [ 42.839160] scsi(0:0): Resetting SCSI BUS > >> [ 45.964232] scsi host0: QLogic QLA1040 PCI to SCSI Host Adapter > >> [ 45.964232] Firmware version: 7.65.06, Driver version 3.27.1 > >> [ 46.118149] qla1280: QLA1040 found on PCI bus 0, dev 1 > >> [ 46.178192] PCI: Enabling device 0002:00:01.0 (0006 -> 0007) > >> [ 46.289985] scsi 0:0:1:0: Direct-Access SEAGATE SX150176LC BA12 PQ: 0 ANSI: 2 > >> [ 46.387174] scsi(0:0:1:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31 > >> [ 46.486875] scsi 0:0:2:0: Direct-Access SEAGATE SX150176LC BA08 PQ: 0 ANSI: 2 > >> [ 46.583893] scsi(0:0:2:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31 > >> [ 46.683736] scsi 0:0:3:0: Direct-Access SEAGATE SX150176LC BA08 PQ: 0 ANSI: 2 > >> [ 46.780696] scsi(0:0:3:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31 > >> [ 46.880821] scsi 0:0:4:0: Direct-Access SEAGATE SX150176LC BA11 PQ: 0 ANSI: 2 > >> [ 46.977716] scsi(0:0:4:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31 > >> [ 47.077797] scsi 0:0:5:0: Direct-Access SEAGATE SX150176LC BA11 PQ: 0 ANSI: 2 > >> [ 47.174768] scsi(0:0:5:0):[ 47.181219] scsi(1:0): Resetting SCSI BUS > >> [ 47.251839] Sync: period 10, offset 12, Wide, Tagged queuing: depth 31 > >> [ 47.639912] scsi 0:0:6:0: CD-ROM TOSHIBA CD-ROM XM-5701TA 0167 PQ: 0 ANSI: 2 > >> [ 47.736890] scsi(0:0:6:0): Sync: period 10, offset 12 > >> [ 51.553976] scsi host1: QLogic QLA1040 PCI to SCSI Host Adapter > >> [ 51.553976] Firmware version: 7.65.06, Driver version 3.27.1 > >> [ 51.998859] ioc3: part: [], serial: [] => class IP27 BaseIO > >> [ 52.068783] ioc3-eth: Ethernet address is 08:00:69:05:64:74. > >> [ 52.136831] IOC3 0002:00:02.0 eth0: link up, 100Mbps, full-duplex, lpa 0x8DE1 > >> [ 52.220897] eth0: Using PHY 31, vendor 0x20005c0, model 0, rev 0. > >> [ 52.294232] eth0: IOC3 SSRAM has 128 kbyte. > >> [ 52.345852] rtc-m48t35 rtc-m48t35: rtc core: registered m48t35 as rtc0 > >> [ 52.588471] console [ttyS0] enabled > >> [ 52.489601] 0002:00:02.0: ttyS0 at IOC3 0xf620178 (irq = 0, base_baud = 458333) is a 16550A > >> [ 52.671141] bootconsole [early0] disabled > >> [ 54.146680] 0002:00:02.0: ttyS1 at IOC3 0xf620170 (irq = 0, base_baud = 458333) is a 16550A > >> [ 54.248408] IOC3 Master Driver loaded for 0002:00:02.0 > >> [ 54.313895] ioc3: NIC search failed. > >> [ 54.356992] ioc3: part: [], serial: [] => class IP27 BaseIO > >> [ 54.424356] ioc3: request_irq fails for IRQ 0xffffffff > >> [ 54.490948] ioc3-eth: Ethernet address is 00:00:00:00:00:00. > >> [ 54.561038] IOC3 0002:00:06.0 eth1: link down > >> [ 54.613597] eth1: Using PHY 31, vendor 0x0, model 0, rev 0. > >> [ 54.680825] eth1: IOC3 SSRAM has 64 kbyte. > >> [ 54.765266] 0002:00:06.0: ttyS2 at IOC3 0xfa20178 (irq = 0, base_baud = 458333) is a 16550A > >> [ 54.888655] 0002:00:06.0: ttyS3 at IOC3 0xfa20170 (irq = 0, base_baud = 458333) is a 16550A > >> [ 54.990198] IOC3 Master Driver loaded for 0002:00:06.0 > >> [ 55.052840] NET: Registered protocol family 26 > >> [ 55.509832] sd 0:0:1:0: [sda] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB) > >> [ 55.515697] sr 0:0:6:0: [sr0] scsi-1 drive > >> [ 55.515709] cdrom: Uniform CD-ROM driver Revision: 3.20 > >> [ 55.520834] sd 0:0:4:0: [sdd] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB) > >> [ 55.523611] sd 0:0:4:0: [sdd] Write Protect is off > >> [ 55.526511] sd 0:0:4:0: [sdd] Write cache: disabled, read cache: enabled, supports DPO and FUA > >> [ 55.550956] sdd: sdd1 sdd2 sdd3 sdd4 sdd5 sdd6 sdd9 sdd11 > >> [ 55.562326] sd 0:0:4:0: [sdd] Attached SCSI disk > >> [ 55.564803] sd 0:0:5:0: [sde] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB) > >> [ 55.567564] sd 0:0:5:0: [sde] Write Protect is off > >> [ 55.570463] sd 0:0:5:0: [sde] Write cache: disabled, read cache: enabled, supports DPO and FUA > >> [ 55.600908] sd 0:0:5:0: [sde] Attached SCSI disk > >> [ 55.604715] sd 0:0:2:0: [sdb] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB) > >> [ 55.605297] sd 0:0:3:0: [sdc] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB) > >> [ 55.607054] sd 0:0:2:0: [sdb] Write Protect is off > >> [ 55.607592] sd 0:0:3:0: [sdc] Write Protect is off > >> [ 55.609959] sd 0:0:2:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA > >> [ 55.610460] sd 0:0:3:0: [sdc] Write cache: disabled, read cache: enabled, supports DPO and FUA > >> [ 55.631199] sdb: sdb1 sdb2 sdb3 sdb4 sdb5 sdb6 sdb9 sdb11 > >> [ 55.634048] sdc: sdc1 sdc2 sdc3 sdc4 sdc5 sdc6 sdc9 sdc11 > >> [ 55.646437] sd 0:0:3:0: [sdc] Attached SCSI disk > >> [ 55.647162] sd 0:0:2:0: [sdb] Attached SCSI disk > >> [ 57.150140] sd 0:0:1:0: [sda] Write Protect is off > >> [ 57.173988] NET: Registered protocol family 10 > >> [ 57.264566] sd 0:0:1:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA > >> [ 57.386960] sda: sda1 sda2 sda3 sda4 sda5 sda6 sda9 sda11 > >> [ 57.464000] sd 0:0:1:0: [sda] Attached SCSI disk > >> [ 57.544064] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver > >> [ 57.617430] NET: Registered protocol family 17 > >> [ 57.671360] 8021q: 802.1Q VLAN Support v1.8 > >> [ 57.722075] sctp: Hash tables configured (bind 1024/1024) > >> [ 58.093962] Key type dns_resolver registered > >> [ 58.148015] rtc-m48t35 rtc-m48t35: setting system clock to 2016-10-17 23:49:58 UTC (1476748198) > >> [ 59.626451] Freeing unused kernel memory: 5616K (a800000000854000 - a800000000dd0000) > >> [ 59.720950] This architecture does not have kernel memory protection. > >> > >> > >> Next is /proc/iomem and /proc/ioports: > >> # cat /proc/iomem > >> 0f600000-0f6fffff : ioc3 > >> 0f680000-0f687fff : rtc-m48t35 > >> 0fa00000-0fafffff : ioc3 > >> 920000000b200000-920000000b9fffff : Bridge MEM > >> 920000000c200000-920000000c9fffff : Bridge MEM > >> 920000000f200000-920000000f9fffff : Bridge MEM > >> > >> / # cat /proc/ioports > >> 920000000ba00000-920000000bbfffff : Bridge IO > >> 920000000ca00000-920000000cbfffff : Bridge IO > >> 920000000fa00000-920000000fbfffff : Bridge IO > >> > >> > >> And lspci -vvvx: > >> > >> / # /usr/sbin/lspci -vvvx > >> 0001:00:00.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. Linc I/O controller > >> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > >> NUMA node: 0 > >> Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=134217729] > >> Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Expansion ROM at <unassigned> [disabled] [size=2] > >> lspci: Unable to load libkmod resources: error -12 > >> 00: a9 10 02 00 00 00 00 02 00 00 00 ff 00 f8 80 00 > >> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> > >> 0001:00:01.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. Linc I/O controller > >> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > >> NUMA node: 0 > >> Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=134217729] > >> Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Expansion ROM at <unassigned> [disabled] [size=2] > >> 00: a9 10 02 00 00 00 00 02 00 00 00 ff 00 f8 80 00 > >> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> > >> 0002:00:00.0 SCSI storage controller: QLogic Corp. ISP1020 Fast-wide SCSI (rev 05) > >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > >> Latency: 64, Cache Line Size: 256 bytes > >> Interrupt: pin A routed to IRQ 1 > >> NUMA node: 0 > >> Region 0: I/O ports at f200000 [size=257] > >> Region 1: Memory at 0f200000 (32-bit, non-prefetchable) [size=4097] > >> Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Expansion ROM at <unassigned> [disabled] [size=65537] > >> Kernel driver in use: qla1280 > >> 00: 77 10 20 10 07 00 00 02 05 00 00 01 40 40 00 00 > >> 10: 01 00 20 0f 00 00 20 0f 00 00 00 00 00 00 00 00 > >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 > >> > >> 0002:00:01.0 SCSI storage controller: QLogic Corp. ISP1020 Fast-wide SCSI (rev 05) > >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > >> Latency: 64, Cache Line Size: 256 bytes > >> Interrupt: pin A routed to IRQ 2 > >> NUMA node: 0 > >> Region 0: I/O ports at f400000 [size=257] > >> Region 1: Memory at 0f400000 (32-bit, non-prefetchable) [size=4097] > >> Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Expansion ROM at <unassigned> [disabled] [size=65537] > >> Kernel driver in use: qla1280 > >> 00: 77 10 20 10 07 00 00 02 05 00 00 01 40 40 00 00 > >> 10: 01 00 40 0f 00 00 40 0f 00 00 00 00 00 00 00 00 > >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 > >> > >> 0002:00:02.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. IOC3 I/O controller (rev 01) > >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > >> Latency: 64 > >> Interrupt: pin A routed to IRQ 3 > >> NUMA node: 0 > >> Region 0: Memory at 0f600000 (32-bit, non-prefetchable) [size=1048577] > >> Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Expansion ROM at <unassigned> [disabled] [size=2] > >> Kernel driver in use: IOC3 > >> 00: a9 10 03 00 06 00 80 02 01 00 00 ff 00 40 00 00 > >> 10: 00 00 60 0f 00 00 00 00 00 00 00 00 00 00 00 00 > >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 > >> > >> 0002:00:06.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. IOC3 I/O controller (rev 01) > >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > >> Latency: 64 > >> Interrupt: pin A routed to IRQ 5 > >> NUMA node: 0 > >> Region 0: Memory at 0fa00000 (32-bit, non-prefetchable) [size=1048577] > >> Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2] > >> Expansion ROM at <unassigned> [disabled] [size=2] > >> Kernel driver in use: IOC3 > >> 00: a9 10 03 00 06 00 80 02 01 00 00 ff 00 40 00 00 > >> 10: 00 00 a0 0f 00 00 00 00 00 00 00 00 00 00 00 00 > >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 > >> > >> 0002:00:07.0 Non-VGA unclassified device: Silicon Graphics Intl. Corp. RAD Audio (rev 70) > >> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > >> NUMA node: 0 > >> Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=8193] > >> Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2] > >> Expansion ROM at <unassigned> [disabled] [size=2] > >> 00: a9 10 05 00 00 00 80 04 70 00 00 00 00 00 00 00 > >> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > >> > >> > >> IP27 hardware does some interesting things with address spaces, depending on > >> which "mode" you run the machine in. "M mode" is the only one that's really > >> usable these days (the other, "N mode", doesn't boot). Under M mode, > >> there's five address spaces, of which, "I/O Space" is selectable by turning on > >> specific bits in the 64-bit memory address. > >> > >> So, given 0x920000000b200000, bits 63:62 select physical addressing (10) and > >> bits 58:57 select I/O space (01), resulting in all PCI addresses (or any I/O > >> address) appearing in the 0x92xxxxxxxxxxxxxx range. The rest of the bits go > >> on to select which NUMA node the I/O is targeted at and which Crosstalk widget > >> to access, and so on. I still haven't fully understood it all myself just > >> yet. > >> > >> You can google for document 007-3410-001, "Origin and Onyx2 Programmer's > >> Reference Manual" for more details (it's no longer available via SGI's > >> TechPubs, but it is archived on plenty of other sites). > >> > >> This particular machine that I have has three "BRIDGE" chips in it, all > >> attached to the Crosstalk bus. Each BRIDGE can address up to 8 PCI devices, > >> but some crosstalk boards slap a BRIDGE chip on for just a single device. In > >> this instance, the BRIDGE w/ address 0x920000000fxxxxxx is the IO6 at xtalk > >> widget 0xf (this has the basic I/O devices for this machine). The BRIDGE at > >> 0x920000000cxxxxxx (widget 0xc) is a PCI "shoebox" that can hold up to three > >> PCI-X cards, and the BRIDGE at 0x920000000bxxxxxx (widget 0xb) is an > >> SGI-specific HIPPI board (ID'ed as a LINC I/O controller by pci.ids).
