On Wed, Dec 5, 2018 at 5:43 PM Michal Hocko <mhocko@xxxxxxxxxx> wrote:
>
> On Wed 05-12-18 17:29:31, Pingfan Liu wrote:
> > On Wed, Dec 5, 2018 at 5:21 PM Michal Hocko <mhocko@xxxxxxxxxx> wrote:
> > >
> > > On Wed 05-12-18 13:38:17, Pingfan Liu wrote:
> > > > On Tue, Dec 4, 2018 at 4:56 PM Michal Hocko <mhocko@xxxxxxxxxx> wrote:
> > > > >
> > > > > On Tue 04-12-18 16:20:32, Pingfan Liu wrote:
> > > > > > On Tue, Dec 4, 2018 at 3:22 PM Michal Hocko <mhocko@xxxxxxxxxx> wrote:
> > > > > > >
> > > > > > > On Tue 04-12-18 11:05:57, Pingfan Liu wrote:
> > > > > > > > During my test on some AMD machine, with kexec -l nr_cpus=x option, the
> > > > > > > > kernel failed to bootup, because some node's data struct can not be allocated,
> > > > > > > > e.g, on x86, initialized by init_cpu_to_node()->init_memory_less_node(). But
> > > > > > > > device->numa_node info is used as preferred_nid param for
> > > > > > > > __alloc_pages_nodemask(), which causes NULL reference
> > > > > > > > ac->zonelist = node_zonelist(preferred_nid, gfp_mask);
> > > > > > > > This patch tries to fix the issue by falling back to the first online node,
> > > > > > > > when encountering such corner case.
> > > > > > >
> > > > > > > We have seen similar issues already and the bug was usually that the
> > > > > > > zonelists were not initialized yet or the node is completely bogus.
> > > > > > > Zonelists should be initialized by build_all_zonelists quite early so I
> > > > > > > am wondering whether the later is the case. What is the actual node
> > > > > > > number the device is associated with?
> > > > > > >
> > > > > > The device's node num is 2. And in my case, I used nr_cpus param. Due
> > > > > > to init_cpu_to_node() initialize all the possible node. It is hard
> > > > > > for me to figure out without this param, how zonelists is accessed
> > > > > > before page allocator works.
> > > > >
> > > > > I believe we should focus on this. Why does the node have no zonelist
> > > > > even though all zonelists should be initialized already? Maybe this is
> > > > > nr_cpus pecularity and we do not initialize all the existing numa nodes.
> > > > > Or maybe the device is associated to a non-existing node with that
> > > > > setup. A full dmesg might help us here.
> > > > >
> > > > Requiring the machine again, and I got the following without nr_cpus option
> > > > [root@dell-per7425-03 ~]# cd /sys/devices/system/node/
> > > > [root@dell-per7425-03 node]# ls
> > > > has_cpu has_memory has_normal_memory node0 node1 node2 node3
> > > > node4 node5 node6 node7 online possible power uevent
> > > > [root@dell-per7425-03 node]# cat has_cpu
> > > > 0-7
> > > > [root@dell-per7425-03 node]# cat has_memory
> > > > 1,5
> > > > [root@dell-per7425-03 node]# cat online
> > > > 0-7
> > > > [root@dell-per7425-03 node]# cat possible
> > > > 0-7
> > > > And lscpu shows the following numa-cpu info:
> > > > NUMA node0 CPU(s): 0,8,16,24
> > > > NUMA node1 CPU(s): 2,10,18,26
> > > > NUMA node2 CPU(s): 4,12,20,28
> > > > NUMA node3 CPU(s): 6,14,22,30
> > > > NUMA node4 CPU(s): 1,9,17,25
> > > > NUMA node5 CPU(s): 3,11,19,27
> > > > NUMA node6 CPU(s): 5,13,21,29
> > > > NUMA node7 CPU(s): 7,15,23,31
> > > >
> > > > For the full panic message (I masked some hostname info with xx),
> > > > please see the attachment.
> > > > In a short word, it seems a problem with nr_cpus, if without this
> > > > option, the kernel can bootup correctly.
> > >
> > > Yep.
> > > [ 0.007418] Early memory node ranges
> > > [ 0.007419] node 1: [mem 0x0000000000001000-0x000000000008efff]
> > > [ 0.007420] node 1: [mem 0x0000000000090000-0x000000000009ffff]
> > > [ 0.007422] node 1: [mem 0x0000000000100000-0x000000005c3d6fff]
> > > [ 0.007422] node 1: [mem 0x00000000643df000-0x0000000068ff7fff]
> > > [ 0.007423] node 1: [mem 0x000000006c528000-0x000000006fffffff]
> > > [ 0.007424] node 1: [mem 0x0000000100000000-0x000000047fffffff]
> > > [ 0.007425] node 5: [mem 0x0000000480000000-0x000000087effffff]
> > >
> > > There is clearly no node2. Where did the driver get the node2 from?
> > Since using nr_cpus=4 , the node2 is not be instanced by x86 initalizing code.
> > For the normal bootup, having the following:
> > [ 0.007704] Movable zone start for each node
> > [ 0.007707] Early memory node ranges
> > [ 0.007708] node 1: [mem 0x0000000000001000-0x000000000008efff]
> > [ 0.007709] node 1: [mem 0x0000000000090000-0x000000000009ffff]
> > [ 0.007711] node 1: [mem 0x0000000000100000-0x000000005c3d6fff]
> > [ 0.007712] node 1: [mem 0x00000000643df000-0x0000000068ff7fff]
> > [ 0.007712] node 1: [mem 0x000000006c528000-0x000000006fffffff]
> > [ 0.007713] node 1: [mem 0x0000000100000000-0x000000047fffffff]
> > [ 0.007714] node 5: [mem 0x0000000480000000-0x000000087effffff]
> > [ 0.008434] Zeroed struct page in unavailable ranges: 46490 pages
>
> Hmm, this is even more interesting. So even a normal boot doesn't have
> node 2. So where exactly does the device get its affinity from?
>
I am afraid that there is maybe some misunderstanding, but for the
normal bootup, the full boot msg shows the existence of node 2
First, the following can be observed:
[ 0.007385] SRAT: PXM 0 -> APIC 0x00 -> Node 0
[ 0.007386] SRAT: PXM 0 -> APIC 0x01 -> Node 0
[ 0.007387] SRAT: PXM 0 -> APIC 0x08 -> Node 0
[ 0.007388] SRAT: PXM 0 -> APIC 0x09 -> Node 0
[ 0.007389] SRAT: PXM 1 -> APIC 0x10 -> Node 1
[ 0.007390] SRAT: PXM 1 -> APIC 0x11 -> Node 1
[ 0.007391] SRAT: PXM 1 -> APIC 0x18 -> Node 1
[ 0.007392] SRAT: PXM 1 -> APIC 0x19 -> Node 1
[ 0.007393] SRAT: PXM 2 -> APIC 0x20 -> Node 2
[ 0.007394] SRAT: PXM 2 -> APIC 0x21 -> Node 2
[ 0.007395] SRAT: PXM 2 -> APIC 0x28 -> Node 2
[ 0.007396] SRAT: PXM 2 -> APIC 0x29 -> Node 2
[ 0.007397] SRAT: PXM 3 -> APIC 0x30 -> Node 3
[ 0.007398] SRAT: PXM 3 -> APIC 0x31 -> Node 3
[ 0.007399] SRAT: PXM 3 -> APIC 0x38 -> Node 3
[ 0.007400] SRAT: PXM 3 -> APIC 0x39 -> Node 3
[ 0.007401] SRAT: PXM 4 -> APIC 0x40 -> Node 4
[ 0.007402] SRAT: PXM 4 -> APIC 0x41 -> Node 4
[ 0.007403] SRAT: PXM 4 -> APIC 0x48 -> Node 4
[ 0.007403] SRAT: PXM 4 -> APIC 0x49 -> Node 4
[ 0.007404] SRAT: PXM 5 -> APIC 0x50 -> Node 5
[ 0.007405] SRAT: PXM 5 -> APIC 0x51 -> Node 5
[ 0.007406] SRAT: PXM 5 -> APIC 0x58 -> Node 5
[ 0.007407] SRAT: PXM 5 -> APIC 0x59 -> Node 5
[ 0.007408] SRAT: PXM 6 -> APIC 0x60 -> Node 6
[ 0.007409] SRAT: PXM 6 -> APIC 0x61 -> Node 6
[ 0.007410] SRAT: PXM 6 -> APIC 0x68 -> Node 6
[ 0.007411] SRAT: PXM 6 -> APIC 0x69 -> Node 6
[ 0.007412] SRAT: PXM 7 -> APIC 0x70 -> Node 7
[ 0.007413] SRAT: PXM 7 -> APIC 0x71 -> Node 7
[ 0.007414] SRAT: PXM 7 -> APIC 0x78 -> Node 7
[ 0.007414] SRAT: PXM 7 -> APIC 0x79 -> Node 7
Second:
[ 0.024497] NODE_DATA(2) allocated [mem 0x87ef4b000-0x87ef75fff]
[ 0.024498] NODE_DATA(2) on node 5
[ 0.024530] Initmem setup node 2 [mem 0x0000000000000000-0x0000000000000000]
Besides these, as I pasted, when normal bootup, lscpu shows the: NUMA
node2 CPU(s): 4,12,20,28
and
[root@dell-per7425-03 node]# cat has_cpu
0-7
[root@dell-per7425-03 node]# cat has_memory
1,5
So I think node 2 exists, which has cpus but no memory.
I attach SRAT, which show there are eight Proximity Domain. As my
understanding, each Proximity Domain will correspond to a numa node.
Cced acpi guys and hope them can give some hints.
> I suspect we are looking at two issues here. The first one, and a more
> important one is that there is a NUMA affinity configured for the device
> to a non-existing node. The second one is that nr_cpus affects
> initialization of possible nodes.
The dev->numa_node info is extracted from acpi table, not depends on
the instance of numa-node, which may be limited by nr_cpus. Hence the
node is existing, just not instanced.
Thanks,
Pingfan
/*
* Intel ACPI Component Architecture
* AML/ASL+ Disassembler version 20160527-64
* Copyright (c) 2000 - 2016 Intel Corporation
*
* Disassembly of ./SRAT.dat, Wed Dec 5 21:56:44 2018
*
* ACPI Data Table [SRAT]
*
* Format: [HexOffset DecimalOffset ByteLength] FieldName : FieldValue
*/
[000h 0000 4] Signature : "SRAT" [System Resource Affinity Table]
[004h 0004 4] Table Length : 000002D0
[008h 0008 1] Revision : 03
[009h 0009 1] Checksum : 39
[00Ah 0010 6] Oem ID : "DELL "
[010h 0016 8] Oem Table ID : "PE_SC3 "
[018h 0024 4] Oem Revision : 00000001
[01Ch 0028 4] Asl Compiler ID : "AMD "
[020h 0032 4] Asl Compiler Revision : 00000001
[024h 0036 4] Table Revision : 00000001
[028h 0040 8] Reserved : 0000000000000000
[030h 0048 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[031h 0049 1] Length : 10
[032h 0050 1] Proximity Domain Low(8) : 00
[033h 0051 1] Apic ID : 00
[034h 0052 4] Flags (decoded below) : 00000001
Enabled : 1
[038h 0056 1] Local Sapic EID : 00
[039h 0057 3] Proximity Domain High(24) : 000000
[03Ch 0060 4] Clock Domain : 00000000
[040h 0064 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[041h 0065 1] Length : 10
[042h 0066 1] Proximity Domain Low(8) : 00
[043h 0067 1] Apic ID : 01
[044h 0068 4] Flags (decoded below) : 00000001
Enabled : 1
[048h 0072 1] Local Sapic EID : 00
[049h 0073 3] Proximity Domain High(24) : 000000
[04Ch 0076 4] Clock Domain : 00000000
[050h 0080 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[051h 0081 1] Length : 10
[052h 0082 1] Proximity Domain Low(8) : 00
[053h 0083 1] Apic ID : 08
[054h 0084 4] Flags (decoded below) : 00000001
Enabled : 1
[058h 0088 1] Local Sapic EID : 00
[059h 0089 3] Proximity Domain High(24) : 000000
[05Ch 0092 4] Clock Domain : 00000000
[060h 0096 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[061h 0097 1] Length : 10
[062h 0098 1] Proximity Domain Low(8) : 00
[063h 0099 1] Apic ID : 09
[064h 0100 4] Flags (decoded below) : 00000001
Enabled : 1
[068h 0104 1] Local Sapic EID : 00
[069h 0105 3] Proximity Domain High(24) : 000000
[06Ch 0108 4] Clock Domain : 00000000
[070h 0112 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[071h 0113 1] Length : 10
[072h 0114 1] Proximity Domain Low(8) : 01
[073h 0115 1] Apic ID : 10
[074h 0116 4] Flags (decoded below) : 00000001
Enabled : 1
[078h 0120 1] Local Sapic EID : 00
[079h 0121 3] Proximity Domain High(24) : 000000
[07Ch 0124 4] Clock Domain : 00000000
[080h 0128 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[081h 0129 1] Length : 10
[082h 0130 1] Proximity Domain Low(8) : 01
[083h 0131 1] Apic ID : 11
[084h 0132 4] Flags (decoded below) : 00000001
Enabled : 1
[088h 0136 1] Local Sapic EID : 00
[089h 0137 3] Proximity Domain High(24) : 000000
[08Ch 0140 4] Clock Domain : 00000000
[090h 0144 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[091h 0145 1] Length : 10
[092h 0146 1] Proximity Domain Low(8) : 01
[093h 0147 1] Apic ID : 18
[094h 0148 4] Flags (decoded below) : 00000001
Enabled : 1
[098h 0152 1] Local Sapic EID : 00
[099h 0153 3] Proximity Domain High(24) : 000000
[09Ch 0156 4] Clock Domain : 00000000
[0A0h 0160 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[0A1h 0161 1] Length : 10
[0A2h 0162 1] Proximity Domain Low(8) : 01
[0A3h 0163 1] Apic ID : 19
[0A4h 0164 4] Flags (decoded below) : 00000001
Enabled : 1
[0A8h 0168 1] Local Sapic EID : 00
[0A9h 0169 3] Proximity Domain High(24) : 000000
[0ACh 0172 4] Clock Domain : 00000000
[0B0h 0176 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[0B1h 0177 1] Length : 10
[0B2h 0178 1] Proximity Domain Low(8) : 02
[0B3h 0179 1] Apic ID : 20
[0B4h 0180 4] Flags (decoded below) : 00000001
Enabled : 1
[0B8h 0184 1] Local Sapic EID : 00
[0B9h 0185 3] Proximity Domain High(24) : 000000
[0BCh 0188 4] Clock Domain : 00000000
[0C0h 0192 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[0C1h 0193 1] Length : 10
[0C2h 0194 1] Proximity Domain Low(8) : 02
[0C3h 0195 1] Apic ID : 21
[0C4h 0196 4] Flags (decoded below) : 00000001
Enabled : 1
[0C8h 0200 1] Local Sapic EID : 00
[0C9h 0201 3] Proximity Domain High(24) : 000000
[0CCh 0204 4] Clock Domain : 00000000
[0D0h 0208 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[0D1h 0209 1] Length : 10
[0D2h 0210 1] Proximity Domain Low(8) : 02
[0D3h 0211 1] Apic ID : 28
[0D4h 0212 4] Flags (decoded below) : 00000001
Enabled : 1
[0D8h 0216 1] Local Sapic EID : 00
[0D9h 0217 3] Proximity Domain High(24) : 000000
[0DCh 0220 4] Clock Domain : 00000000
[0E0h 0224 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[0E1h 0225 1] Length : 10
[0E2h 0226 1] Proximity Domain Low(8) : 02
[0E3h 0227 1] Apic ID : 29
[0E4h 0228 4] Flags (decoded below) : 00000001
Enabled : 1
[0E8h 0232 1] Local Sapic EID : 00
[0E9h 0233 3] Proximity Domain High(24) : 000000
[0ECh 0236 4] Clock Domain : 00000000
[0F0h 0240 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[0F1h 0241 1] Length : 10
[0F2h 0242 1] Proximity Domain Low(8) : 03
[0F3h 0243 1] Apic ID : 30
[0F4h 0244 4] Flags (decoded below) : 00000001
Enabled : 1
[0F8h 0248 1] Local Sapic EID : 00
[0F9h 0249 3] Proximity Domain High(24) : 000000
[0FCh 0252 4] Clock Domain : 00000000
[100h 0256 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[101h 0257 1] Length : 10
[102h 0258 1] Proximity Domain Low(8) : 03
[103h 0259 1] Apic ID : 31
[104h 0260 4] Flags (decoded below) : 00000001
Enabled : 1
[108h 0264 1] Local Sapic EID : 00
[109h 0265 3] Proximity Domain High(24) : 000000
[10Ch 0268 4] Clock Domain : 00000000
[110h 0272 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[111h 0273 1] Length : 10
[112h 0274 1] Proximity Domain Low(8) : 03
[113h 0275 1] Apic ID : 38
[114h 0276 4] Flags (decoded below) : 00000001
Enabled : 1
[118h 0280 1] Local Sapic EID : 00
[119h 0281 3] Proximity Domain High(24) : 000000
[11Ch 0284 4] Clock Domain : 00000000
[120h 0288 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[121h 0289 1] Length : 10
[122h 0290 1] Proximity Domain Low(8) : 03
[123h 0291 1] Apic ID : 39
[124h 0292 4] Flags (decoded below) : 00000001
Enabled : 1
[128h 0296 1] Local Sapic EID : 00
[129h 0297 3] Proximity Domain High(24) : 000000
[12Ch 0300 4] Clock Domain : 00000000
[130h 0304 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[131h 0305 1] Length : 10
[132h 0306 1] Proximity Domain Low(8) : 04
[133h 0307 1] Apic ID : 40
[134h 0308 4] Flags (decoded below) : 00000001
Enabled : 1
[138h 0312 1] Local Sapic EID : 00
[139h 0313 3] Proximity Domain High(24) : 000000
[13Ch 0316 4] Clock Domain : 00000000
[140h 0320 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[141h 0321 1] Length : 10
[142h 0322 1] Proximity Domain Low(8) : 04
[143h 0323 1] Apic ID : 41
[144h 0324 4] Flags (decoded below) : 00000001
Enabled : 1
[148h 0328 1] Local Sapic EID : 00
[149h 0329 3] Proximity Domain High(24) : 000000
[14Ch 0332 4] Clock Domain : 00000000
[150h 0336 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[151h 0337 1] Length : 10
[152h 0338 1] Proximity Domain Low(8) : 04
[153h 0339 1] Apic ID : 48
[154h 0340 4] Flags (decoded below) : 00000001
Enabled : 1
[158h 0344 1] Local Sapic EID : 00
[159h 0345 3] Proximity Domain High(24) : 000000
[15Ch 0348 4] Clock Domain : 00000000
[160h 0352 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[161h 0353 1] Length : 10
[162h 0354 1] Proximity Domain Low(8) : 04
[163h 0355 1] Apic ID : 49
[164h 0356 4] Flags (decoded below) : 00000001
Enabled : 1
[168h 0360 1] Local Sapic EID : 00
[169h 0361 3] Proximity Domain High(24) : 000000
[16Ch 0364 4] Clock Domain : 00000000
[170h 0368 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[171h 0369 1] Length : 10
[172h 0370 1] Proximity Domain Low(8) : 05
[173h 0371 1] Apic ID : 50
[174h 0372 4] Flags (decoded below) : 00000001
Enabled : 1
[178h 0376 1] Local Sapic EID : 00
[179h 0377 3] Proximity Domain High(24) : 000000
[17Ch 0380 4] Clock Domain : 00000000
[180h 0384 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[181h 0385 1] Length : 10
[182h 0386 1] Proximity Domain Low(8) : 05
[183h 0387 1] Apic ID : 51
[184h 0388 4] Flags (decoded below) : 00000001
Enabled : 1
[188h 0392 1] Local Sapic EID : 00
[189h 0393 3] Proximity Domain High(24) : 000000
[18Ch 0396 4] Clock Domain : 00000000
[190h 0400 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[191h 0401 1] Length : 10
[192h 0402 1] Proximity Domain Low(8) : 05
[193h 0403 1] Apic ID : 58
[194h 0404 4] Flags (decoded below) : 00000001
Enabled : 1
[198h 0408 1] Local Sapic EID : 00
[199h 0409 3] Proximity Domain High(24) : 000000
[19Ch 0412 4] Clock Domain : 00000000
[1A0h 0416 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[1A1h 0417 1] Length : 10
[1A2h 0418 1] Proximity Domain Low(8) : 05
[1A3h 0419 1] Apic ID : 59
[1A4h 0420 4] Flags (decoded below) : 00000001
Enabled : 1
[1A8h 0424 1] Local Sapic EID : 00
[1A9h 0425 3] Proximity Domain High(24) : 000000
[1ACh 0428 4] Clock Domain : 00000000
[1B0h 0432 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[1B1h 0433 1] Length : 10
[1B2h 0434 1] Proximity Domain Low(8) : 06
[1B3h 0435 1] Apic ID : 60
[1B4h 0436 4] Flags (decoded below) : 00000001
Enabled : 1
[1B8h 0440 1] Local Sapic EID : 00
[1B9h 0441 3] Proximity Domain High(24) : 000000
[1BCh 0444 4] Clock Domain : 00000000
[1C0h 0448 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[1C1h 0449 1] Length : 10
[1C2h 0450 1] Proximity Domain Low(8) : 06
[1C3h 0451 1] Apic ID : 61
[1C4h 0452 4] Flags (decoded below) : 00000001
Enabled : 1
[1C8h 0456 1] Local Sapic EID : 00
[1C9h 0457 3] Proximity Domain High(24) : 000000
[1CCh 0460 4] Clock Domain : 00000000
[1D0h 0464 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[1D1h 0465 1] Length : 10
[1D2h 0466 1] Proximity Domain Low(8) : 06
[1D3h 0467 1] Apic ID : 68
[1D4h 0468 4] Flags (decoded below) : 00000001
Enabled : 1
[1D8h 0472 1] Local Sapic EID : 00
[1D9h 0473 3] Proximity Domain High(24) : 000000
[1DCh 0476 4] Clock Domain : 00000000
[1E0h 0480 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[1E1h 0481 1] Length : 10
[1E2h 0482 1] Proximity Domain Low(8) : 06
[1E3h 0483 1] Apic ID : 69
[1E4h 0484 4] Flags (decoded below) : 00000001
Enabled : 1
[1E8h 0488 1] Local Sapic EID : 00
[1E9h 0489 3] Proximity Domain High(24) : 000000
[1ECh 0492 4] Clock Domain : 00000000
[1F0h 0496 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[1F1h 0497 1] Length : 10
[1F2h 0498 1] Proximity Domain Low(8) : 07
[1F3h 0499 1] Apic ID : 70
[1F4h 0500 4] Flags (decoded below) : 00000001
Enabled : 1
[1F8h 0504 1] Local Sapic EID : 00
[1F9h 0505 3] Proximity Domain High(24) : 000000
[1FCh 0508 4] Clock Domain : 00000000
[200h 0512 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[201h 0513 1] Length : 10
[202h 0514 1] Proximity Domain Low(8) : 07
[203h 0515 1] Apic ID : 71
[204h 0516 4] Flags (decoded below) : 00000001
Enabled : 1
[208h 0520 1] Local Sapic EID : 00
[209h 0521 3] Proximity Domain High(24) : 000000
[20Ch 0524 4] Clock Domain : 00000000
[210h 0528 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[211h 0529 1] Length : 10
[212h 0530 1] Proximity Domain Low(8) : 07
[213h 0531 1] Apic ID : 78
[214h 0532 4] Flags (decoded below) : 00000001
Enabled : 1
[218h 0536 1] Local Sapic EID : 00
[219h 0537 3] Proximity Domain High(24) : 000000
[21Ch 0540 4] Clock Domain : 00000000
[220h 0544 1] Subtable Type : 00 [Processor Local APIC/SAPIC Affinity]
[221h 0545 1] Length : 10
[222h 0546 1] Proximity Domain Low(8) : 07
[223h 0547 1] Apic ID : 79
[224h 0548 4] Flags (decoded below) : 00000001
Enabled : 1
[228h 0552 1] Local Sapic EID : 00
[229h 0553 3] Proximity Domain High(24) : 000000
[22Ch 0556 4] Clock Domain : 00000000
[230h 0560 1] Subtable Type : 01 [Memory Affinity]
[231h 0561 1] Length : 28
[232h 0562 4] Proximity Domain : 00000001
[236h 0566 2] Reserved1 : 0000
[238h 0568 8] Base Address : 0000000000000000
[240h 0576 8] Address Length : 00000000000A0000
[248h 0584 4] Reserved2 : 00000000
[24Ch 0588 4] Flags (decoded below) : 00000001
Enabled : 1
Hot Pluggable : 0
Non-Volatile : 0
[250h 0592 8] Reserved3 : 0000000000000000
[258h 0600 1] Subtable Type : 01 [Memory Affinity]
[259h 0601 1] Length : 28
[25Ah 0602 4] Proximity Domain : 00000001
[25Eh 0606 2] Reserved1 : 0000
[260h 0608 8] Base Address : 0000000000100000
[268h 0616 8] Address Length : 000000007FF00000
[270h 0624 4] Reserved2 : 00000000
[274h 0628 4] Flags (decoded below) : 00000001
Enabled : 1
Hot Pluggable : 0
Non-Volatile : 0
[278h 0632 8] Reserved3 : 0000000000000000
[280h 0640 1] Subtable Type : 01 [Memory Affinity]
[281h 0641 1] Length : 28
[282h 0642 4] Proximity Domain : 00000001
[286h 0646 2] Reserved1 : 0000
[288h 0648 8] Base Address : 0000000100000000
[290h 0656 8] Address Length : 0000000380000000
[298h 0664 4] Reserved2 : 00000000
[29Ch 0668 4] Flags (decoded below) : 00000001
Enabled : 1
Hot Pluggable : 0
Non-Volatile : 0
[2A0h 0672 8] Reserved3 : 0000000000000000
[2A8h 0680 1] Subtable Type : 01 [Memory Affinity]
[2A9h 0681 1] Length : 28
[2AAh 0682 4] Proximity Domain : 00000005
[2AEh 0686 2] Reserved1 : 0000
[2B0h 0688 8] Base Address : 0000000480000000
[2B8h 0696 8] Address Length : 0000000400000000
[2C0h 0704 4] Reserved2 : 00000000
[2C4h 0708 4] Flags (decoded below) : 00000001
Enabled : 1
Hot Pluggable : 0
Non-Volatile : 0
[2C8h 0712 8] Reserved3 : 0000000000000000
Raw Table Data: Length 720 (0x2D0)
0000: 53 52 41 54 D0 02 00 00 03 39 44 45 4C 4C 20 20 // SRAT.....9DELL
0010: 50 45 5F 53 43 33 20 20 01 00 00 00 41 4D 44 20 // PE_SC3 ....AMD
0020: 01 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 // ................
0030: 00 10 00 00 01 00 00 00 00 00 00 00 00 00 00 00 // ................
0040: 00 10 00 01 01 00 00 00 00 00 00 00 00 00 00 00 // ................
0050: 00 10 00 08 01 00 00 00 00 00 00 00 00 00 00 00 // ................
0060: 00 10 00 09 01 00 00 00 00 00 00 00 00 00 00 00 // ................
0070: 00 10 01 10 01 00 00 00 00 00 00 00 00 00 00 00 // ................
0080: 00 10 01 11 01 00 00 00 00 00 00 00 00 00 00 00 // ................
0090: 00 10 01 18 01 00 00 00 00 00 00 00 00 00 00 00 // ................
00A0: 00 10 01 19 01 00 00 00 00 00 00 00 00 00 00 00 // ................
00B0: 00 10 02 20 01 00 00 00 00 00 00 00 00 00 00 00 // ... ............
00C0: 00 10 02 21 01 00 00 00 00 00 00 00 00 00 00 00 // ...!............
00D0: 00 10 02 28 01 00 00 00 00 00 00 00 00 00 00 00 // ...(............
00E0: 00 10 02 29 01 00 00 00 00 00 00 00 00 00 00 00 // ...)............
00F0: 00 10 03 30 01 00 00 00 00 00 00 00 00 00 00 00 // ...0............
0100: 00 10 03 31 01 00 00 00 00 00 00 00 00 00 00 00 // ...1............
0110: 00 10 03 38 01 00 00 00 00 00 00 00 00 00 00 00 // ...8............
0120: 00 10 03 39 01 00 00 00 00 00 00 00 00 00 00 00 // ...9............
0130: 00 10 04 40 01 00 00 00 00 00 00 00 00 00 00 00 // ...@............
0140: 00 10 04 41 01 00 00 00 00 00 00 00 00 00 00 00 // ...A............
0150: 00 10 04 48 01 00 00 00 00 00 00 00 00 00 00 00 // ...H............
0160: 00 10 04 49 01 00 00 00 00 00 00 00 00 00 00 00 // ...I............
0170: 00 10 05 50 01 00 00 00 00 00 00 00 00 00 00 00 // ...P............
0180: 00 10 05 51 01 00 00 00 00 00 00 00 00 00 00 00 // ...Q............
0190: 00 10 05 58 01 00 00 00 00 00 00 00 00 00 00 00 // ...X............
01A0: 00 10 05 59 01 00 00 00 00 00 00 00 00 00 00 00 // ...Y............
01B0: 00 10 06 60 01 00 00 00 00 00 00 00 00 00 00 00 // ...`............
01C0: 00 10 06 61 01 00 00 00 00 00 00 00 00 00 00 00 // ...a............
01D0: 00 10 06 68 01 00 00 00 00 00 00 00 00 00 00 00 // ...h............
01E0: 00 10 06 69 01 00 00 00 00 00 00 00 00 00 00 00 // ...i............
01F0: 00 10 07 70 01 00 00 00 00 00 00 00 00 00 00 00 // ...p............
0200: 00 10 07 71 01 00 00 00 00 00 00 00 00 00 00 00 // ...q............
0210: 00 10 07 78 01 00 00 00 00 00 00 00 00 00 00 00 // ...x............
0220: 00 10 07 79 01 00 00 00 00 00 00 00 00 00 00 00 // ...y............
0230: 01 28 01 00 00 00 00 00 00 00 00 00 00 00 00 00 // .(..............
0240: 00 00 0A 00 00 00 00 00 00 00 00 00 01 00 00 00 // ................
0250: 00 00 00 00 00 00 00 00 01 28 01 00 00 00 00 00 // .........(......
0260: 00 00 10 00 00 00 00 00 00 00 F0 7F 00 00 00 00 // ................
0270: 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 // ................
0280: 01 28 01 00 00 00 00 00 00 00 00 00 01 00 00 00 // .(..............
0290: 00 00 00 80 03 00 00 00 00 00 00 00 01 00 00 00 // ................
02A0: 00 00 00 00 00 00 00 00 01 28 05 00 00 00 00 00 // .........(......
02B0: 00 00 00 80 04 00 00 00 00 00 00 00 04 00 00 00 // ................
02C0: 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 // ................
