Good day,
First off -- a tiny disclaimer: the following are just my observations
which may or may not be valid (with likelihood shifted to the latter).
In a nutshell they come down to a question -- how many bits should
be used to store PXM domain ID in SRAT's memory affinity structure?
Examination of include/acpi/actbl1.h suggests number 32
(vide: proximity_domain field of struct acpi_srat_mem_affinity)
whereas the only SRAT-related spec I've managed to find
(Microsoft's "SRAT Version 1.2, March 31, 2003")[1] mentions
8 bits.
Where my interest is coming from, you might wonder --
I have a board (Supermicro X8DA3) SRAT of which looks like...
SRAT @ 0xbf79a4b0
0000: 53 52 41 54 28 01 00 00 01 0a 30 33 33 30 30 39 SRAT(.....033009
0010: 4f 45 4d 53 52 41 54 20 01 00 00 00 49 4e 54 4c OEMSRAT ....INTL
0020: 01 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 ................
0030: 00 10 00 00 01 00 00 00 00 00 00 00 00 00 00 00 ................
0040: 00 10 00 02 01 00 00 00 00 00 00 00 00 00 00 00 ................
0050: 00 10 00 04 01 00 00 00 00 00 00 00 00 00 00 00 ................
0060: 00 10 00 06 01 00 00 00 00 00 00 00 00 00 00 00 ................
0070: 01 28 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .(..............
0080: 00 00 0a 00 00 00 00 00 00 00 00 00 01 00 00 00 ................
0090: 00 00 00 00 00 00 00 00 01 28 00 00 00 00 00 00 .........(......
00a0: 00 00 10 00 00 00 00 00 00 00 f0 bf 00 00 00 00 ................
00b0: 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 ................
00c0: 00 10 01 10 01 00 00 00 00 00 00 00 00 00 00 00 ................
00d0: 00 10 01 12 01 00 00 00 00 00 00 00 00 00 00 00 ................
00e0: 00 10 01 14 01 00 00 00 00 00 00 00 00 00 00 00 ................
00f0: 00 10 01 16 01 00 00 00 00 00 00 00 00 00 00 00 ................
0100: 01 28 01 01 00 00 00 00 00 00 00 00 01 00 00 00 .(..............
0110: 00 00 00 c0 00 00 00 00 00 00 00 00 01 00 00 00 ................
0120: 00 00 00 00 00 00 00 00 ........
this. At offset 0x100 there's memory affinity structure with PXM domain
of either 1 (MS spec) or 257 (Linux code). Thing is -- no processor
affinity structures refer PXM domain 257. It seems they just ... can't (?).
In other words -- when I boot Linux, SRAT parsing code gives
me the following:
SRAT: PXM 0 -> APIC 0 -> Node 0
SRAT: PXM 0 -> APIC 2 -> Node 0
SRAT: PXM 0 -> APIC 4 -> Node 0
SRAT: PXM 0 -> APIC 6 -> Node 0
SRAT: PXM 1 -> APIC 16 -> Node 1
SRAT: PXM 1 -> APIC 18 -> Node 1
SRAT: PXM 1 -> APIC 20 -> Node 1
SRAT: PXM 1 -> APIC 22 -> Node 1
SRAT: Node 0 PXM 0 0-a0000
SRAT: Node 0 PXM 0 100000-c0000000
SRAT: Node 2 PXM 257 100000000-1c0000000
Such affinity results in all processors using node 0 which is not
exactly what I want.
Obviously, this board's SRAT isn't perfect but I believe a good practice
is not to make use reserved bits (if to assume MS doc's validity).
I've attached a PoC patch which makes things a bit better here. Better, as in:
SRAT: PXM 0 -> APIC 0 -> Node 0
SRAT: PXM 0 -> APIC 2 -> Node 0
SRAT: PXM 0 -> APIC 4 -> Node 0
SRAT: PXM 0 -> APIC 6 -> Node 0
SRAT: PXM 1 -> APIC 16 -> Node 1
SRAT: PXM 1 -> APIC 18 -> Node 1
SRAT: PXM 1 -> APIC 20 -> Node 1
SRAT: PXM 1 -> APIC 22 -> Node 1
SRAT: Node 0 PXM 0 0-a0000
SRAT: Node 0 PXM 0 100000-c0000000
SRAT: Node 1 PXM 1 100000000-1c0000000
Your thoughts?
Thanks,
Kris
References:
1. http://www.microsoft.com/whdc/system/sysinternals/sratdwn.mspx
--- include/acpi/actbl1.h.u32 2009-05-02 18:45:21.776000000 -0600
+++ include/acpi/actbl1.h 2009-05-02 18:43:50.260375000 -0600
@@ -1196,7 +1196,10 @@
struct acpi_srat_mem_affinity {
struct acpi_subtable_header header;
- u32 proximity_domain;
+ u8 proximity_domain;
+ u8 reserved3;
+ u8 reserved4;
+ u8 reserved5;
u16 reserved; /* Reserved, must be zero */
u64 base_address;
u64 length;
