Hi Gregory, On Mon, 10 Mar 2025 10:22:11 -0400 Gregory Price <gourry@xxxxxxxxxx> wrote: > On Mon, Mar 10, 2025 at 09:26:48PM +0900, Honggyu Kim wrote: > > Yeah, the proximity domain detects the node correctly as follows in dmesg. > > > > [ 0.009915] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff] > > [ 0.009917] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0x207fffffff] > > [ 0.009919] ACPI: SRAT: Node 1 PXM 1 [mem 0x60f80000000-0x64f7fffffff] > > [ 0.009924] ACPI: SRAT: Node 2 PXM 6 [mem 0x2080000000-0x807fffffff] hotplug > > [ 0.009925] ACPI: SRAT: Node 3 PXM 7 [mem 0x64f80000000-0x6cf7fffffff] hotplug > > > > It is printed even before CXL detection. > > > > I wrote a some documentation on some example configurations last friday > > https://lore.kernel.org/linux-mm/Z226PG9t-Ih7fJDL@gourry-fedora-PF4VCD3F/T/#m2780e47df7f0962a79182502afc99843bb046205 > I've checked our CFMWS configurations using the information from this link and wrote them below. > This isn't exhaustive, but as I said with Rakie, I think your > configuration is probably ok - if slightly confusing. > I wonder if the below data is sufficient for your review in Rakies's email [1]. Please let me know. > What I'm going to guess is happening is you have 1 CFMWS per device that > do not have matching SRAT entries, and then the CFMWS covering these: > > > [ 0.009924] ACPI: SRAT: Node 2 PXM 6 [mem 0x2080000000-0x807fffffff] hotplug > > [ 0.009925] ACPI: SRAT: Node 3 PXM 7 [mem 0x64f80000000-0x6cf7fffffff] hotplug > > have interleave set up > In my understanding, both SRAT and CFMWS have the above device and interleave setup. Below are the SRAT entries (with some unnecessary lines removed) : [A128h 41256 001h] Subtable Type : 01 [Memory Affinity] [A12Ah 41258 004h] Proximity Domain : 00000006 [A130h 41264 008h] Base Address : 0000002080000000 [A138h 41272 008h] Address Length : 0000006000000000 Enabled : 1 Hot Pluggable : 1 ... [A150h 41296 001h] Subtable Type : 01 [Memory Affinity] [A152h 41298 004h] Proximity Domain : 00000007 [A158h 41304 008h] Base Address : 0000064F80000000 [A160h 41312 008h] Address Length : 0000008000000000 Enabled : 1 Hot Pluggable : 1 and below are the CFMWS configurations (with some unnecessary lines removed): [0A4h 0164 001h] Subtable Type : 01 [CXL Fixed Memory Window Structure] [0ACh 0172 008h] Window base address : 0000002080000000 <- Memory region [0B4h 0180 008h] Window size : 0000032780000000 [0BCh 0188 001h] Interleave Members : 01 <-- 2-way interleave [0BDh 0189 001h] Interleave Arithmetic : 01 [0C8h 0200 004h] First Target : 00000043 <-- host bridge id [0CCh 0204 004h] Next Target : 00000053 <-- host bridge id ... [170h 0368 001h] Subtable Type : 01 [CXL Fixed Memory Window Structure] [178h 0376 008h] Window base address : 0000064F80000000 [180h 0384 008h] Window size : 0000033000000000 [188h 0392 001h] Interleave Members : 01 <-- 2-way interleave [189h 0393 001h] Interleave Arithmetic : 01 [194h 0404 004h] First Target : 00000143 <-- host bridge id [198h 0408 004h] Next Target : 00000153 <-- host bridge id [1] https://lore.kernel.org/linux-mm/Z87zpg3TLRReikgu@gourry-fedora-PF4VCD3F/ Best regards, Yunjeong
