Hi, David, Thanks a lot for comments! David Hildenbrand <david@xxxxxxxxxx> writes: > On 30.09.24 07:51, Huang Ying wrote: >> On systems with TDX (Trust Domain eXtensions) enabled, memory ranges >> hot-added must be checked for compatibility by TDX. This is currently >> implemented through memory hotplug notifiers for each memory_block. >> If a memory range which isn't TDX compatible is hot-added, for >> example, some CXL memory, the command line as follows, >> $ echo 1 > /sys/devices/system/node/nodeX/memoryY/online >> will report something like, >> bash: echo: write error: Operation not permitted >> If pr_debug() is enabled, the error message like below will be shown >> in the kernel log, >> online_pages [mem 0xXXXXXXXXXX-0xXXXXXXXXXX] failed >> Both are too general to root cause the problem. This will confuse >> users. One solution is to print some error messages in the TDX memory >> hotplug notifier. However, memory hotplug notifiers are called for >> each memory block, so this may lead to a large volume of messages in >> the kernel log if a large number of memory blocks are onlined with a >> script or automatically. For example, the typical size of memory >> block is 128MB on x86_64, when online 64GB CXL memory, 512 messages >> will be logged. > > ratelimiting would likely help here a lot, but I agree that it is > suboptimal. > >> Therefore, in this patch, the whole hot-adding memory range is >> checked >> for TDX compatibility through a newly added architecture specific >> function (arch_check_hotplug_memory_range()). If rejected, the memory >> hot-adding will be aborted with a proper kernel log message. Which >> looks like something as below, >> virt/tdx: Reject hot-adding memory range: 0xXXXXXXXX-0xXXXXXXXX >> for TDX compatibility. >> > The target use case is to support CXL memory on TDX enabled systems. >> If the CXL memory isn't compatible with TDX, the whole CXL memory >> range hot-adding will be rejected. While the CXL memory can still be >> used via devdax interface. > > I'm curious, why can that memory be used through devdax but not > through the buddy? I'm probably missing something important :) Because only TDX compatible memory can be used for TDX guest. The buddy is used to allocate memory for TDX guest. While devdax will not be used for that. >> This also makes the original TDX memory hotplug notifier useless, so >> delete it. > > The online-notifier would even be too late when used with the > memmap-on-memory feature I assume, as we might be touching that memory > even before being able to call memory online notifiers. This should be OK. Because we will not use the memory for TDX guest in this way. > One way to handle that would be to switch to the MEM_PREPARE_ONLINE > notifier, but it's still called per-memory block. > > Nothing jumped at me, so > > Acked-by: David Hildenbrand <david@xxxxxxxxxx> Thank you very much! -- Best Regards, Huang, Ying
