>> Of course, other interfaces might make sense. >> >> You can then start using these memory blocks and hinder them from >> getting onlined (as a safety net) via memory notifiers. >> >> That would at least avoid you having to call >> add_memory/remove_memory/offline_pages/device_online/modifying >> memblock >> states manually. > > I see what you're saying and that definitely sounds safer. > > We would still need to call remove_memory and add_memory from memtrace > as > just offlining memory doesn't remove it from the linear page tables > (if > it's still in the page tables then hardware can prefetch it and if > hardware tracing is using it then the box checkstops). That prefetching part is interesting (and nasty as well). If we could at least get rid of the manual onlining/offlining, I would be able to sleep better at night ;) One step at a time. > >> >> (binding the memory block devices to a driver would be nicer, but the >> infrastructure is not really there yet - we have no such drivers in >> place yet) >> >>> I don't know the mm code nor how the notifiers work very well so I >>> can't quite see how the above would work. I'm assuming memtrace >>> would >>> register a hotplug notifier and when memory is offlined from >>> userspace, >>> the callback func in memtrace would be called if the priority was >>> high >>> enough? But how do we know that the memory being offlined is >>> intended >>> for usto touch? Is there a way to offline memory from userspace not >>> using sysfs or have I missed something in the sysfs interface? >> >> The notifier would really only be used to hinder onlining as a safety >> net. User space prepares (offlines) the memory blocks and then tells >> the >> drivers which memory blocks to use. >> >>> On a second read, perhaps you are assuming that memtrace is used >>> after >>> adding new memory at runtime? If so, that is not the case. If not, >>> then >>> would you be able to clarify what I'm not seeing? >> >> The main problem I see is that you are calling >> add_memory/remove_memory() on memory your device driver doesn't own. >> It >> could reside on a DIMM if I am not mistaking (or later on >> paravirtualized memory devices like virtio-mem if I ever get to >> implement them ;) ). > > This is just for baremetal/powernv so shouldn't affect virtual memory > devices. Good to now. > >> >> How is it guaranteed that the memory you are allocating does not >> reside >> on a DIMM for example added via add_memory() by the ACPI driver? > > Good point. We don't have ACPI on powernv but currently this would try > to remove memory from any online memory node, not just the ones that > are backed by RAM. oops. Okay, so essentially no memory hotplug/unplug along with memtrace. (can we document that somewhere?). I think add_memory()/try_remove_memory() could be tolerable in these environments (as it's only boot memory). -- Thanks, David / dhildenb
