On Wed, 2013-06-19 at 11:58 +0900, Yasuaki Ishimatsu wrote: > 2013/06/19 8:59, Toshi Kani wrote: > > On Tue, 2013-06-18 at 19:05 +0200, Vasilis Liaskovitis wrote: > >> Hi, > >> > >> On Thu, Jun 13, 2013 at 09:03:52PM +0800, Tang Chen wrote: > >>> The following patch-set from Yinghai allocates pagetables to local nodes. > >>> v1: https://lkml.org/lkml/2013/3/7/642 > >>> v2: https://lkml.org/lkml/2013/3/10/47 > >>> v3: https://lkml.org/lkml/2013/4/4/639 > >>> v4: https://lkml.org/lkml/2013/4/11/829 > >>> > >>> Since pagetable pages are used by the kernel, they cannot be offlined. > >>> As a result, they cannot be hot-remove. > >>> > >>> This patch fix this problem with the following solution: > >>> > >>> 1. Introduce a new bootmem type LOCAL_NODE_DATAL, and register local > >>> pagetable pages as LOCAL_NODE_DATAL by setting page->lru.next to > >>> LOCAL_NODE_DATAL, just like we register SECTION_INFO pages. > >>> > >>> 2. Skip LOCAL_NODE_DATAL pages in offline/online procedures. When the > >>> whole memory block they reside in is offlined, the kernel can > >>> still access the pagetables. > >>> (This changes the semantics of offline/online a little bit.) > >> > >> This could be a design problem of part3: if we allow local pagetable memory > >> to not be offlined but allow the offlining to return successfully, then > >> hot-remove is going to succeed. But the direct mapped pagetable pages are still > >> mapped in the kernel. The hot-removed memblocks will suddenly disappear (think > >> physical DIMMs getting disabled in real hardware, or in a VM case the > >> corresponding guest memory getting freed from the emulator e.g. qemu/kvm). The > >> system can crash as a result. > >> > >> I think these local pagetables do need to be unmapped from kernel, offlined and > >> removed somehow - otherwise hot-remove should fail. Could they be migrated > >> alternatively e.g. to node 0 memory? But Iiuc direct mapped pages cannot be > >> migrated, correct? > >> > >> What is the original reason for local node pagetable allocation with regards > >> to memory hotplug? I assume we want to have hotplugged nodes use only their local > >> memory, so that there are no inter-node memory dependencies for hot-add/remove. > >> Are there other reasons that I am missing? > > > > I second Vasilis. The part1/2/3 series could be much simpler & less > > riskier if we focus on the SRAT changes first, and make the local node > > pagetable changes as a separate item. Is there particular reason why > > they have to be done at a same time? > > If my understanding is correct: > Main purpose of Yinghai's work is to put pagetable on local node ram. > For this, he needs to know SRAT information before setting pagetable. > So part1 does them same time. Thanks Yasuaki. I like Tang Chen's new plan, and I think it is going to be easier to proceed in that way. -Toshi -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>