Hi Tang, On Wed, Jun 19, 2013 at 03:29:06PM +0800, Tang Chen wrote: > Hi Vasilis, Yinghai, > > On 06/19/2013 01:05 AM, Vasilis Liaskovitis wrote: > ...... > > > >This could be a design problem of part3: if we allow local pagetable memory > >to not be offlined but allow the offlining to return successfully, then > >hot-remove is going to succeed. But the direct mapped pagetable pages are still > >mapped in the kernel. The hot-removed memblocks will suddenly disappear (think > >physical DIMMs getting disabled in real hardware, or in a VM case the > >corresponding guest memory getting freed from the emulator e.g. qemu/kvm). The > >system can crash as a result. > > > > Yes. Since the pagetable pages is only allocated to local node, a node may > have more than one device, hot-remove only one memory device could be > problematic. > > But I think it will work if we hot-remove a whole node. I should have > mentioned it. And sorry for the not fully test. ok, the crash I saw was also for the partial node removal. > I think allocating pagetable pages to local device will resolve this > problem. ok. Yes, you mentioned this approach before I think. > And need to restructure this patch-set. > > >I think these local pagetables do need to be unmapped from kernel, offlined and > >removed somehow - otherwise hot-remove should fail. Could they be migrated > >alternatively e.g. to node 0 memory? But Iiuc direct mapped pages cannot be > >migrated, correct? > > I think we have unmapped the local pagetables. in functions > free_pud/pmd/pte_table(), we cleared pud, pmd, and pte. We just didn't > free the pagetable pages to buddy. ok, thanks for explaining. > > But when we are not hot-removing the whole node, it is still problematic. > This is true, and it is my design problem. > > > > >What is the original reason for local node pagetable allocation with regards > >to memory hotplug? I assume we want to have hotplugged nodes use only their local > >memory, so that there are no inter-node memory dependencies for hot-add/remove. > >Are there other reasons that I am missing? > > I think the original reason to do local node pagetable is to improve > performance. > Using local pagetable, vmemmap and so on will be faster. > > But actually I think there is no particular reason to implement > memory hot-remove > and local node pagetable at the same time. And before this > patch-set, I also > suggested once that implement memory hot-remove first, and then > improve it to > local pagetable. But Yinghai has done the local pagetable work in > has patches (part1). > And my work is based on his patches. So I just did it. > > But obviously it is more complicated than I thought. > > And now, it seems tj has some more thinking on part1. > > So how about the following plan: > 1. Implement arranging hotpluggable memory with SRAT first, without > local pagetable. > (The main work in part2. And of course, need some patches in part1.) agreed (and yes, several patches from part1 will be needed to do the early srat parsing here) > 2. Do the local device pagetable work, not local node. > 3. Improve memory hotplug to support local device pagetable. ok, I 'll think about these as well, and help out. thanks, - Vasilis -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
