On Wed, 17 May 2006, Christoph Lameter wrote: > On Wed, 17 May 2006, Steven Rostedt wrote: > > > > Well I'd like to see a comprehensive solution including a fix for the > > > problems with allocper_cpu() allocations (allocper_cpu has to allocate > > > memory for potential processors... which could be a lot on > > > some types of systems and its allocated somewhere not on the nodes of the > > > processor since they may not yet be online). > > > > OK, now you're beyond what I'm working with ;) No hot plug CPUs for me. > > Well, at least not yet! > > You need to at least consider how this could be handled by the per_cpu > memory manangement. The VM thingie with dynamic per cpu memory would allow > a fixup of allocpercpu. > Last night, while aimlessly wondering the streets of Karlsruhe, I thought of some ideas. Maybe not very good ideas, but ideas never-the-less. How about a hybrid? Have the normal in kernel code use what is there today, with the indirection. Have modules and add on CPUs use an allocated vm area. Here's the thought: Have the boot time per_cpu variables (BTPCV) allocated like it is today. But make sure that they are paged align. Store away the initial values of per_cpu variables (PCV), for systems with hot pluggable CPUs. For modules, have a VM dedicated area. The assumption is that the modules will not have more PCVs than the kernel. Hopefully the PCVs of a module will not strain the TLB too much. As long as the modules PCVs are separated per cpu the same as the BTPCV then this will work. We can even use the extra space that was added in the alignment, if the modules PCV section is small enough to fit. Now for allocated PCV for a hot plugged CPU. We can dynamically allocate them when the CPU is loaded, and copy in the saved BTPCV. The hotplug CPU handling might be hard to work with the module handling, but both should be simple by themselves. So if we concentrate on just the hotplug first, then this might actually benefit you. So, Allocate a page alligned BTPCV for each online CPU and copy the section into them. Keep the initial section around. When a CPU comes on line, allocate the memory for the PCV in VM and copy the saved BTPCV into it. Then have the indirect pointer array (or CPU private register/variable) point to this section. It only puts strain on the TLB of the newly online CPU, but it gives us the option of placing the PCV into memory that we want (NUMA friendly). So I should forget about the modules for now, and get a hotplug PCV solution working. All the archs would need to do is to give a VM address where to store these variables. Hmm... Thoughts? -- Steve - : send the line "unsubscribe linux-ia64" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
![]() |