On Tue, 2006-03-28 at 12:43 -0600, Robin Holt wrote:
> The system was a fully populated 512 node SGI machine. The way that
> memory is physically layed out results in a single pgdat which covers
> the node with two holes in it. This is new hardware with larger gaps
> between the chunks of memory that earlier version had. As show_mem()
> is traversing the entire systems memory to print out stats on remaining
> memory, it takes faults while trying to look at holes in the array of
> struct pages.
Could you explain a bit how this works on ia64? I know about the
vmem_map. Is the time spent on filling TLB entries when you hit a
'struct page' that isn't backed by real memory?
> At this point, I am looking for any sort of direction on what would be
> a reasonable fix. Should show_mem() be made to skip to a page aligned
> point in the array when the fault fails?
Yeah, this would be my first instinct. Perhaps a function like:
unsigned long hole_nr_pages(unsigned long pfn)
{
}
For sparsemem, it could just return PAGES_PER_SECTION. For
architectures like ia64, it could either return the minimum hole size,
or be smarter and go look in some arch-specific information to find the
real hole size.
Maybe something like this in your show_mem():
for_each_pgdat(pgdat) {
...
for(i = 0; i < pgdat->node_spanned_pages; i++) {
struct page *page;
if (pfn_valid(pgdat->node_start_pfn + i))
page = pfn_to_page(pgdat->node_start_pfn + i);
else
- continue;
+ /* -1 to offset i++ */
+ pfn += hole_nr_pages(pfn) - 1;
> Should we add the information
> about start and end of hole to the pgdat()?
No. No. Please, no. :)
Sparsemem is pretty good at this already. Also, the whole idea of
DISCONTIGMEM was to have a pgdat that describes a contiguous area.
We've massacred that concept with NUMA stuff since then, but that _was_
the original idea.
> Should we have one pgdat per chunk?
That's one concept that probably won't work today. I went and tried to
untangle DISCONTIG node ids from NUMA node ids one day and failed
miserably. They're too intertwined.
> Are there other better ideas out there? Any direction would
> be greatly appreciated.
Get rid of the silly vmem_map[] :)
-- Dave
-
: send the line "unsubscribe linux-ia64" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
|