On 12/31/19 at 09:23am, Baoquan He wrote:
> On 12/30/19 at 12:38pm, Kirill A. Shutemov wrote:
> > memmap_init_zone() can be called on the ranges with holes during the
> > boot. It will skip any non-valid PFNs one-by-one. It works fine as long
> > as holes are not too big.
> >
> > But huge holes in the memory map causes a problem. It takes over 20
> > seconds to walk 32TiB hole. x86-64 with 5-level paging allows for much
> > larger holes in the memory map which would practically hang the system.
> >
> > Deferred struct page init doesn't help here. It only works on the
> > present ranges.
> >
> > Skipping non-present sections would fix the issue.
> >
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@xxxxxxxxxxxxxxx>
> > ---
> >
> > The situation can be emulated using the following QEMU patch:
> >
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index ac08e6360437..f5f2258092e1 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -1159,13 +1159,14 @@ void pc_memory_init(PCMachineState *pcms,
> > memory_region_add_subregion(system_memory, 0, ram_below_4g);
> > e820_add_entry(0, x86ms->below_4g_mem_size, E820_RAM);
> > if (x86ms->above_4g_mem_size > 0) {
> > + int shift = 45;
> > ram_above_4g = g_malloc(sizeof(*ram_above_4g));
> > memory_region_init_alias(ram_above_4g, NULL, "ram-above-4g", ram,
> > x86ms->below_4g_mem_size,
> > x86ms->above_4g_mem_size);
> > - memory_region_add_subregion(system_memory, 0x100000000ULL,
> > + memory_region_add_subregion(system_memory, 1ULL << shift,
> > ram_above_4g);
> > - e820_add_entry(0x100000000ULL, x86ms->above_4g_mem_size, E820_RAM);
> > + e820_add_entry(1ULL << shift, x86ms->above_4g_mem_size, E820_RAM);
> > }
> >
> > if (!pcmc->has_reserved_memory &&
> > diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> > index cde2a16b941a..694c26947bf6 100644
> > --- a/target/i386/cpu.h
> > +++ b/target/i386/cpu.h
> > @@ -1928,7 +1928,7 @@ uint64_t cpu_get_tsc(CPUX86State *env);
> > /* XXX: This value should match the one returned by CPUID
> > * and in exec.c */
> > # if defined(TARGET_X86_64)
> > -# define TCG_PHYS_ADDR_BITS 40
> > +# define TCG_PHYS_ADDR_BITS 52
> > # else
> > # define TCG_PHYS_ADDR_BITS 36
> > # endif
> >
> > ---
> > mm/page_alloc.c | 28 +++++++++++++++++++++++++++-
> > 1 file changed, 27 insertions(+), 1 deletion(-)
> >
> > diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> > index df62a49cd09e..442dc0244bb4 100644
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -5873,6 +5873,30 @@ overlap_memmap_init(unsigned long zone, unsigned long *pfn)
> > return false;
> > }
> >
> > +#ifdef CONFIG_SPARSEMEM
> > +/* Skip PFNs that belong to non-present sections */
> > +static inline __meminit unsigned long next_pfn(unsigned long pfn)
> > +{
> > + unsigned long section_nr;
> > +
> > + section_nr = pfn_to_section_nr(++pfn);
> > + if (present_section_nr(section_nr))
> > + return pfn;
> > +
> > + while (++section_nr <= __highest_present_section_nr) {
> > + if (present_section_nr(section_nr))
> > + return section_nr_to_pfn(section_nr);
> > + }
> > +
> > + return -1;
> > +}
> > +#else
> > +static inline __meminit unsigned long next_pfn(unsigned long pfn)
> > +{
> > + return pfn++;
> > +}
> > +#endif
> > +
> > /*
> > * Initially all pages are reserved - free ones are freed
> > * up by memblock_free_all() once the early boot process is
> > @@ -5912,8 +5936,10 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
> > * function. They do not exist on hotplugged memory.
> > */
> > if (context == MEMMAP_EARLY) {
> > - if (!early_pfn_valid(pfn))
> > + if (!early_pfn_valid(pfn)) {
> > + pfn = next_pfn(pfn) - 1;
>
> Just pass by, I think this is a necessary optimization. Wondering why
> next_pfn(pfn) is not put in for loop:
> - for (pfn = start_pfn; pfn < end_pfn; pfn++) {
> + for (pfn = start_pfn; pfn < end_pfn; pfn=next_pfn(pfn)) {
>
>
> > continue;
> > + }
> > if (!early_pfn_in_nid(pfn, nid))
> > continue;
>
> Why the other two 'continue' don't need be worried on the huge hole
> case?
OK, I see. early_pfn_valid() may have encountered the huge hole case,
the check in patch sounds reasonable.
FWIW, looks good to me.
Reviewed-by: Baoquan He <bhe@xxxxxxxxxx>
Thanks
Baoquan
