On Tue, May 26, 2020 at 07:27:15AM -0700, Dave Hansen wrote:
> On 5/25/20 8:08 AM, Kirill A. Shutemov wrote:
> >>>> + if (not_addressable) {
> >>>> + pr_err("%lldGB of physical memory is not addressable in the paging mode\n",
> >>>> + not_addressable >> 30);
> >>>> + if (!pgtable_l5_enabled())
> >>>> + pr_err("Consider enabling 5-level paging\n");
> >> Could this happen at all when l5 is enabled?
> >> Does it mean we need kmap() for 64-bit?
> > It's future-profing. Who knows what paging modes we would have in the
> > future.
>
> Future-proofing and firmware-proofing. :)
>
> In any case, are we *really* limited to 52 bits of physical memory with
> 5-level paging?
Yes. It's architectural. SDM says "MAXPHYADDR is at most 52" (Vol 3A,
4.1.4).
I guess it can be extended with an opt-in feature and relevant changes to
page table structure. But as of today there's no such thing.
> Previously, we said we were limited to 46 bits, and now
> we're saying that the limit is 52 with 5-level paging:
>
> #define MAX_PHYSMEM_BITS (pgtable_l5_enabled() ? 52 : 46)
>
> The 46 was fine with the 48 bits of address space on 4-level paging
> systems since we need 1/2 of the address space for userspace, 1/4 for
> the direct map and 1/4 for the vmalloc-and-friends area. At 46 bits of
> address space, we fill up the direct map.
>
> The hardware designers know this and never enumerated a MAXPHYADDR from
> CPUID which was higher than what we could cover with 46 bits. It was
> nice and convenient that these two separate things matched:
> 1. The amount of physical address space addressable in a direct map
> consuming 1/4 of the virtual address space.
> 2. The CPU-enumerated MAXPHYADDR which among other things dictates how
> much physical address space is addressable in a PTE.
>
> But, with 5-level paging, things are a little different. The limit in
> addressable memory because of running out of the direct map actually
> happens at 55 bits (57-2=55, analogous to the 4-level 48-2=46).
>
> So shouldn't it technically be this:
>
> #define MAX_PHYSMEM_BITS (pgtable_l5_enabled() ? 55 : 46)
>
> ?
Bits above 52 are ignored in the page table entries and accessible to
software. Some of them got claimed by HW features (XD-bit, protection
keys), but such features require explicit opt-in on software side.
Kernel could claim bits 53-55 for the physical address, but it doesn't get
us anything: if future HW would provide such feature it would require
opt-in. On other hand claiming them now means we cannot use them for other
purposes as SW bit. I don't see a point.
--
Kirill A. Shutemov
