On Fri, Jan 29, 2021 at 2:06 PM Pavel Tatashin <pasha.tatashin@xxxxxxxxxx> wrote: > > > > Definitely, but we should try figuring out what's going on here. I > > > assume on x86-64 it behaves differently? > > > > Yes, we should root cause. I highly suspect that there is somewhere > > alignment miscalculations happen that cause this memory waste with the > > offset 16M. I am also not sure why the 2M label size was increased, > > and why 16M is now an alignment requirement. > > This appears to be because even if we set vmemmap to be outside of the > dax device, the alignment calculates the maximum size of vmemmap for > this device, and subtracts it from the devdax size. > See [1], line 795 is where this offset is calculated. > > This also explains why with 64K pages, the 16M offset worked: because > fewer struct pages were able to fit within 16M - label size. > > [1] https://soleen.com/source/xref/linux/drivers/nvdimm/pfn_devs.c?r=b7b3c01b&mo=18459&fi=718#795 Actually, strike the previous e-mail. The extra space is when we reserve vmemmap from devdax. IFF we do it from mem, the extra space is not added. Now, this alignment makes total sense. Pasha
