On 8 Apr 2024, at 12:30, Matthew Wilcox wrote: > On Thu, Apr 04, 2024 at 11:57:03AM -0700, Christoph Lameter (Ampere) wrote: >> On Mon, 1 Apr 2024, Jonathan Cameron wrote: >> >>> Sounds like useful data, but is it a suitable topic for LSF-MM? >>> What open questions etc is it raising? >> >> >> mTHP is new functionality that will require additional work to support more >> use cases. It is also unclear at this point in what usecases mTHP is useful >> and where no benefit can so far be seen. Also the effect of coalescing >> multiple PTE entries into one TLB entry is new to MM (CONT_PTE). I think we need a clarification of CONT_PTE from Christoph. From the context of ARM CPUs, CONT_PTE might be a group of PTEs with contiguous bit set. It was used by hugetlb and kernel linear mapping before Ryan added CONT_PTE support for mTHPs. This requires software support (setting contiguous bits) to be able to coalesce PTEs. But ARM also has this Hardware Page Aggregation (HPA) feature[1], which can coalesce PTEs without software intervention. I am not sure which ARM CPUs actually implement it. From the context of all CPUs, AMD has "PTE coalescing/clustering"[2] feature from Zen1. It is similar to ARM's HPA, not requiring software changes to coalesce PTEs. RISC-V also has Svnapot (Naturally-Aligned Power-of-Two Address-Translation Contiguity) [3], which requires software help. So with Matthew's folio patches back in 2020, hardware-only CONT_PTE would work since then. But software-assist CONT_PTE just began to work on ARM CPUs with Ryan's cont-pte patchset for anonymous memory and page cache. >> >> Ultimately it would be useful to have mTHP support also provide larger >> blocksize capabilities for filesystem etc etc. mTHP needs to mature and an >> analysis of the arguable a bit experimental state of affairs can help a lot >> in getting there. > > Have you been paying attention to anything that's been happening in Linux > development in the last three years? 7b230db3b8d3 introduced folios > in December 2020 (was merged in November 2021 for v5.16). v5.17 (March > 2022) did everything short of enabling large folios for the page cache, > which landed in v5.18 (May 2022). We started using cont-PTEs for large > folios in August 2023. Again, the page cache led the way here and we're > just adding support for anonymous large folios (called mTHP) now. Matthew, your cont-PTE here is "New page table range API" right? There is no ARM contiguous bit manipulation, right? > > There's still a ton of work to do, but we've been busy doing it since > LSFMM in Puerto Rico (2019) with READ_ONLY_THP_FOR_FS being the very > first result from the group of interested developers. > > And if you haven't seen the results that Ryan Roberts has posted for > the tests he's run, I suggest you look them up. He does a great job > of breaking down how much benefit he sees from the hardware side (use of > contPTE) vs the software side (shorter LRU lists, fewer atomic ops). It is definitely helpful to distinguish hardware and software benefits, since not all CPUs can coalesce PTEs. [1] https://developer.arm.com/documentation/100616/0301/register-descriptions/aarch64-system-registers/cpuectlr-el1--cpu-extended-control-register--el1 [2] https://www.eliot.so/memsys23.pdf [3] https://github.com/riscv/virtual-memory?tab=readme-ov-file#svnapot-naturally-aligned-power-of-two-address-translation-contiguity -- Best Regards, Yan, Zi
Attachment:
signature.asc
Description: OpenPGP digital signature
