On Tue, Jan 29, 2019 at 7:03 PM Jerome Glisse <jglisse@xxxxxxxxxx> wrote: [..] > > > 1) Convert ODP to use HMM underneath so that we share code between > > > infiniband ODP and GPU drivers. ODP do support DAX today so i can > > > not convert ODP to HMM without also supporting DAX in HMM otherwise > > > i would regress the ODP features. > > > > > > 2) I expect people will be running GPGPU on computer with file that > > > use DAX and they will want to use HMM there too, in fact from user- > > > space point of view wether the file is DAX or not should only change > > > one thing ie for DAX file you will never be able to use GPU memory. > > > > > > 3) I want to convert as many user of GUP to HMM (already posted > > > several patchset to GPU mailing list for that and i intend to post > > > a v2 of those latter on). Using HMM avoids GUP and it will avoid > > > the GUP pin as here we abide by mmu notifier hence we do not want to > > > inhibit any of the filesystem regular operation. Some of those GPU > > > driver do allow GUP on DAX file. So again i can not regress them. > > > > Is this really a GUP to HMM conversion, or a GUP to mmu_notifier > > solution? It would be good to boil this conversion down to the base > > building blocks. It seems "HMM" can mean several distinct pieces of > > infrastructure. Is it possible to replace some GUP usage with an > > mmu_notifier based solution without pulling in all of HMM? > > Kind of both, some of the GPU driver i am converting will use HMM for > more than just this GUP reason. But when and for what hardware they > will use HMM for is not something i can share (it is up to each vendor > to announce their hardware and feature on their own timeline). Typically a spec document precedes specific hardware announcement and Linux enabling is gated on public spec availability. > So yes you could do the mmu notifier solution without pulling HMM > mirror (note that you do not need to pull all of HMM, HMM as many > kernel config option and for this you only need to use HMM mirror). > But if you are not using HMM then you will just be duplicating the > same code as HMM mirror. So i believe it is better to share this > code and if we want to change core mm then we only have to update > HMM while keeping the API/contract with device driver intact. No. Linux should not end up with the HMM-mm as distinct from the Core-mm. For long term maintainability of Linux, the target should be one mm. > This > is one of the motivation behind HMM ie have it as an impedence layer > between mm and device drivers so that mm folks do not have to under- > stand every single device driver but only have to understand the > contract HMM has with all device driver that uses it. This gets to heart of my critique of the approach taken with HMM. The above statement is antithetical to Documentation/process/stable-api-nonsense.rst. If HMM is trying to set expectations that device-driver projects can write to a "stable" HMM api then HMM is setting those device-drivers up for failure. The possibility of refactoring driver code *across* vendors is a core tenet of Linux maintainability. If the refactoring requires the API exported to drivers to change then so be it. The expectation that all out-of-tree device-drivers should have is that the API they are using in kernel version X may not be there in version X+1. Having the driver upstream is the only surefire insurance against that thrash. HMM seems a bold experiment in trying to violate Linux development norms. > Also having each driver duplicating this code increase the risk of > one getting a little detail wrong. The hope is that sharing same > HMM code with all the driver then everyone benefit from debugging > the same code (i am hopping i do not have many bugs left :)) "each driver duplicating code" begs for refactoring driver code to common code and this refactoring is hindered if it must adhere to an "HMM" api.
