On 2018-12-04 2:11 p.m., Logan Gunthorpe wrote: > Also, in the same vein, I think it's wrong to have the API enumerate all > the different memory available in the system. The API should simply > allow userspace to say it wants memory that can be accessed by a set of > initiators with a certain set of attributes and the bind call tries to > fulfill that or fallback on system memory/hmm migration/whatever. That gets pretty complex when you also have take into account contention of links and bridges when multiple initiators are accessing multiple targets simultaneously. If you want the kernel to make sane decisions, it needs a lot more information about the expected memory access patterns. Highly optimized algorithms that use multiple GPUs and collective communications between them want to be able to place their memory objects in the right location to avoid such contention. You don't want such an algorithm to guess about opaque policy decisions in the kernel. If the policy changes, you have to re-optimize the algorithm. Regards, Felix > > Logan
