On 10/17/2017 08:23 PM, Mike Kravetz wrote: > On 10/17/2017 07:20 AM, Guy Shattah wrote: >> 1. CMA has to preconfigured. We're suggesting mechanism that works 'out of the box' >> 2. Due to the pre-allocation techniques CMA imposes limitation on maximum >> allocated memory. RDMA users often require 1Gb or more, sometimes more. >> 3. CMA reserves memory in advance, our suggestion is using existing kernel memory >> mechanisms (THP for example) to allocate memory. > > I would not totally rule out the use of CMA. I like the way that it reserves > memory, but does not prohibit use by others. In addition, there can be > device (or purpose) specific reservations. I think the use case are devices that *cannot* function without contiguous memory, typical examples IIRC are smartphone cameras on with Android where only single app is working with the device at given time, so it's ok to reserve single area for the device, and allocation is done by the driver. Here we are talking about allocations done by potentially multiple userspace applications, so how do we reconcile that with the reservations? How does a single flag identify which device's area to use? How do we prevent one process depleting the area for other processes? IMHO it's another indication that a generic interface is infeasible and it should be driver-specific. BTW, does RDMA need a specific NUMA node to work optimally? (one closest to the device I presume?) Will it be the job of userspace to discover and bind itself to that node, in addition to using MAP_CONTIG? Or would that be another thing best handled by the driver? > However, since reservations need to happen quite early it is often done on > the kernel command line. IMO, this should be avoided if possible. There > are interfaces for arch specific code to make reservations. I do not know > the system initialization sequence well enough to know if it would be > possible for driver code to make CMA reservations. But, it looks doubtful. > -- To unsubscribe from this list: send the line "unsubscribe linux-api" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
