On 19-08-2020 02:05 am, John Hubbard wrote:
On 8/18/20 10:06 AM, Ralph Campbell wrote:
On 8/18/20 12:15 AM, Valmiki wrote:
Hi All,
Im trying to understand heterogeneous memory management, i have
following doubts.
If HMM is being used we dont have to use DMA controller on device for
memory transfers ?
Hi,
Nothing about HMM either requires or prevents using DMA controllers.
Without DMA if software is managing page faults and migrations, will
there be any performance impacts ?
Is HMM targeted for any specific use cases where DMA controller is
not there on device ?
Regards,
Valmiki
There are two APIs that are part of "HMM" and are independent of each
other.
hmm_range_fault() is for getting the physical address of a system
resident memory page that
a device can map but is not pinned in the usual way I/O increases the
page reference count
to pin the page. The device driver has to handle invalidation
callbacks to remove the device
mapping. This lets the device access the page without moving it.
migrate_vma_setup(), migrate_vma_pages(), and migrate_vma_finalize()
are used by the device
driver to migrate data to device private memory. After migration, the
system memory is freed
and the CPU page table holds an invalid PTE that points to the device
private struct page
(similar to a swap PTE). If the CPU process faults on that address,
there is a callback
to the driver to migrate it back to system memory. This is where
device DMA engines can
be used to copy data to/from system memory and device private memory.
The use case for the above is to be able to run code such as OpenCL on
GPUs and CPUs using
the same virtual addresses without having to call special memory
allocators.
In other words, just use mmap() and malloc() and not clSVMAlloc().
There is a performance consideration here. If the GPU accesses the
data over PCIe to
system memory, there is much less bandwidth than accessing local GPU
memory. If the
data is to be accessed/used many times, it can be more efficient to
migrate the data
to local GPU memory. If the data is only accessed a few times, then it
is probably
more efficient to map system memory.
Ralph, that's a good write-up!
Valmiki, did you already read Documentation/vm/hmm.rst, before posting
your question?
It's OK to say "no"--I'm not asking in order to criticize, but in order
to calibrate
the documentation. Because, we should consider merging in Ralph's
write-up above
into hmm.rst, depending on if it helps (which I expect it does, but I'm
tainted by
reading hmm.rst too many times and now I can't see what might be missing).
Any time someone new tries to understand the system, it's an opportunity
to "unit test"
the documentation. Ideally, hmm.rst would answer many of a first-time
reader's questions,
that's where we'd like to end up.
thanks,
Hi John, i did give initial reading, but Ralph details on migration_*
API's above helped a lot to clarify things better, yes adding above
details into hmm.rst would help beginners.
Regards,
Valmiki
