As Vlastimil mentioned at [1], it would be nice to have some guide about memory allocation. I've drafted an initial version that tries to summarize "best practices" for allocation functions and GFP usage. [1] https://www.spinics.net/lists/netfilter-devel/msg55542.html >From 8027c0d4b750b8dbd687234feda63305d0d5a057 Mon Sep 17 00:00:00 2001 From: Mike Rapoport <rppt@xxxxxxxxxxxxxxxxxx> Date: Wed, 15 Aug 2018 09:10:06 +0300 Subject: [RFC PATCH] docs/core-api: add memory allocation guide Signed-off-by: Mike Rapoport <rppt@xxxxxxxxxxxxxxxxxx> --- Documentation/core-api/gfp_mask-from-fs-io.rst | 2 + Documentation/core-api/index.rst | 1 + Documentation/core-api/memory-allocation.rst | 117 +++++++++++++++++++++++++ Documentation/core-api/mm-api.rst | 2 + 4 files changed, 122 insertions(+) create mode 100644 Documentation/core-api/memory-allocation.rst diff --git a/Documentation/core-api/gfp_mask-from-fs-io.rst b/Documentation/core-api/gfp_mask-from-fs-io.rst index e0df8f4..e7c32a8 100644 --- a/Documentation/core-api/gfp_mask-from-fs-io.rst +++ b/Documentation/core-api/gfp_mask-from-fs-io.rst @@ -1,3 +1,5 @@ +.. _gfp_mask_from_fs_io: + ================================= GFP masks used from FS/IO context ================================= diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst index cdc2020..8afc0da 100644 --- a/Documentation/core-api/index.rst +++ b/Documentation/core-api/index.rst @@ -27,6 +27,7 @@ Core utilities errseq printk-formats circular-buffers + memory-allocation mm-api gfp_mask-from-fs-io timekeeping diff --git a/Documentation/core-api/memory-allocation.rst b/Documentation/core-api/memory-allocation.rst new file mode 100644 index 0000000..b1f2ad5 --- /dev/null +++ b/Documentation/core-api/memory-allocation.rst @@ -0,0 +1,117 @@ +======================= +Memory Allocation Guide +======================= + +Linux supplies variety of APIs for memory allocation. You can allocate +small chunks using `kmalloc` or `kmem_cache_alloc` families, large +virtually contiguous areas using `vmalloc` and it's derivatives, or +you can directly request pages from the page allocator with +`__get_free_pages`. It is also possible to use more specialized +allocators, for instance `cma_alloc` or `zs_malloc`. + +Most of the memory allocations APIs use GFP flags to express how that +memory should be allocated. The GFP acronym stands for "get free +pages", the underlying memory allocation function. + +Diversity of the allocation APIs combined with the numerous GFP flags +makes the question "How should I allocate memory?" not that easy to +answer, although very likely you should use + +:: + + kzalloc(<size>, GFP_KERNEL); + +Of course there are cases when other allocation APIs and different GFP +flags must be used. + +Get Free Page flags +=================== + +The GFP flags control the allocators behavior. They tell what memory +zones can be used, how hard the allocator should try to find a free +memory, whether the memory can be accessed by the userspace etc. The +:ref:`Documentation/core-api/mm-api.rst <mm-api-gfp-flags>` provides +reference documentation for the GFP flags and their combinations and +here we briefly outline their recommended usage: + + * Most of the times ``GFP_KERNEL`` is what you need. Memory for the + kernel data structures, DMAable memory, inode cache, all these and + many other allocations types can use ``GFP_KERNEL``. Note, that + using ``GFP_KERNEL`` implies ``GFP_RECLAIM``, which means that + direct reclaim may be triggered under memory pressure; the calling + context must be allowed to sleep. + * If the allocation is performed from an atomic context, e.g + interrupt handler, use ``GFP_ATOMIC``. + * Untrusted allocations triggered from userspace should be a subject + of kmem accounting and must have ``__GFP_ACCOUNT`` bit set. There + is handy ``GFP_KERNEL_ACCOUNT`` shortcut for ``GFP_KERNEL`` + allocations that should be accounted. + * Userspace allocations should use either of the ``GFP_USER``, + ``GFP_HIGHUSER`` and ``GFP_HIGHUSER_MOVABLE`` flags. The longer + the flag name the less restrictive it is. + + The ``GFP_HIGHUSER_MOVABLE`` does not require that allocated + memory will be directly accessible by the kernel or the hardware + and implies that the data may move. + + The ``GFP_HIGHUSER`` means that the allocated memory is not + movable, but it is not required to be directly accessible by the + kernel or the hardware. An example may be a hardware allocation + that maps data directly into userspace but has no addressing + limitations. + + The ``GFP_USER`` means that the allocated memory is not movable + and it must be directly accessible by the kernel or the + hardware. It is typically used by hardware for buffers that are + mapped to userspace (e.g. graphics) that hardware still must DMA + to. + +You may notice that quite a few allocations in the existing code +specify ``GFP_NOIO`` and ``GFP_NOFS``. Historically, they were used to +prevent recursion deadlocks caused by direct memory reclaim calling +back into the FS or IO paths and blocking on already held +resources. Since 4.12 the preferred way to address this issue is to +use new scope APIs described in +:ref:`Documentation/core-api/gfp_mask-from-fs-io.rst <gfp_mask_from_fs_io>`. + +Another legacy GFP flags are ``GFP_DMA`` and ``GFP_DMA32``. They are +used to ensure that the allocated memory is accessible by hardware +with limited addressing capabilities. So unless you are writing a +driver for a device with such restrictions, avoid using these flags. + +Selecting memory allocator +========================== + +The most straightforward way to allocate memory is to use a function +from the `kmalloc` family. And, to be on the safe size it's best to +use routines that set memory to zero, like `kzalloc`. If you need to +allocate memory for an array, there are `kmalloc_array` and `kcalloc` +helpers. + +The maximal size of a chunk that can be allocated with `kmalloc` is +limited. The actual limit depends on the hardware and the kernel +configuration, but it is a good practice to use `kmalloc` for objects +smaller than page size. + +For large allocations you can use `vmalloc` and `vzalloc`, or directly +request pages from the page allocator. The memory allocated by +`vmalloc` and related functions is not physically contiguous. + +If you are not sure whether the allocation size is too large for +`kmalloc` it is possible to use `kvmalloc` and its derivatives. It +will try to allocate memory with `kmalloc` and if the allocation fails +it will be retried with `vmalloc`. There are restrictions on which GFP +flags can be used with `kvmalloc`, please see :c:func:`kvmalloc_node` +reference documentation. Note, that `kvmalloc` may return memory that +is not physically contiguous. + +If you need to allocate many identical objects you can use slab cache +allocator. The cache should be set up with `kmem_cache_create` before +it can be used. Afterwards `kmem_cache_alloc` and its convenience +wrappers can allocate memory from that cache. + +When the allocated memory is no longer needed it must be freed. You +can use `kvfree` for the memory allocated with `kmalloc`, `vmalloc` +and `kvmalloc`. The slab caches should be freed with +`kmem_cache_free`. And don't forget to destroy the cache with +`kmem_cache_destroy`. diff --git a/Documentation/core-api/mm-api.rst b/Documentation/core-api/mm-api.rst index 46ae353..5ce1ec1 100644 --- a/Documentation/core-api/mm-api.rst +++ b/Documentation/core-api/mm-api.rst @@ -14,6 +14,8 @@ User Space Memory Access .. kernel-doc:: mm/util.c :functions: get_user_pages_fast +.. _mm-api-gfp-flags: + Memory Allocation Controls ========================== -- 2.7.4
