On Wed, Aug 15, 2018 at 10:15:39AM +0200, Michal Hocko wrote: > On Wed 15-08-18 09:36:49, Mike Rapoport wrote: > > (this time with the subject, sorry for the noise) > > > > On Wed, Aug 15, 2018 at 09:34:47AM +0300, Mike Rapoport wrote: > > > As Vlastimil mentioned at [1], it would be nice to have some guide about > > > memory allocation. I've drafted an initial version that tries to summarize > > > "best practices" for allocation functions and GFP usage. > > > > > > [1] https://www.spinics.net/lists/netfilter-devel/msg55542.html > > > > > > From 8027c0d4b750b8dbd687234feda63305d0d5a057 Mon Sep 17 00:00:00 2001 > > > From: Mike Rapoport <rppt@xxxxxxxxxxxxxxxxxx> > > > Date: Wed, 15 Aug 2018 09:10:06 +0300 > > > Subject: [RFC PATCH] docs/core-api: add memory allocation guide > > > > > > Signed-off-by: Mike Rapoport <rppt@xxxxxxxxxxxxxxxxxx> > > > --- > > > Documentation/core-api/gfp_mask-from-fs-io.rst | 2 + > > > Documentation/core-api/index.rst | 1 + > > > Documentation/core-api/memory-allocation.rst | 117 +++++++++++++++++++++++++ > > > Documentation/core-api/mm-api.rst | 2 + > > > 4 files changed, 122 insertions(+) > > > create mode 100644 Documentation/core-api/memory-allocation.rst > > > > > > diff --git a/Documentation/core-api/gfp_mask-from-fs-io.rst b/Documentation/core-api/gfp_mask-from-fs-io.rst > > > index e0df8f4..e7c32a8 100644 > > > --- a/Documentation/core-api/gfp_mask-from-fs-io.rst > > > +++ b/Documentation/core-api/gfp_mask-from-fs-io.rst > > > @@ -1,3 +1,5 @@ > > > +.. _gfp_mask_from_fs_io: > > > + > > > ================================= > > > GFP masks used from FS/IO context > > > ================================= > > > diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst > > > index cdc2020..8afc0da 100644 > > > --- a/Documentation/core-api/index.rst > > > +++ b/Documentation/core-api/index.rst > > > @@ -27,6 +27,7 @@ Core utilities > > > errseq > > > printk-formats > > > circular-buffers > > > + memory-allocation > > > mm-api > > > gfp_mask-from-fs-io > > > timekeeping > > > diff --git a/Documentation/core-api/memory-allocation.rst b/Documentation/core-api/memory-allocation.rst > > > new file mode 100644 > > > index 0000000..b1f2ad5 > > > --- /dev/null > > > +++ b/Documentation/core-api/memory-allocation.rst > > > @@ -0,0 +1,117 @@ > > > +======================= > > > +Memory Allocation Guide > > > +======================= > > > + > > > +Linux supplies variety of APIs for memory allocation. You can allocate > > > +small chunks using `kmalloc` or `kmem_cache_alloc` families, large > > > +virtually contiguous areas using `vmalloc` and it's derivatives, or > > > +you can directly request pages from the page allocator with > > > +`__get_free_pages`. It is also possible to use more specialized > > I would rather not mention __get_free_pages. alloc_pages is a more > generic API and less subtle one. If you want to mention __get_free_pages > then please make sure to mention the subtlety (namely that is can > allocate only lowmem memory). > > > > +allocators, for instance `cma_alloc` or `zs_malloc`. > > > + > > > +Most of the memory allocations APIs use GFP flags to express how that > > > +memory should be allocated. The GFP acronym stands for "get free > > > +pages", the underlying memory allocation function. > > > + > > > +Diversity of the allocation APIs combined with the numerous GFP flags > > > +makes the question "How should I allocate memory?" not that easy to > > > +answer, although very likely you should use > > > + > > > +:: > > > + > > > + kzalloc(<size>, GFP_KERNEL); > > > + > > > +Of course there are cases when other allocation APIs and different GFP > > > +flags must be used. > > > + > > > +Get Free Page flags > > > +=================== > > > + > > > +The GFP flags control the allocators behavior. They tell what memory > > > +zones can be used, how hard the allocator should try to find a free > > > +memory, whether the memory can be accessed by the userspace etc. The > > > +:ref:`Documentation/core-api/mm-api.rst <mm-api-gfp-flags>` provides > > > +reference documentation for the GFP flags and their combinations and > > > +here we briefly outline their recommended usage: > > > + > > > + * Most of the times ``GFP_KERNEL`` is what you need. Memory for the > > > + kernel data structures, DMAable memory, inode cache, all these and > > > + many other allocations types can use ``GFP_KERNEL``. Note, that > > > + using ``GFP_KERNEL`` implies ``GFP_RECLAIM``, which means that > > > + direct reclaim may be triggered under memory pressure; the calling > > > + context must be allowed to sleep. > > > + * If the allocation is performed from an atomic context, e.g > > > + interrupt handler, use ``GFP_ATOMIC``. > > GFP_NOWAIT please. GFP_ATOMIC should be only used if accessing memory > reserves is justified. E.g. fallback allocation would be too costly. It > should be also noted that these allocation are quite likely to fail > especially under memory pressure. How about: * If the allocation is performed from an atomic context, e.g interrupt handler, use ``GFP_NOWARN``. This flag prevents direct reclaim and IO or filesystem operations. Consequently, under memory pressure ``GFP_NOWARN`` allocation is likely to fail. * If you think that accessing memory reserves is justified and the kernel will be stressed unless allocation succeeds, you may use ``GFP_ATOMIC``. > > > + * Untrusted allocations triggered from userspace should be a subject > > > + of kmem accounting and must have ``__GFP_ACCOUNT`` bit set. There > > > + is handy ``GFP_KERNEL_ACCOUNT`` shortcut for ``GFP_KERNEL`` > > > + allocations that should be accounted. > > > + * Userspace allocations should use either of the ``GFP_USER``, > > > + ``GFP_HIGHUSER`` and ``GFP_HIGHUSER_MOVABLE`` flags. The longer > > > + the flag name the less restrictive it is. > > > + > > > + The ``GFP_HIGHUSER_MOVABLE`` does not require that allocated > > > + memory will be directly accessible by the kernel or the hardware > > > + and implies that the data may move. > > @may move@is movable@ Ok > > > + The ``GFP_HIGHUSER`` means that the allocated memory is not > > > + movable, but it is not required to be directly accessible by the > > > + kernel or the hardware. An example may be a hardware allocation > > > + that maps data directly into userspace but has no addressing > > > + limitations. > > > + > > > + The ``GFP_USER`` means that the allocated memory is not movable > > > + and it must be directly accessible by the kernel or the > > > + hardware. It is typically used by hardware for buffers that are > > > + mapped to userspace (e.g. graphics) that hardware still must DMA > > > + to. > > > + > > > +You may notice that quite a few allocations in the existing code > > > +specify ``GFP_NOIO`` and ``GFP_NOFS``. Historically, they were used to > > > +prevent recursion deadlocks caused by direct memory reclaim calling > > > +back into the FS or IO paths and blocking on already held > > > +resources. Since 4.12 the preferred way to address this issue is to > > > +use new scope APIs described in > > > +:ref:`Documentation/core-api/gfp_mask-from-fs-io.rst <gfp_mask_from_fs_io>`. > > > + > > > +Another legacy GFP flags are ``GFP_DMA`` and ``GFP_DMA32``. They are > > > +used to ensure that the allocated memory is accessible by hardware > > > +with limited addressing capabilities. So unless you are writing a > > > +driver for a device with such restrictions, avoid using these flags. > > And even with HW with restrictions it is preferable to use dma_alloc* > APIs Will add. > Looks nice otherwise. Thanks! With the above changes feel free to add > Acked-by: Michal Hocko <mhocko@xxxxxxxx> Thanks! > -- > Michal Hocko > SUSE Labs > -- Sincerely yours, Mike.
