Re: [RFC/PATCH 0/5] Contiguous Memory Allocator and get_user_pages()

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello,

On 3/6/2013 9:47 AM, Minchan Kim wrote:
Hello,

On Tue, Mar 5, 2013 at 3:57 PM, Marek Szyprowski
<m.szyprowski@xxxxxxxxxxx> wrote:
> Hello,
>
> Contiguous Memory Allocator is very sensitive about migration failures
> of the individual pages. A single page, which causes permanent migration
> failure can break large conitguous allocations and cause the failure of
> a multimedia device driver.
>
> One of the known issues with migration of CMA pages are the problems of
> migrating the anonymous user pages, for which the others called
> get_user_pages(). This takes a reference to the given user pages to let
> kernel to operate directly on the page content. This is usually used for
> preventing swaping out the page contents and doing direct DMA to/from
> userspace.
>
> To solving this issue requires preventing locking of the pages, which
> are placed in CMA regions, for a long time. Our idea is to migrate
> anonymous page content before locking the page in get_user_pages(). This
> cannot be done automatically, as get_user_pages() interface is used very
> often for various operations, which usually last for a short period of
> time (like for example exec syscall). We have added a new flag
> indicating that the given get_user_space() call will grab pages for a
> long time, thus it is suitable to use the migration workaround in such
> cases.
>
> The proposed extensions is used by V4L2/VideoBuf2
> (drivers/media/v4l2-core/videobuf2-dma-contig.c), but that is not the
> only place which might benefit from it, like any driver which use DMA to
> userspace with get_user_pages(). This one is provided to demonstrate the
> use case.
>
> I would like to hear some comments on the presented approach. What do
> you think about it? Is there a chance to get such workaround merged at
> some point to mainline?
>

I discussed similar patch from memory-hotplug guys with Mel.
Look at http://marc.info/?l=linux-mm&m=136014458829566&w=2

The conern is that we ends up forcing using FOLL_DURABLE/GUP_NM for
all drivers and subsystems for making sure CMA/memory-hotplug works
well.

You mentioned driver grab a page for a long time should use
FOLL_DURABLE flag but "for a long time" is very ambiguous. For
example, there is a driver

get_user_pages()
some operation.
put_pages

You can make sure some operation is really fast always?

Well, in our case (judging from the logs) we observed 2 usage patterns
for get_user_pages() calls. One group was lots of short time locks, whose
call stacks originated in various kernel places, the second group was
device drivers which used get_user_pages() to create a buffer for the
DMA. Such buffers were used for the whole lifetime of the session to
the given device, what was equivalent to infinity from the migration/CMA
point of view. This was however based on the specific use case at out
target system, that's why I wanted to start the discussion and find
some generic approach.


For example, what if it depends on other event which is normally very
fast but quite slow once a week or try to do dynamic memory allocation
but memory pressure is severe?

For 100% working well, at last we need to change all GUP user with
GUP_NM or your FOLL_DURABLE whatever but the concern Mel pointed out
is it could cause lowmem exhaustion problem.

This way we sooner or later end up without any movable pages at all.
I assume that keeping some temporary references on movable/cma pages
must be allowed, because otherwise we limit the functionality too much.

At the moment, there is other problem against migratoin, which are not
related with your patch. ex, zcache, zram, zswap. Their pages couldn't
be migrated out so I think below Mel's suggestion or some generic
infrastructure can move pinned page is  more proper way to go.

zcache/zram/zswap (vsmalloc based code) can be also extended to support
migration. It requires some significant amount of work, but it is really
doable.

"To guarantee CMA can migrate pages pinned by drivers I think you need
migrate-related callsbacks to unpin, barrier the driver until migration
completes and repin."

Right, this might improve the migration reliability. Are there any works
being done in this direction?

Best regards
--
Marek Szyprowski
Samsung Poland R&D Center


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>


[Index of Archives]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [ECOS]     [Asterisk Internet PBX]     [Linux API]