On 01/23/2017 08:34 PM, NeilBrown wrote:
On Tue, Jan 24 2017, Theodore Ts'o wrote:
On Sun, Jan 22, 2017 at 10:05:44PM -0800, Matthew Wilcox wrote:
I don't have a clear picture in my mind of when Java promotes objects
from nursery to tenure
It's typically on the order of minutes. :-)
... which is not too different from my lack of
understanding of what the MM layer considers "temporary" :-) Is it
acceptable usage to allocate a SCSI command (guaranteed to be freed
within 30 seconds) from the temporary area? Or should it only be used
for allocations where the thread of control is not going to sleep between
allocation and freeing?
What the mm folks have said is that it's to prevent fragmentation. If
that's the optimization, whether or not you the process is allocating
the memory sleeps for a few hundred milliseconds, or even seconds, is
really in the noise compared with the average lifetime of an inode in
the inode cache, or a page in the page cache....
Why do you think it matters whether or not we sleep? I've not heard
any explanation for the assumption for why this might be important.
Because "TEMPORARY" implies a limit to the amount of time, and sleeping
is the thing that causes a process to take a large amount of time. It
seems like an obvious connection to me.
There's no simple connection to time, it depends on the larger picture - what's
the state of the allocator and what other allocations/free's are happening
around this one. Perhaps let me try to explain what the flag does and what
benefits are expected.
GFP_TEMPORARY, compared to GFP_KERNEL, adds __GFP_RECLAIMABLE, which tries to
place the allocation within MIGRATE_RECLAIMABLE pageblocks - GFP_KERNEL implies
MIGRATE_UNMOVABLE pageblocks, and userspace allocations are typically
MIGRATE_MOVABLE. The main goal of this "mobility grouping" is to prevent the
unmovable pages spreading all over the memory, making it impossible to get
larger blocks by defragmentation (compaction). Ideally we would have all these
problematic pages fit neatly into the smallest possible number of pageblocks
that can accomodate them. But we can't know in advance how many, and we don't
know their lifetimes, so there are various heuristics for relabeling pageblocks
between the 3 types as we exceed the existing ones.
Now GFP_TEMPORARY means we tell the allocator about the relatively shorter
lifetime, so it places the allocation within the RECLAIMABLE pageblocks, which
are also used for slab caches that have shrinkers. The expected benefit of this
is that we potentially prevent growing the number of UNMOVABLE pageblocks
(either directly by this allocation, or a subsequent GFP_KERNEL one, that would
otherwise fit within the existing pageblocks). While the RECLAIMABLE pages also
cannot be defragmented (at least currently, there are some proposals for the
slab caches...), we can at least shrink them, so the negative impact on
compaction is considered less severe in the longer term.
Imagine I want to allocate a large contiguous region in the
ZONE_MOVEABLE region. I find a mostly free region, so I just need to
move those last few pages. If there is a limit on how long a process
can sleep while holding an allocation from ZONE_MOVEABLE, then I know
how long, at most, I need to wait before those pages become either free
or movable. If those processes can wait indefinitely, then I might have
to wait indefinitely to get this large region.
Yeah so this is not relevant, because GFP_TEMPORARY does not make the allocation
__GFP_MOVABLE, so it still is not allowed to end up within a ZONE_MOVABLE zone.
Unfortunately the issue similar to that you mention does still exist due to
uncontrolled pinning of the movable pages, which affects both ZONE_MOVABLE and
CMA, but that's another story...
"temporary" doesn't mean anything without a well defined time limit.
But maybe I completely misunderstand.
HTH,
Vlastimil
NeilBrown
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
