Re: Re: Re: Implementation of Virtual Memory on Linux

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi...

> What I could conclude from your reply is that if there is any mmaped
> file, it will be resynced and then the page will be discarded. What
> if the dirty page is a page containing data section from code? Is OOM
> inevitable in this scenario?

Data section....ok...let me think, you mean, the data that originally 
resides on heap or user mode stack? This is anynomous page AFAIK, thus 
in no swap scenario, it will be kept on RAM. 

> The problem that I am facing is not consistent. Out of some 30 runs
> of few of the applications, the OOM occurs, though there is request
> for just one page. Why the behavior is not consistent?

Can you explain in detail, in those 30 runs, what kind of result did you 
get, how the application did memory allocation, the output of 
/proc/buddyinfo at that moment and other valuable informations? Armed 
with these data, perhaps we can reveal the reason of that 
"inconsistency"....

> Does it mean a malloced page? And freeing is done tho' free()?

This is purely "my speculation":
IIRC, malloc() is just extending heap area. So, if you do malloc, 
actually you just extend heap VMA. The page itself, as usual, is only 
allocated when you write/read to it (demand paging).

As for free(), this is actually not marking the page as "free" inside 
the Linux memory management. free() in glibc means "returning the 
allocated RAM to the pool of free memory". Thus, there is a chance, the 
page itself ( i mean the page_struct of the related page) might still 
be marked as used. 

The conclusion, to make sure the page is really freed or allocated, 
referencing directly to page_struct and the related page table entry 
(PTE) are the best ways to do

> What role is played by overcommit_memory in /proc/sys/vm in such
> scenarios?

Well, it determines how it will handle overcommit aka allocating pages 
exceeding the real available space (RAM+swaps). Remember that by 
default, Linux MM tends to be "optimistic" when doing memory 
allocation. It will even accept such a big request (exceeding available 
RAM+swap), but in the end it depends on how much RAM the application 
really use. 

overcommit_memory tunable help us to decide which one we really need. Do 
we want to loosely control overcommit? Do we want to be strict? Do we 
want to do some heuristics?

refer to Documentation/vm/overcommit_accounting for complete explanation 
of various options. IMHO, option #2 (aka don't overcommit) is the best 
way to prevent OOM in most cases. After all, if the application ask for 
something bigger than RAM+swap, why allow it at first place, right? Of 
course, once again we can't assume that all the allocated memory will 
be used, but there is a tendency that if someone ask N 
megabyte/kilobyte, eventually, it will eat all those N. So, sooner or 
later, OOM will bite.

As always, somebody CMIIW...

regards,

Mulyadi

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive:       http://mail.nl.linux.org/kernelnewbies/
FAQ:           http://kernelnewbies.org/faq/


[Index of Archives]     [Newbies FAQ]     [Linux Kernel Mentors]     [Linux Kernel Development]     [IETF Annouce]     [Git]     [Networking]     [Security]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux RAID]     [Linux SCSI]     [Linux ACPI]
  Powered by Linux