Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, 28 Aug 2008, david@xxxxxxx wrote:

On Thu, 28 Aug 2008, Matthew Wakeling wrote:

On Wed, 27 Aug 2008, david@xxxxxxx wrote:
if memory overcommit is disabled, the kernel checks to see if you have an extra 1G of ram available, if you do it allows the process to continue, if you don't it tries to free memory (by throwing away cache, swapping to disk, etc), and if it can't free the memory will return a memroy allocation error (which I believe will cause firefox to exit).

Remember that the memory overcommit check is checking against the amount of RAM + swap you have - not just the amount of RAM. When a fork occurs, hardly any extra actual RAM is used (due to copy on write), but the potential is there for the process to use it. If overcommit is switched off, then you just need to make sure there is *plenty* of swap to convince the kernel that it can actually fulfil all of the memory requests if all the processes behave badly and all shared pages become unshared. Then the consequences of processes actually using that memory are that the machine will swap, rather than the OOM killer having to act.

if you are correct that it just checks against memory+swap then it's not a big deal, but I don't think it does that. I think it actually allocates the memory, and if it does that it will push things out of ram to do the allocation, I don't believe that it will allocate swap space directly.

I just asked on the kernel mailing list and Alan Cox responded.

he is saying that you are correct, it only allocates against the total available, it doesn't actually allocate ram.

so with sufficiant swap overcommit off should be fine.

but you do need to allocate more swap as the total memory 'used' can be significantly higher that with overcommit on.

David Lang

David Lang

Of course, it's generally bad to run a machine with more going on than will fit in RAM.

Neither swapping nor OOM killing are particularly good - it's just a consequence of the amount of memory needed being unpredictable.

Probably the best solution is to just tell the kernel somehow to never kill the postmaster.

Matthew





[Postgresql General]     [Postgresql PHP]     [PHP Users]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Yosemite]

  Powered by Linux