Re: backend proccess memory accumulates

[Rui DeSousa <rui@xxxxxxxxxxxxx>]

On Nov 22, 2018, at 12:31 PM, Sebastian Gabbert <pg@xxxxxxxxxxxxxx> wrote:

Thanks for the reply, but I think i sorted this out, that was why i spoke of “RES”. As far as I understand memory stats shown by top or glances there are two columns, one for “virtual memory” (top->Size, glances->VIRT) and one for residential memory (RES in both cases). And my understanding is that virtual shows the memory including shared memory chunks, RES shows the actually used memory by this process alone.
I absolutely could be wrong in my interpretation - since I read at many places that theses distinctions are difficult.

I don’t think that is correct.  I look at them mean the following.

VIRT - is the total address space allocated.

RES - is the actual memory mapped to real memory (including shared memory).

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND

70596 postgres      1  89    0 49971M 23581M CPU41  41   5:51  56.88% postgres

65566 postgres      1  88    0 50005M 24384M CPU38  38   8:04  56.79% postgres

19650 postgres      1  89    0 50154M 48762M CPU67  67 281:16  55.86% postgres

As you can see from the above; process 19650 has more of the shared memory mapped than the other two processes.

Note: You can use "procstat -v” to look at the memory map of each process.

Bottom line is: My server swapes and pagedeamon is one of the busiest processes :) So RAM is used in vast amounts.

Not good; sounds like your memory is not correctly allocated and/or you have some very large/bad queries. 

Which file system are you using? If you’re using ZFS then it needs to be bound.  ZFS will try to use as much memory as possible which doesn’t work well for Postgres.

You want to adjust ZFS's arc_max and Postgres shared buffers to be less than the total memory while leaving enough memory for the OS and processes to allocate their own memory. I would recommend starting with arc_max + shared buffers being around 60-70% of memory.

Configure ZFS’s arch_max in /boot/loader.conf and reboot (adjust appropriately):

vfs.zfs.arc_max=“256G"