Re: memory consumption

[Rich Megginson <rmeggins@xxxxxxxxxx>]

On 04/16/2012 11:59 AM, Russell Beall wrote:

Hi,

I have been working with 389 as a potential replacement for Sun DS and I have found it to be an excellent choice in every aspect except the final tests I have been running.

I am running with the current version for RedHat 6, but not the latest from the rmeggins repo:

Name        : 389-ds

Arch        : noarch

Version     : 1.2.2

Name        : 389-ds-base

Arch        : x86_64

Version     : 1.2.9.14

I have searched through all the release notes and part of the 389-users list archive for clues as to the possible memory leaks and/or patches in the latest releases, but no information has been forthcoming.

The behavior I am seeing is a total memory consumption occurring over a large quantity of ldapmodify operations.  To test this, I reduced the size of the directory from 10GB down to about 1.7GB.  Then I set up a loop that would run an ldapmodify that would delete of most of those entries followed by an ldamodify that would add all those deleted back in (and then repeat indefinitely).

The directory starts out by loading all the entries into the cache and using a few GB of ram to hold everything.  Eventually, this loop causes the entire 32GB of ram to be consumed even though the total size of the directory does not change (e.g. currententrycachesize: 1791096898).

Replication is enabled to a consumer and the change log is up to 7.4 GB, and it doesn't seem to want to clean out the change log even with short purge times and short entry lifetimes configured, so perhaps something is at issue here.

Can you post your purge/trimming configuration parameters?

The consumer has processed all updates and also seems to exhibit overconsumption of memory.

Do you see any issue if you don't use replication at all?   That is, is this issue related to replication?

Are there any pointers related to this?

Are you seeing https://fedorahosted.org/389/ticket/51 ?
When you start to see memory growth, are you using all of your cache?

If there is no information about this, is there a documentation page that might instruct me in the correct way to attach valgrind to the ns-slapd process so I can see if there is some kind of huge leak?

AFAIK you can't attach valgrind to a running process.
try this:

service dirsrv stop
( . /etc/sysconfig/dirsrv ; . /etc/sysconfig/dirsrv-INSTANCENAME ; valgrind -q  --tool=memcheck --leak-check=yes --leak-resolution=high --num-callers=50 --log-file=/var/log/dirsrv/slapd-INSTANCENAME/valgrind.log /usr/sbin/ns-slapd -D /etc/dirsrv/slapd-INSTANCENAME -i /var/run/dirsrv/slapd-INSTANCENAME.pid -w /var/run/dirsrv/slapd-INSTANCENAME.startpid -d 0 ) &

valgrind will log to /var/log/dirsrv/slapd-INSTANCENAME/valgrind.log

Note that running your server with valgrind will really cripple the performance - may be unusable in a production environment - you may also run afoul of selinux
valgrind will not report memory leaks until you shutdown the server (just kill -15 <pid of ns-slapd or valgrind>)

Thanks very much,

Russ.

==============================

Russell Beall
Programmer Analyst IV

Enterprise Identity Management

University of Southern California

beall@xxxxxxx

==============================

--
389 users mailing list
389-users@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/389-users

--
389 users mailing list
389-users@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/389-users