Rich Megginson wrote: > Tamas Bagyal wrote: >> Rich Megginson wrote: >>> Bagyal Tamas wrote: >>>> Rich Megginson wrote: >>>>> Tamas Bagyal wrote: >>>>>> hello Ryan, >>>>>> >>>>>> you tried this version? i have two fedora-ds 1.0.4 in mmr >>>>>> configuration. i migrate one of those to 1.1 (builded by your and >>>>>> Rich's instrutctions). but i have a problem with memory usage of >>>>>> ns-slapd process. initially mem usage is 18.5% but after 2 hours >>>>>> this changed to 23.1% and growed until killed by kernel. (i think...) >>>>>> >>>>>> mostly read transactions happen (dns) with a few write (cups). >>>>>> this is a debian etch, mem size is 512 mbyte (i know this is too >>>>>> low, but this is a test environment). cache size of slapd is >>>>>> 67108864. >>>>> Are you using SSL? Anything interesting in your server error log? >>>> >>>> I running the setupssl2.sh but not use any ssl connection. error log >>>> shows nothing, only the server start. >>> The reason I ask is that older versions of the NSS crypto/SSL >>> libraries had a memory leak. NSS 3.11.7 does not have this problem. >>> But you would only see the problem if you were using SSL connections. >> >> ok. I tried again from begining. fresh install, no ssl, no migration, >> used the setup-ds-admi.pl and setup the mmr with a fedora-ds 1.0.4. >> but nothing changed, memory usage growing... >> All setting is default except the mmr/changelog and access.log is off. >> >> errors: >> >> Fedora-Directory/1.1.0 B2008.059.1017 >> tower.fmintra.hu:389 (/opt/dirsrv/etc/dirsrv/slapd-tower) >> >> >> [05/Mar/2008:10:19:20 +0100] - dblayer_instance_start: pagesize: 4096, >> pages: 128798, procpages: 5983 >> [05/Mar/2008:10:19:20 +0100] - cache autosizing: import cache: 204800k >> [05/Mar/2008:10:19:21 +0100] - li_import_cache_autosize: 50, >> import_pages: 51200, pagesize: 4096 >> [05/Mar/2008:10:19:21 +0100] - WARNING: Import is running with >> nsslapd-db-private-import-mem on; No other process is allowed to >> access the database >> [05/Mar/2008:10:19:21 +0100] - dblayer_instance_start: pagesize: 4096, >> pages: 128798, procpages: 5983 >> [05/Mar/2008:10:19:21 +0100] - cache autosizing: import cache: 204800k >> [05/Mar/2008:10:19:21 +0100] - li_import_cache_autosize: 50, >> import_pages: 51200, pagesize: 4096 >> [05/Mar/2008:10:19:21 +0100] - import userRoot: Beginning import job... >> [05/Mar/2008:10:19:21 +0100] - import userRoot: Index buffering >> enabled with bucket size 100 >> [05/Mar/2008:10:19:21 +0100] - import userRoot: Processing file >> "/tmp/ldifZHth0D.ldif" >> [05/Mar/2008:10:19:21 +0100] - import userRoot: Finished scanning file >> "/tmp/ldifZHth0D.ldif" (9 entries) >> [05/Mar/2008:10:19:21 +0100] - import userRoot: Workers finished; >> cleaning up... >> [05/Mar/2008:10:19:21 +0100] - import userRoot: Workers cleaned up. >> [05/Mar/2008:10:19:21 +0100] - import userRoot: Cleaning up producer >> thread... >> [05/Mar/2008:10:19:21 +0100] - import userRoot: Indexing complete. >> Post-processing... >> [05/Mar/2008:10:19:21 +0100] - import userRoot: Flushing caches... >> [05/Mar/2008:10:19:21 +0100] - import userRoot: Closing files... >> [05/Mar/2008:10:19:21 +0100] - All database threads now stopped >> [05/Mar/2008:10:19:21 +0100] - import userRoot: Import complete. >> Processed 9 entries in 0 seconds. (inf entries/sec) >> [05/Mar/2008:10:19:22 +0100] - Fedora-Directory/1.1.0 B2008.059.1017 >> starting up >> [05/Mar/2008:10:19:22 +0100] - I'm resizing my cache now...cache was >> 209715200 and is now 8000000 >> [05/Mar/2008:10:19:22 +0100] - slapd started. Listening on All >> Interfaces port 389 for LDAP requests >> [05/Mar/2008:10:22:23 +0100] NSMMReplicationPlugin - changelog program >> - cl5Open: failed to open changelog >> [05/Mar/2008:10:22:24 +0100] NSMMReplicationPlugin - changelog program >> - changelog5_config_add: failed to start changelog >> [05/Mar/2008:10:26:49 +0100] NSMMReplicationPlugin - agmt="cn=replica >> to backup" (backup:389): Replica has a different generation ID than >> the local data. >> [05/Mar/2008:10:32:00 +0100] NSMMReplicationPlugin - >> repl_set_mtn_referrals: could not set referrals for replica >> dc=fmintra,dc=hu: 32 >> [05/Mar/2008:10:32:00 +0100] NSMMReplicationPlugin - >> multimaster_be_state_change: replica dc=fmintra,dc=hu is going >> offline; disabling replication >> [05/Mar/2008:10:32:00 +0100] - WARNING: Import is running with >> nsslapd-db-private-import-mem on; No other process is allowed to >> access the database >> [05/Mar/2008:10:32:13 +0100] - import userRoot: Workers finished; >> cleaning up... >> [05/Mar/2008:10:32:13 +0100] - import userRoot: Workers cleaned up. >> [05/Mar/2008:10:32:13 +0100] - import userRoot: Indexing complete. >> Post-processing... >> [05/Mar/2008:10:32:13 +0100] - import userRoot: Flushing caches... >> [05/Mar/2008:10:32:13 +0100] - import userRoot: Closing files... >> [05/Mar/2008:10:32:14 +0100] - import userRoot: Import complete. >> Processed 12242 entries in 13 seconds. (941.69 entries/sec) >> [05/Mar/2008:10:32:14 +0100] NSMMReplicationPlugin - >> multimaster_be_state_change: replica dc=fmintra,dc=hu is coming >> online; enabling replication >> >> memory usage by top: >> >> top - 10:58:21 up 25 days, 22:36, 2 users, load average: 0.01, 0.13, >> 0.22 >> Tasks: 61 total, 2 running, 59 sleeping, 0 stopped, 0 zombie >> Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, >> 0.0%si, 0.0%st >> Mem: 515192k total, 189600k used, 325592k free, 36472k buffers >> Swap: 489848k total, 18292k used, 471556k free, 106188k cached >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >> 27647 fds 15 0 464m 47m 25m S 0.0 9.4 1:34.57 ns-slapd >> >> >> top - 11:23:12 up 25 days, 23:01, 2 users, load average: 0.36, 0.27, >> 0.20 >> Tasks: 61 total, 2 running, 59 sleeping, 0 stopped, 0 zombie >> Cpu(s): 3.0%us, 0.0%sy, 0.0%ni, 96.0%id, 1.0%wa, 0.0%hi, >> 0.0%si, 0.0%st >> Mem: 515192k total, 210700k used, 304492k free, 36488k buffers >> Swap: 489848k total, 18288k used, 471560k free, 117204k cached >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >> 27647 fds 15 0 473m 59m 28m S 3.0 11.9 2:52.77 ns-slapd >> >> >> top - 11:48:26 up 25 days, 23:26, 2 users, load average: 0.02, 0.08, >> 0.10 >> Tasks: 61 total, 1 running, 60 sleeping, 0 stopped, 0 zombie >> Cpu(s): 3.0%us, 0.0%sy, 0.0%ni, 97.0%id, 0.0%wa, 0.0%hi, >> 0.0%si, 0.0%st >> Mem: 515192k total, 222756k used, 292436k free, 36520k buffers >> Swap: 489848k total, 18288k used, 471560k free, 118932k cached >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >> 27647 fds 15 0 483m 72m 30m S 0.0 14.4 4:12.04 ns-slapd >> >> >> top - 13:31:42 up 26 days, 1:09, 2 users, load average: 0.28, 0.17, >> 0.15 >> Tasks: 61 total, 2 running, 59 sleeping, 0 stopped, 0 zombie >> Cpu(s): 1.1%us, 0.0%sy, 0.0%ni, 98.9%id, 0.0%wa, 0.0%hi, >> 0.0%si, 0.0%st >> Mem: 515192k total, 285572k used, 229620k free, 36540k buffers >> Swap: 489848k total, 18288k used, 471560k free, 140412k cached >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >> 27647 fds 15 0 523m 116m 34m S 0.0 23.3 9:35.65 ns-slapd > Can you post your dse.ldif to pastebin.com? Be sure to omit or obscure > any sensitive data first. I'd like to see what all of your cache > settings are. Normally the server will increase in memory usage until > the caches are full, then memory usage should level off. The speed at > which this occurs depends on usage. > http://www.pastebin.org/22477 i forget a thing. i use some custom schema (ldapdns, ibm... etc.) if this is changed anything. (but i think this is not relevant info) > When the kernel kills your server, how much memory is it using? Is > there anything in the server error log at around the time the kernel > kills it? > i'm not sure, but at the time use the maximum as possible (512ram + 512 swap available) i think around 940mb, the kernel first kill some other processes, like mc, and after these the ns-slapd. I can't see anything in the log file, just the server start. > Finally, if you are convinced that there is a real memory leak in the > server, would it be possible for you to run it under valgrind? Just > running it under valgrind for 30 minutes or so should reveal any memory > leaks in normal usage. http://www.pastebin.org/22484 I can't understand this output, I never used valgrind before. I hope used the right options for valgrind. >> >>>>>> >>>>>> can you give any help? >>>>>> >>>>>> thanks, >>>>>> >>>>>> KeeF >>>>>> >>>>>> Ryan Braun wrote: >>>>>>>>> A couple little bugs creeped up during the build. I think it >>>>>>>>> was during >>>>>>>>> the make install of ldapserver. One of the binaries (the first >>>>>>>>> one I >>>>>>>>> guess) was copied to /opt/dirsrv/bin (the bin being a file not a >>>>>>>>> directory) so the /opt/dirsrv/bin directory isn't getting >>>>>>>>> created. Quick >>>>>>>>> fix was just renaming /opt/dirsrv/bin to >>>>>>>>> /opt/dirsrv/bin.something and >>>>>>>>> rerunning make. Executing /opt/dirsrv/bin.something looks like >>>>>>>>> the binary >>>>>>>>> might be ldappasswd? >>>>>>>> Probably a bug in ds/mozldap/Makefile in the install section. >>>>>>> >>>>>>> I had a peek in there, it looks ok, but I'll add a mkdir -p >>>>>>> /opt/dirsrv/bin before the copy loop and see if that works next >>>>>>> time I build. >>>>>>>>> Second, there seems to be a missing library. >>>>>>>>> >>>>>>>>> Starting admin server . . . >>>>>>>>> output: ERROR: ld.so: object '/opt/dirsrv/lib/libssl3.so' from >>>>>>>>> LD_PRELOAD >>>>>>>>> cannot be preloaded: ignored. >>>>>>>>> output: apache2: Syntax error on line 123 >>>>>>>>> of /opt/dirsrv/etc/dirsrv/admin-serv/httpd.conf: module >>>>>>>>> log_config_module >>>>>>>>> is built-in and can't be loaded >>>>>>>>> Could not start the admin server. Error: 256 >>>>>>>>> Failed to create and configure the admin server >>>>>>>>> Exiting . . . >>>>>>>>> >>>>>>>>> I assumed the libssl3.so was supposed to be provided by >>>>>>>>> building nss from >>>>>>>>> source. So I just symlinked the system's libssl3.so provided by >>>>>>>>> libnss3-0d back to /opt/dirsrv/lib/. >>>>>>>> Ok. Or just edit the start-ds-admin script. Looks like a bug - it >>>>>>>> should use the correct path to libssl3.so. But then the NSS devel >>>>>>>> support in etch is not quite there. >>>>>>> >>>>>>> Gotcha >>>>>>> >>>>>>>>> Which leads me to my next question. The java components, are >>>>>>>>> they only >>>>>>>>> required for running the console on your client machines? So >>>>>>>>> building >>>>>>>>> with NOJAVA=1 will provide a fully working adminserver and >>>>>>>>> ldapserver, just no console binaries? >>>>>>>> Mostly correct. The only thing is that the way the console >>>>>>>> works, it >>>>>>>> downloads the ds and ds-admin jar files from the admin server. >>>>>>>> However, >>>>>>>> if you build them on the client machine and install them into >>>>>>>> $HOME/.fedora-idm-console/jars then the console will just use >>>>>>>> the local >>>>>>>> ones. >>>>>>> >>>>>>> Ok, well I tried installing the windows console on one of the >>>>>>> windows boxes around here (easier then downloading fc isos :) ), >>>>>>> fired up the console and am able to connect and it looks like it >>>>>>> wants to work, then it reports back that it can't find the >>>>>>> jars. So that being said, is there an easy way to use FC jars, >>>>>>> or do I need to build them for debian? (I have started trying to >>>>>>> build jss but am having some issues) >>>>>>> >>>>>>>>> To be honest, I haven't really looked into the different post >>>>>>>>> install >>>>>>>>> process' with 1.1.0 since 1.0.4 so the reason I could have missing >>>>>>>>> entries in the console could very well be my own fault :) >>>>>>>>> >>>>>>>>> Also, if I want to fine tune the location of some of >>>>>>>>> directories during >>>>>>>>> build. is it safe to modify the CONFIGURE_ARGS variable in the >>>>>>>>> adminserver and ldapserver's Makefile? I want to put >>>>>>>>> /opt/dirsrv/etc/dirsrv into /etc/dirsrv aswell as >>>>>>>>> /opt/dirsrv/var into >>>>>>>>> /var? >>>>>>>> Yes, for those components whose configure respect --sysconfdir and >>>>>>>> --localstatedir - which means not the mozilla components >>>>>>>> (mozldap, etc.) >>>>>>>> but everything else should work just fine. You'll also have to >>>>>>>> tweak >>>>>>>> the --prefix argument which is set by default. >>>>>>> >>>>>>> I'll play around with some options. I've started a wiki page for >>>>>>> the debian build. I don't have it linked onto the main page, >>>>>>> but you can check it out in recent changes. >>>>>>> >>>>>>> Ryan >>>>>>> >> >> >> -- >> Fedora-directory-users mailing list >> Fedora-directory-users at redhat.com >> https://www.redhat.com/mailman/listinfo/fedora-directory-users > > > ------------------------------------------------------------------------ > > -- > Fedora-directory-users mailing list > Fedora-directory-users at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-directory-users
