On Tue, Aug 07, 2007 at 10:26:46AM -0600, Richard Megginson wrote: > Jonathan Barber wrote: > >Hello all, currently we have a FDS instance running on RHEL4 with a > >small number of entries (6,000), we also have a linux compute cluster of > >100 nodes which uses LDAP for user account data (via libnss_ldap). > > > >nss_ldap on the cluster is configured to use SSL, and everything is fine > >most of the time. However, occasionally, when a large job is started on > >the cluster, the number of connections increases from 100/minute to > >1600/minute (26/sec). > > > >This causes the server to become generally unresponsive, and FDS > >especially so (as judged by the time required to retrieve the DSE via > >TLS). Which is a right pain as it causes our samba PDC to timeout and > >everything goes wrong very quickly. > > > >I can reproducably, impact on FDS performance by running: > >$ getent passwd | cut -d: -f 1 | while read i; do id $i; done > > > >across the cluster. When SSL is off, the command to run fine and doesn't > >impact on other searches. > > > >As a short term measure, we've disabled LDAPS on the cluster nodes, > >which is fine as users don't log into them, but we had planned to expand > >the use of LDAP to cover more hosts (Macs and Linux) that require a > >confidential channal for authentication. So this experience is giving us > >some trepidation about moving forward with that plan. > > > >Our system is configured following the guidance of the wiki [0], with a > >maximum of 16834 available file descriptors and 50M of cache (more than > >enough to hold the DB) - and the ratio of cache hits/misses look good > >with little paging out. Running logconv.pl on the access logs doesn't > >show any unindexed searches, so that isn't an issue. > > > >Our server CPU is a 3Ghz Xeon with 1G of RAM, and looking at the > >performance of NSS 3.2 [1], I would expect the machine to be able to > >setup and tear down many more connections than we are currently seeing. > >Indeed, running the test described in [1] with the nss-3.11.4 binaries, > >I get over 1200 connections per second [2], so it certainly doesn't seem > >to be a problem with NSS. > > > >This suggests to me that the problem lies in FDS somewhere. So, does > >anyone have any suggestions as to how to improve the SSL/TLS performance > >of FDS, or point me at tuning docs for the SSL side of FDS? > > > > I don't know. But opening and closing SSL connections is pretty > expensive, with all of the TLS/SSL protocol operations. Is it possible > you could configure the client machines to use LDAP (not LDAPS) and use > the LDAP startTLS operation to start up the TLS session on the > non-secure port? This might allow the server to process the connection > + TLS session creation more efficiently. I'll give it a go and see how it works. I had assumed SSL would be less expensive than a start TLS. Do you have any benchmarks (even rough numbers) available as to how many connections FDS can copes with TLS/SSL vs. plain LDAP? I've read Howard Chu's presentation (http://highlandsun.com/hyc/SambaXP.pdf) but it doesn't compare against SSL, and I didn't do any SSL benchmarks with FDS when I evaluted LDAP servers. I don't have any real feeling as to how many TLS/SSL connection you get compared to plain TCP/IP. Ta. > >Cheers. > > > >[0] http://directory.fedoraproject.org/wiki/Performance_Tuning > >[1] > >http://www.mozilla.org/projects/security/pki/nss/nss-3.2-performance-results > >[2] server$ ./selfserv -n "Server-Cert" -p 6000 > > client$ time ./strsclnt -p 6000 server -c 1000 > > strsclnt: -- SSL: Server Certificate Validated. > > strsclnt: 0 cache hits; 1 cache misses, 0 cache not reusable > > strsclnt: 999 cache hits; 1 cache misses, 0 cache not reusable > > > > real 0m0.605s > > user 0m0.795s > > sys 0m0.226s > > > > -- > Fedora-directory-users mailing list > Fedora-directory-users@xxxxxxxxxx > https://www.redhat.com/mailman/listinfo/fedora-directory-users -- Jonathan Barber High Performance Computing Analyst Tel. +44 (0) 1382 386389 -- Fedora-directory-users mailing list Fedora-directory-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/fedora-directory-users
