Re: FileDescriptors exhausted

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



It sounds like you may have stale client connections. There are a lot
of bad LDAP clients that never close their connections properly.
the way to work around this is to set a TCP keepalive to a shorter
than default. here is a good article on the subject
https://www.looklinux.com/how-to-configure-tcp-keepalive-setting-in-linux/
i recommend that heartbeats be initiated after 60 seconds after the
connection becomes idle and only a few failures before they are
dropped.
Also it's important to know all 3 settings must be set for this to
work. If you don't set all 3 related settings or if any of them are
set to 0 it will not do anything.

an other thing is the open file limits they must be set in two places
first is explained here
https://access.redhat.com/solutions/1257953#:~:text=service.,descriptors%20is%20limited%20to%20500%2C000.&text=The%20change%20is%20applied%20after,configuration%20and%20restarting%20the%20service.
The other is in the dse.ldif file
https://docs.kolab.org/administrator-guide/tweaking-389ds.html#:~:text=A%20389%20Directory%20Server%20is,open%20file%20descriptors%20from%201024.

On Fri, Nov 11, 2022 at 11:37 PM Howard Chu <hyc@xxxxxxxxx> wrote:
>
> > Date: Fri, 11 Nov 2022 14:25:57 +0000
> > From: Tobias Ernstberger <tobias.ernstberger@xxxxxxxxxx>
>
> > Hello,
> >
> > we're observing the following error message:
> > "ERR - accept_and_configure - PR_Accept() failed, Netscape Portable Runtime error -5971 (Process open FD table is full.)"
> > Looks like the file descriptors are exhausted, probably mainly used by incoming TCP Connections (based on our investigation regarding open FDs).
> > We've set (and checked using the runtime information in /proc/PID/limits) the ulimits and the nsslapd-maxdescriptors to many thousands (while having about 1000 open connection regularly)
> >
> > We are investigating in multiple directions here, and have some questions - any input is appreciated:
>
> Have you checked with e.g. lsof to see where all the FDs are being used? If you think there's only 1000
> incoming connections, probably there are other things going on e.g. DNS reverse lookups on client
> addresses, miscellaneous other files being opened, etc.
> >
> > 1) We acknowledge that exhausted FDs prevent additional connections to be opened. But we also see, that existing connections are getting unusable, too. Is this a known behaviour? Can this be avoided?
> > 2) Is there any chance to limit the number of open connections (lower than the max FDs)? (trying to achieve that existing connections still work)
> > 3) What are best practice to prevent the ldap server from getting completely useless (until restart) if a client opens many connections?
> > 4) Any additional remarks to prevent this situation?
>
> Fyi, OpenLDAP has no issue with multiple thousands of FDs being served concurrently...
> >
> >
> > Kind regards
> >
> > Tobias Ernstberger
> > IBM Security
> >
> > IBM Deutschland GmbH
> > Vorsitzender des Aufsichtsrats: Sebastian Krause
> > Geschäftsführung: Gregor Pillen (Vorsitzender), Nicole Reimer, Gabriele Schwarenthorer, Christine Rupp, Frank Theisen
> > Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart, HRB 14562 / WEEE-Reg.-Nr. DE 99369940
> > https://www.ibm.com/privacy/us/en/
>
> --
>   -- Howard Chu
>   CTO, Symas Corp.           http://www.symas.com
>   Director, Highland Sun     http://highlandsun.com/hyc/
>   Chief Architect, OpenLDAP  http://www.openldap.org/project/
> _______________________________________________
> 389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
_______________________________________________
389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue




[Index of Archives]     [Fedora User Discussion]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [Fedora News]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Maintainers]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Legacy]     [Fedora Desktop]     [Fedora Fonts]     [ATA RAID]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Centos]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora QA]     [Fedora Triage]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Tux]     [Yosemite News]     [Yosemite Photos]     [Linux Apps]     [Maemo Users]     [Gnome Users]     [KDE Users]     [Fedora Tools]     [Fedora Art]     [Fedora Docs]     [Maemo Users]     [Asterisk PBX]     [Fedora Sparc]     [Fedora Universal Network Connector]     [Fedora ARM]

  Powered by Linux