"Resource temporarily unavailable", systemd and your FAQ Webpage

Stephan Lauffer via Info-cyrus <info-cyrus@xxxxxxxxxxxxxxxxxxxx> · Fri, 16 Dec 2016 09:54:43 +0100

Hello!

While testing 2.5 we struggled over (known) problems like this...:

  ...ssh can't fork process to run service imaps/ipv4: Resource  
temporarily unavailable

I searchd... and I found your FAQ...

  https://cyrusimap.org/imap/faqs/o-toomanyprocesses.html

...and this "tcp_keepalive = 1" hint was doing a good job, but not  
fixing our problem completely. Less processes as without this setting  
but too many at all.

Btw we have a murder with 5 backends and each has about 5k mboxes.  
With 2.4 we have about 400 imapd processes on the backends (in normal  
state) and about 800 on the proxy. Peaks raises this values for sure.

But we still wasn't able to get over a limit of about 500 imapds on  
our new system which is running 2.5. Yesterday night I found our  
problem:

  SYSTEMD

(and not cyrus-imapd-2.5, but 2.5 was running with systemd here...)

I guess there may be more people out there which may be interested in  
a solition (we will add fixes to our systemd scripts of our suse bulds  
soon, see   
https://build.opensuse.org/package/show/home:nixda:devel/cyrus-imapd).

The first problem there is:

Systemd does not care about your /etc/security/limits.conf

The second problem is:

Systemd comes with a new limit called "TasksMax". You will find this  
limit f.e. in /etc/systemd/system.conf as "DefaultTasksMax=512". But  
you will NOT find this limit in "cat /proc/`cat  
/var/run/cyrus.pid`/limits".

The solution is to add a file with a systemd override for this  
service. F.e. use the systemctl edit <sevice> command... (like  
"systemctl edit cyrus-imapd.service") or place a file in  
/etc/systemd/system/cyrus-imapd.service.d/override.conf with f.e.:

[Service]
TasksMax=2048
LimitNOFILE=10000

The LimitNOFILE is just a guess... maybe you may not need this here.

After setting this override you will see a status information about  
your process count and limit like this:

 # systemctl status cyrus-imapd.service
● cyrus-imapd.service - The Cyrus IMAP and POP Mail Server
   Loaded: loaded (/usr/lib/systemd/system/cyrus-imapd.service;  
enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/cyrus-imapd.service.d
           └─override.conf
   Active: active (running) since Thu 2016-12-15 20:51:17 CET; 12h ago
 Main PID: 12901 (master)
    Tasks: 326 (limit: 2048)
[...]

What about adding a short hint to your faq? Yes, it is not a cyrus  
problem. And yes, a sysadmin should know... but... I guess systemd in  
depth may not be good known as the old ulimit thing and if you are  
searching in the web 99% of the hits deal with ulimit.

--
Liebe Gruesse, with best regards
Stephan Lauffer

Pedagogical University Freiburg - Germany
http://www.ph-freiburg.de/zik/
Fon/ Fax: +49 761 682 -559/ -486
Attachment:
smime.p7s

Description: S/MIME-Signatur
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus