"Resource temporarily unavailable", systemd and your FAQ Webpage

[Stephan Lauffer]

Hello!

While testing 2.5 we struggled over (known) problems like this...:

   ...ssh can't fork process to run service imaps/ipv4: Resource  
temporarily unavailable

I searchd... and I found your FAQ...

   https://cyrusimap.org/imap/faqs/o-toomanyprocesses.html

...and this "tcp_keepalive = 1" hint was doing a good job, but not  
fixing our problem completely. Less processes as without this setting  
but too many at all.

Btw we have a murder with 5 backends and each has about 5k mboxes.  
With 2.4 we have about 400 imapd processes on the backends (in normal  
state) and about 800 on the proxy. Peaks raises this values for sure.

But we still wasn't able to get over a limit of about 500 imapds on  
our new system which is running 2.5. Yesterday night I found our  
problem:

   SYSTEMD

(and not cyrus-imapd-2.5, but 2.5 was running with systemd here...)

I guess there may be more people out there which may be interested in  
a solition (we will add fixes to our systemd scripts of our suse bulds  
soon, see   
https://build.opensuse.org/package/show/home:nixda:devel/cyrus-imapd).

The first problem there is:

Systemd does not care about your /etc/security/limits.conf

The second problem is:

Systemd comes with a new limit called "TasksMax". You will find this  
limit f.e. in /etc/systemd/system.conf as "DefaultTasksMax=512". But  
you will NOT find this limit in "cat /proc/`cat  
/var/run/cyrus.pid`/limits".

The solution is to add a file with a systemd override for this  
service. F.e. use the systemctl edit <sevice> command... (like  
"systemctl edit cyrus-imapd.service") or place a file in  
/etc/systemd/system/cyrus-imapd.service.d/override.conf with f.e.:

[Service]
TasksMax=2048
LimitNOFILE=10000

The LimitNOFILE is just a guess... maybe you may not need this here.

After setting this override you will see a status information about  
your process count and limit like this:

  # systemctl status cyrus-imapd.service
● cyrus-imapd.service - The Cyrus IMAP and POP Mail Server
    Loaded: loaded (/usr/lib/systemd/system/cyrus-imapd.service;  
enabled; vendor preset: disabled)
   Drop-In: /etc/systemd/system/cyrus-imapd.service.d
            └─override.conf
    Active: active (running) since Thu 2016-12-15 20:51:17 CET; 12h ago
  Main PID: 12901 (master)
     Tasks: 326 (limit: 2048)
[...]


What about adding a short hint to your faq? Yes, it is not a cyrus  
problem. And yes, a sysadmin should know... but... I guess systemd in  
depth may not be good known as the old ulimit thing and if you are  
searching in the web 99% of the hits deal with ulimit.


-- 
Liebe Gruesse, with best regards
Stephan Lauffer

Pedagogical University Freiburg - Germany
http://www.ph-freiburg.de/zik/
Fon/ Fax: +49 761 682 -559/ -486
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5783 bytes
Desc: S/MIME-Signatur
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20161216/79092151/attachment.p7s>