"Resource temporarily unavailable", systemd and your FAQ Webpage

[Nic Bernstein]

Stephan,
Thank you for this very interesting description of both the problem and 
the solution.  I'm curious which Linux distribution you're using?  We're 
using Ubuntu Xenial (16.04) and find the following default settings:

    $ grep -i tasks /etc/systemd/system.conf
    #DefaultTasksAccounting=no
    #DefaultTasksMax=

so this would seem not to be an issue on Ubuntu systems, at least.

Cheers,
     -nic

On 12/16/2016 02:54 AM, Stephan Lauffer via Info-cyrus wrote:
> Hello!
>
> While testing 2.5 we struggled over (known) problems like this...:
>
>   ...ssh can't fork process to run service imaps/ipv4: Resource 
> temporarily unavailable
>
> I searchd... and I found your FAQ...
>
>   https://cyrusimap.org/imap/faqs/o-toomanyprocesses.html
>
> ...and this "tcp_keepalive = 1" hint was doing a good job, but not 
> fixing our problem completely. Less processes as without this setting 
> but too many at all.
>
> Btw we have a murder with 5 backends and each has about 5k mboxes. 
> With 2.4 we have about 400 imapd processes on the backends (in normal 
> state) and about 800 on the proxy. Peaks raises this values for sure.
>
> But we still wasn't able to get over a limit of about 500 imapds on 
> our new system which is running 2.5. Yesterday night I found our problem:
>
>   SYSTEMD
>
> (and not cyrus-imapd-2.5, but 2.5 was running with systemd here...)
>
> I guess there may be more people out there which may be interested in 
> a solition (we will add fixes to our systemd scripts of our suse bulds 
> soon, see 
> https://build.opensuse.org/package/show/home:nixda:devel/cyrus-imapd).
>
> The first problem there is:
>
> Systemd does not care about your /etc/security/limits.conf
>
> The second problem is:
>
> Systemd comes with a new limit called "TasksMax". You will find this 
> limit f.e. in /etc/systemd/system.conf as "DefaultTasksMax=512". But 
> you will NOT find this limit in "cat /proc/`cat 
> /var/run/cyrus.pid`/limits".
>
> The solution is to add a file with a systemd override for this 
> service. F.e. use the systemctl edit <sevice> command... (like 
> "systemctl edit cyrus-imapd.service") or place a file in 
> /etc/systemd/system/cyrus-imapd.service.d/override.conf with f.e.:
>
> [Service]
> TasksMax=2048
> LimitNOFILE=10000
>
> The LimitNOFILE is just a guess... maybe you may not need this here.
>
> After setting this override you will see a status information about 
> your process count and limit like this:
>
>  # systemctl status cyrus-imapd.service
> ● cyrus-imapd.service - The Cyrus IMAP and POP Mail Server
>    Loaded: loaded (/usr/lib/systemd/system/cyrus-imapd.service; 
> enabled; vendor preset: disabled)
>   Drop-In: /etc/systemd/system/cyrus-imapd.service.d
>            └─override.conf
>    Active: active (running) since Thu 2016-12-15 20:51:17 CET; 12h ago
>  Main PID: 12901 (master)
>     Tasks: 326 (limit: 2048)
> [...]
>
>
> What about adding a short hint to your faq? Yes, it is not a cyrus 
> problem. And yes, a sysadmin should know... but... I guess systemd in 
> depth may not be good known as the old ulimit thing and if you are 
> searching in the web 99% of the hits deal with ulimit.
>
>
>
>
> ----
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-- 
Nic Bernstein                             nic at onlight.com
Onlight, Inc.                             www.onlight.com
6525 W Bluemound Road, Suite 24           v. 414.272.4477
Milwaukee, Wisconsin  53213-4073

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20161216/9fdb5f32/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nic.vcf
Type: text/x-vcard
Size: 278 bytes
Desc: not available
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20161216/9fdb5f32/attachment.vcf>