2.4.17 --> 2.5.3

Sat Nov 21 05:54:43 EST 2015

Ellie sent this helpful email (see below) a couple of months ago, and at 
the time I did not have an answer to this question:

 > At this point, I believe master is still running as root (it doesn't
 > become_cyrus() until a little while later), so it's curious that the
 > request is being rejected.  I would have assumed processes running as
 > root have the capability to do this...
...
 > If you hadn't seen this before on 2.4.17, maybe it was because you
 > were running an older kernel?  Capabilities seem to have appeared
 > in Linux around 2.2 but weren't quite complete until around 2.6.24

The answer, I'm pretty sure, is related to the conversion from the SysV 
Init system to systemd.  In particular, when processes are started by 
systemd they never run as root.  This would explain why it's suddenly 
impossible to reset the resource limits.

I'm an Arch user and have been using systemd for a while.  Red Hat / 
CentOS only fully switched to systemd with release 7, so I'm finding 
that a lot of my RHEL admin colleagues are still out of the loop on 
systemd.  It's more or less great (post learning curve), but be prepared 
for some bumps in the road.  The Red Hat 7 System Administrator's Guide 
(https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System_Administrators_Guide/index.html) 
has a very nice chapter on systemd.

On 9/20/2015 8:27 PM, ellie timoney wrote:
> Hi Patrick,
>
>> Second, where is this coming from?
>>
>>     Sep 19 05:44:54 toad cyrus/master[22860]: setrlimit: Unable to set
>> file descriptors limit to -1: Operation not permitted
>
> This one is interesting.
>
> Resource limits have a "current" and a "maximum" value.  Processes with
> suitable capabilities(7) privileges can set their current and maximum
> values to whatever they want.  Processes without may change their
> current value, but only if it doesn't exceed their maximum; and may
> decrease their maximum value, but never increase it (even if they have
> previously decreased it).
>
> The master process is trying to set its own file descriptor limit
> (RLIMIT_NOFILE -- I believe this is to be read as "Number of Open
> files"), both "current" and "maximum", to RLIM_INFINITY.  The request is
> being rejected though, so it tries again but using the existing maximum
> value (4096) instead, and that succeeds.
>
> At this point, I believe master is still running as root (it doesn't
> become_cyrus() until a little while later), so it's curious that the
> request is being rejected.  I would have assumed processes running as
> root have the capability to do this...
>
> The capability in question for setting resource limits is called
> CAP_SYS_RESOURCE.  Looking around online, there's a few threads on
> stackoverflow, serverfault et al asking how to assign this capability to
> processes running as root, so we're not the only ones finding it
> unexpectedly denied.  Looks like it can be set manually on a file by
> file basis using the setcap(8) command, so perhaps setting it for your
> master binary would tidy this up.  I have no idea what other
> ramifications this might have, so it's at your own risk if you do this.
>
> If you hadn't seen this before on 2.4.17, maybe it was because you were
> running an older kernel?  Capabilities seem to have appeared in Linux
> around 2.2 but weren't quite complete until around 2.6.24, if I'm
> reading capabilities(7) correctly.  This code path hasn't changed in
> cyrus between 2.4.17 and 2.5.6 and looks like it's been fairly static
> since 2002....
>
> I notice that our code for increasing the current and maximum value of
> this limit tries to set both to RLIM_INFINITY in the same setrlimit
> call.  I'm wondering if we need to actually set the maximum to
> RLIM_INFINITY with one call, and then set the current in a subsequent
> call.  I'll experiment with this a bit and see what happens.
>
> For what it's worth, I don't think it's cause for concern, but it's
> pinging me too so I'll have a little rummage and see what shakes out.
>
> Cheers,
>
> ellie
>
> On Sat, Sep 19, 2015, at 10:00 PM, Patrick Goetz wrote:
>> I recall Bron telling us that the upgrade from 2.4.x to 2.5.x would be
>> completely painless.  That was mostly, but not completely true.
>>
>> A bunch of variable names in /etc/cyrus/imapd.conf changed (OK, that was
>> easy to fix), and the upgrade did mostly work out of the box.  There was
>> one issue, however, and there are some new and improved error messages
>> in the logs that I'm sufficiently OCD to have questions about.
>>
>>
>> One one of my installs, I had this line in /etc/cyrus.conf:
>>
>>     squatter_a  cmd="/usr/sbin/squatter" at=0517
>>
>>
>> This worked for 2.4.17, but caused cyrus-master to fail to start with
>> this error message:
>>
>>     Sep 19 05:27:58 www cyrus/master[29646]: configuration file
>> /etc/cyrus/cyrus.conf: bad character '_' in name on line 57
>>     Sep 19 05:27:58 www systemd[1]: cyrus-master.service: Main process
>> exited, code=exited, status=78/n/a
>>
>> For the time being, I just commented out the squatter line. I'm unclear
>> on how necessary it is to re-index the mailboxes every day.  If
>> necessary/useful, did the syntax for this command change?
>>
>> Also, I'm now getting these warnings (maybe some were there for 2.4.17,
>> I can't remember):
>>
>> ---------------------------
>> Sep 19 05:44:54 toad systemd[1]: Starting Cyrus IMAP mail server...
>> Sep 19 05:44:54 toad cyrus/master[22860]: setrlimit: Unable to set file
>> descriptors limit to -1: Operation not permitted
>> Sep 19 05:44:54 toad cyrus/master[22860]: retrying with 4096 (current
>> max)
>> Sep 19 05:44:54 toad systemd[1]: Started Cyrus IMAP mail server.
>> Sep 19 05:44:54 toad cyrus/ctl_cyrusdb[22861]: skiplist: clean shutdown
>> file missing, updating recovery stamp
>> Sep 19 05:44:54 toad cyrus/ctl_cyrusdb[22861]: recovering cyrus databases
>> Sep 19 05:44:54 toad cyrus/ctl_cyrusdb[22861]: done recovering cyrus
>> databases
>> Sep 19 05:44:54 toad cyrus/master[22860]: unable to open imap/ipv6
>> socket: Address family not supported by protocol
>> Sep 19 05:44:54 toad cyrus/master[22860]: unable to open imaps/ipv6
>> socket: Address family not supported by protocol
>> Sep 19 05:44:54 toad cyrus/master[22860]: unable to open sieve/ipv6
>> socket: Address family not supported by protocol
>> Sep 19 05:44:54 toad cyrus/master[22860]: unable to setsocketopt(IP_TOS)
>> service lmtpunix/unix: Operation not supported
>> -----------------------------
>>
>> We're not using ipv6 -- is there any way to let cyrus know so that it
>> doesn't freak out?
>>
>>
>> Second, where is this coming from?
>>
>>     Sep 19 05:44:54 toad cyrus/master[22860]: setrlimit: Unable to set
>> file descriptors limit to -1: Operation not permitted
>>
>>
>> Finally, should I worry about this?
>>
>>     setsocketopt(IP_TOS) service lmtpunix/unix: Operation not supported
>>
>>
>> Thanks.
>>
>> ----
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>