Cyrus, Solaris 10, ZFS? (and NIS?)

[Simon Matter]

>
>
> --On Thursday, October 05, 2006 04:13:18 PM -0400 Elizabeth Schwartz
> <betsy.schwartz at gmail.com> wrote:
>
>> http://www.gsd.harvard.edu/users/betsys/dapptrace.timed
>
> The interesting bit seems to be here:
>       .      . -> mynewstate(0x165769, 0x40404040, 0x0)
>       .      . -> mycanonifyid(0x165769, 0x0, 0x0)
>       .      . -> libcyrus_config_getswitch(0x2, 0x11AF55, 0x5)
>      43      5 <- libcyrus_config_getswitch = 84
>     135     20 <- mycanonifyid = 292
>       .      . -> xmalloc(0x5C, 0x11AF5D, 0x5)
>      38     14 <- xmalloc = 28
>       .      . -> libcyrus_config_getswitch(0x1, 0x0, 0x1647CB)
>      40      3 <- libcyrus_config_getswitch = 84
>       .      . -> xrealloc(0x0, 0x4, 0xE488)
>      68     14 <- xrealloc = 64
>       .      . -> xstrdup(0x16871C, 0x4, 0xE488)
>       .      . -> xmalloc(0x9, 0x2A0031, 0x168724)
>      30      7 <- xmalloc = 28
>      94     16 <- xstrdup = 40
> 8235260 109820 <- mynewstate = 356
>
> mynewstate is taking 8s to run, and very little of the time is taken up in
> local subroutines.
> auth_unix.c:mynewstate calls getpwnam, and then iterates over all the
> groups using getgrent(),
> checking to see what groups the user is in. The fact that imapd does this
> twice might be a bug, but even if it didn't do it twice, it would still be
> slow.
>
> Is running "getent group" slow?

This issue has been discussed several times here. Usually people using
LDAP groups with unix auth realize it first. One could think that running
nscd should solve the problem but it does not, at least not on Linux.
What I did was to implement a group cache feature which is very simple and
maybe that's why it work so well, at least for me :)
I post the readme below, more info can be found here
http://www.invoca.ch/pub/packages/cyrus-imapd/scripts/groupcache/

Regards,
Simon

------%<--------------------------------------
NOTE: This patch is included into the Invoca cyrus-imapd rpm package. It may
be useful for others too, that's why it is here.

USE AT YOUR OWN RISK!!!

If you have a lot of groups or very large groups, cyrus-imapd login can
become
quite slow due to the way cyrus-imapd handles groups. This may become worse
when using nss_ldap or other slow nss backends to resolve groups.
Caching using nscd can increase speed dramatically but unfortunately not for
all functions cyrus-imapd uses. nscd helps speeding up getgrnam() calls
but not
getgrent() calls, which are used by cyrus-imapd to get a complete list of all
groups available.
The groupfile patch implements a quick fix to the problem by using a separate
group.cache file to speed up those operations using getgrent() calls.
Calls to getgrnam() are not touched which means that the group.cache
file must be kept in sync with the group source you are using with nss
configured in /etc/nsswitch.conf. If group.cache doesn't exist,
the patch has no effect and cyrus-imapd handles groups through getgrent().

Quick HOWTO:
- configure group lookup in /etc/nsswitch.conf if not already done
- configure name service cache in /etc/nscd.conf               (not
mandatory)
- start nscd with "service nscd start"                         (not
mandatory)
- configure a service which periodically updates the group.cache file
using the
  upd_groupcache script. Either set up a cronjob or use a event in the cyrus
  configuration.

Sample /etc/cyrus.conf event to update the group cache every 10 minutes:
EVENTS {
  # some events removed here ***
  ...........................***

  # this is only necessary if using group cache feature
  groupcache    cmd="upd_groupcache" period=10
}
------%<--------------------------------------