Terrible performance and Upgrade

[Matthew Baker]

Hi all,

	I recently started using Cyrus Imap 2.1.15 on a system with 5000 mail 
accounts (500 actively used on a daily basis). I have setup Mysql for 
virtual alias lookups for a Postfix mta and also for authentication via 
saslauthd. I initially had major issues with DB errors and general 
instability so I upgraded to Cyrus 2.2.3 and followed procedures to 
convert the Berkeley db formatted mailbox.db and .seen files. However I 
am still having issues with performance, resources and db problems.

Problem 1.
Saslauthd gradually grows in memory size until the login process grinds 
to a halt. I have set it to restart once an hour to clear it. I have 
tried setting the option -n0 to force a child for each auth request but 
that seem to create numerous connections to the database which would 
exceed the max connections variable. I now am thinking that to use 
auxprop mysql plugin might be a better route. I currently have 183 
mysqld processes running, this can't be right.

Problem 2.
Database formats for deliver.db is still Berkeley. I am also getting a 
huge amount of errors in imapd log like this:

Feb  5 18:15:34 mail-store cyr_expire[8176]: DBERROR: mydelete: error 
deleting <5212-45119-1ABV-3RAT9-VZVI6J-H at e-dialog.com>: DB_NOTFOUND: No 
matching key/data pair found

I have turned off the delprune line in /etc/imapd.conf to try and stop 
this. Although I know that's not the way to fix it. Probably hiding 
other issues related to the db formats.

Would there be an advantage to converting the remaining db's to skiplist 
in terms of stability and performance. Also would the correct method to 
do this be the same as that to convert the mailbox.db file as stated in 
the upgrade howto. Also do I need to tell cyrus in imapd.conf of this 
change?


It takes between 7 and 15 seconds to login into any mailbox even if it's 
empty. Regardless of client or OS. Also it takes about 5 minutes after a 
  restart before the deliver.db is ready and logins start. My old PII 
400 I was using before was faster!

Below are my config files and versions.

Versions
Host OS is RedHat 9

compiled source:
Cyrus-imap-2.2.3
Postfix-2.0.16

rpm's:
db4-4.0.14-20
db4-devel-4.0.14-20
cyrus-sasl-devel-2.1.17-1
cyrus-sasl-md5-2.1.17-1
cyrus-sasl-2.1.17-1
cyrus-sasl-gssapi-2.1.17-1
cyrus-sasl-plain-2.1.17-1
MySQL-devel-4.0.16-0
MySQL-client-4.0.16-0
MySQL-server-4.0.16-0

Config:
/etc/imapd.conf

postmaster: postmaster
configdirectory: /home/imap
defaultpartition: default
partition-default: /home/imap/mail
sievedir: /usr/sieve
lmtpsocket: /home/imap/socket/lmtp
servername: mail.blah.com
unixhierarchysep: yes
admins: cyrus
allowanonymouslogin: no
allowplaintext: yes
sasl_mech_list: PLAIN
maxmessagesize: 35000000
hashimapspool: true
deleteright: d
autocreatequota: 100000
reject8bit: no
quotawarn: 95
timeout: 30
poptimeout: 10
dracinterval: 0
drachost: localhost
tls_ca_file: /home/imap/ssl/mail.blah.com.pem
tls_cert_file: /home/imap/ssl/mail.blah.com.pem
tls_key_file: /home/imap/ssl/mail.blah.com.pem
config_at_sub: .
sendmail: /usr/sbin/sendmail
sasl_pwcheck_method: saslauthd

/etc/cyrus.conf


START {
   recover       cmd="ctl_cyrusdb -r"

}

SERVICES {
   imapl         cmd="imapd" listen="localhost:imap" prefork=2
   imap          cmd="imapd" listen="mail.blah.com:imap" prefork=5
   imaps         cmd="imapd -s" listen="mail.blah.com:imaps" prefork=1
   pop3          cmd="pop3d" listen="mail.blah.com:pop3" prefork=3
   pop3s         cmd="pop3d -s" listen="mail.blah.com:pop3s" prefork=1
   sieve         cmd="timsieved" listen="sieve" prefork=1
   imapsh        cmd="imapd -s -C /etc/imapd.conf.blah2" 
listen="mail.blah2.com:imaps" prefork=1
   pop3sh        cmd="pop3d -s -C /etc/imapd.conf.blah2" 
listen="mail.blah2.com:pop3s" prefork=1

   lmtpunix      cmd="lmtpd" listen="/home/imap/socket/lmtp" prefork=1

}

EVENTS {
   checkpoint    cmd="ctl_cyrusdb -c" period=30


   tlsprune      cmd="tls_prune" at=1400
}

### The blah2 is a separate host using a second cert. I have not 
converted the to using service_tls_* in /etc/imapd.conf yet. Although I 
know I should. The imapd.conf.blah2 is the same as the one above but 
with a different tls key/cert/ca file.

Saslauthd is started thus:

/usr/sbin/saslauthd -m /var/run/saslauthd -a pam

Pam uses pam_mysql.

I am sure I am missing something here, but I don't want to dive in a 
start changing things again as it seems (relatively) stable now.

Can anyone help with any of the above?

I'm sorry that was so long winded but I hope someone can help. =]

thanks

Matt

-- 
  [www.gwork.org] -----+
                       |
[www.wheres.co.uk] <--+
---
Home Page: http://asg.web.cmu.edu/cyrus
Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html