problems with database corruption in the frontend/master (additional info)

[João Assad]

João Assad wrote:

> Hello everyone,
>
> We use cyrus-imapd-murder as the solution for our website 
> messaging/e-mail service .
>
> We currently have 1.240.088 users split in 3 backend servers using 1 
> frontend / master server
> (both services running on the same server) for a grand total of  
> 3.479.526 mailboxes and it's subfolders.
>
> the servers are used for imap access only. no pop3, no sieve scripts, 
> no news, no shared folders , nothing
> else besides imap.
>
> after some days running smoothly I start getting errors on the 
> master/frontend server.
> It has happened a few times now..
>
> the errors in question:
>
> cyrus/mupdate[1433]: DBERROR: skiplist recovery 
> /var/lib/imap/mailboxes.db: 141BAD94 should be ADD or DELETE
> cyrus/mupdate[1433]: DBERROR: error updating database user.1215407: 
> cyrusdb error
>
> then after some time I start getting these:
>
> cyrus/lmtp[1276]: authentication to remote mupdate server failed: EOF 
> from server
> cyrus/lmtp[1276]: couldn't connect to 10.1.5.101: no authentication to 
> server
>
> after a little more time
>
> cyrus/imap[30969]: kick_mupdate: can't connect to target: Connection 
> refused
> cyrus/lmtp[2240]: mupdate-client: connection to server closed: end of 
> file reached
> cyrus/lmtp[2240]: couldn't connect to 10.1.5.101: no connection to server
>
> then this is what I get when trying to restart cyrus frontend/master
>
> cyrus/ctl_cyrusdb[30607]: recovering cyrus databases
> cyrus/ctl_cyrusdb[30607]: DBERROR: skiplist recovery 
> /var/lib/imap/mailboxes.db: 141BAD94 should be ADD or DELETE
> cyrus/ctl_cyrusdb[30607]: DBERROR: opening /var/lib/imap/mailboxes.db: 
> cyrusdb error
>
> what I need to do next is completely delete the dbs from the 
> master/frontend and reimport all the mailboxes, but that takes hours.
> I came up with a faster solution wich is to configure a dummy backend 
> on the same server the cyrus frontend/master instance
> runs which reduces the import time by hours, but thats besides the point.
>
> Below is my frontend/master configuration.
>
> ##### cyrus.conf
>
> START {
>  # do not delete this entry!
>  recover       cmd="ctl_cyrusdb -r"
> }
>
> # UNIX sockets start with a slash and are put into /var/lib/imap/sockets
> SERVICES {
>  # add or remove based on preferences
>  mupdate       cmd="mupdate -m" listen=3905 prefork=1
>  lmtp          cmd="lmtpproxyd" listen="lmtp" prefork=0
>  imap          cmd="proxyd" listen="imap" prefork=0
> }
>
> EVENTS {
>  # this is required
>  checkpoint    cmd="ctl_cyrusdb -c" period=240
> }
>
> ###### imapd.conf
>
> configdirectory: /var/lib/imap
> partition-default: /tmp
> admins: cyrus
> sievedir: /var/lib/imap/sieve
> sendmail: /usr/sbin/sendmail
> hashimapspool: true
> sasl_pwcheck_method: saslauthd
> sasl_mech_list: PLAIN
> tls_cert_file: /usr/share/ssl/certs/crt.crt
> tls_key_file: /usr/share/ssl/certs/key.key
> tls_ca_file: /usr/share/ssl/certs/ca.ca
>
> allowusermoves: 1
>
> #Backend servers
> cyrus-be1_password: ***
> cyrus-be2_password: ***
> cyrus-be3_password: ***
> proxy_authname: cyrus
>
>
> #Mupdate server
> mupdate_server:10.1.5.101
> mupdate_authname:cyrus
> mupdate_password: ***
>
> maxmessagesize: 2097152
> syslog_prefix:cyrus
> lmtp_over_quota_perm_failure:1
> quotawarn: 110
> imapidlepoll: 0
> fulldirhash: 1
> munge8bit: 0
> timeout: 10
>
> tls_session_timeout: 0
> mupdate_connections_max: 1024
> berkeley_cachesize:102400
> berkeley_txns_max:500

Fedora core 2

cyrus-sasl-2.1.18-2.2
cyrus-sasl-md5-2.1.18-2.2
cyrus-sasl-plain-2.1.18-2.2
cyrus-imapd-murder-2.2.10-3.fc2
cyrus-imapd-2.2.10-3.fc2
cyrus-imapd-utils-2.2.10-3.fc2

Thank you,

João Assad

---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html