mailboxes.db: cyrusdb error

[Florin Andrei]

Here's my little horror story, and how i got out of it.
I still have no explanation as for WHY it happened, other than maybe
some bug with cyrus-imapd? I'll let you guys draw conclusions.

Using:
cyrus-imapd-2.2.3
Linux Fedora Core 1

Last night i had to move my IMAP server to another location. I shut it
down normally (init 0), no errors being displayed during shutdown, all
was well.

This morning, when powering it up at the new location, Cyrus was not
working anymore:

Mar 12 10:29:02 stantz master[2276]: process started
Mar 12 10:29:02 stantz master[2277]: about to exec
/usr/lib/cyrus-imapd/ctl_cyrusdb
Mar 12 10:29:06 stantz ctl_cyrusdb[2277]: recovering cyrus databases
Mar 12 10:29:06 stantz ctl_cyrusdb[2277]: DBERROR: skiplist recovery:
0090 should be ADD or DELETE
Mar 12 10:29:06 stantz ctl_cyrusdb[2277]: DBERROR: opening
/var/lib/imap/mailboxes.db: cyrusdb error
Mar 12 10:29:06 stantz master[2276]: process 2277 exited, status 75
Mar 12 10:29:06 stantz master[2324]: about to exec
/usr/lib/cyrus-imapd/idled
Mar 12 10:29:06 stantz idled[2324]: DBERROR: skiplist recovery: 0090
should be ADD or DELETE
Mar 12 10:29:06 stantz idled[2324]: DBERROR: opening
/var/lib/imap/mailboxes.db: cyrusdb error
Mar 12 10:29:06 stantz idled[2324]: can't read mailboxes file
Mar 12 10:29:06 stantz idled[2324]: exiting
Mar 12 10:29:06 stantz master[2276]: process 2324 exited, status 75
Mar 12 10:29:06 stantz master[2276]: ready for work
Mar 12 10:29:06 stantz master[2332]: about to exec
/usr/lib/cyrus-imapd/ctl_cyrusdb
Mar 12 10:29:06 stantz master[2333]: about to exec
/usr/lib/cyrus-imapd/imapd
Mar 12 10:29:06 stantz master[2334]: about to exec
/usr/lib/cyrus-imapd/imapd
Mar 12 10:29:06 stantz ctl_cyrusdb[2332]: checkpointing cyrus databases
Mar 12 10:29:06 stantz master[2335]: about to exec
/usr/lib/cyrus-imapd/lmtpd
Mar 12 10:29:06 stantz imap[2333]: executed
Mar 12 10:29:06 stantz imaps[2334]: executed
Mar 12 10:29:06 stantz lmtpunix[2335]: executed
Mar 12 10:29:06 stantz ctl_cyrusdb[2332]: archiving database file:
/var/lib/imap/annotations.db
Mar 12 10:29:07 stantz imap[2333]: DBERROR: skiplist recovery: 0090
should be ADD or DELETE
Mar 12 10:29:07 stantz imap[2333]: DBERROR: opening
/var/lib/imap/mailboxes.db: cyrusdb error
Mar 12 10:29:07 stantz imap[2333]: Fatal error: can't read mailboxes
file
Mar 12 10:29:07 stantz imaps[2334]: DBERROR: skiplist recovery: 0090
should be ADD or DELETE
Mar 12 10:29:07 stantz imaps[2334]: DBERROR: opening
/var/lib/imap/mailboxes.db: cyrusdb error
Mar 12 10:29:07 stantz imaps[2334]: Fatal error: can't read mailboxes
file
Mar 12 10:29:07 stantz master[2276]: process 2333 exited, status 75
Mar 12 10:29:07 stantz master[2276]: service imap pid 2333 in READY
state: terminated abnormally
Mar 12 10:29:07 stantz master[2276]: process 2334 exited, status 75
Mar 12 10:29:07 stantz master[2276]: service imaps pid 2334 in READY
state: terminated abnormally

To my utter dismay, none of the typical recovery procedures worked. I
tried these things:

/usr/lib/cyrus-imapd/ctl_mboxlist -d -f /var/lib/imap/mailboxes.db

(said it cannot read mailboxes.db)

/usr/lib/cyrus-imapd/reconstruct -m

(said it's not implemented! man, that was painful)

Fortunately, i googled up the mailing lists for messages containing
similar errors, and i found a script made by Edward Rudd, which i
modified slightly:

#!/bin/sh 
TAB=`echo -e \\\t`
#cd /var/spool/imap/user
cd /var/spool/imap/f/user
find . -type d  | grep ./ |
    sed -e "s/\.\///" |
    sed -e "s/\//\./g" |
    sed -e
"s/\([a-z]*\)\(.*\)/user\.\1\2${TAB}default${TAB}\1${TAB}lrswipcda${TAB}cyrus${TAB}lrswipcda${TAB}/"
cd -

This essentially rebuilt the database in plan text format. I only have
the f/ directory in /var/spool/imap (this is a single-user IMAP server),
and also a stage./ directory which i have no idea what it represents.
Hopefully i did the right thing.

Then, as the cyrus user, i reimported it:

/usr/lib/cyrus-imapd/ctl_mboxlist -u -f /var/lib/imap/mailboxes.db <
mailboxes.txt

And now it works. I verified, i seem to be able to access all my
folders, i can create and delete folders from my IMAP client
(Evolution), Postfix is able to deliver messages via LMTP, Sieve
filtering seems to do its job...

That was scary. Hopefully someone has an explanation for the problem.

-- 
Florin Andrei

http://florin.myip.org/

---
Home Page: http://asg.web.cmu.edu/cyrus
Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html