DBERROR: critical database situation

[Toni]

Environment:

Fedora Core 1
2.4.22-1.2199.nptlsmp
db4-4.1.25-14
cyrus-imapd-2.1.16 (updated after the problem to cyrus-imapd-2.1.17)

Cyrus has worked flawlessly over year for me, but now out of sudden, DB 
crashed totally.

This is what happened:

--- clip ---
Nov 28 04:20:06 mail ctl_cyrusdb[908]: checkpointing cyrus databases
Nov 28 04:20:06 mail ctl_cyrusdb[908]: DBERROR db4: DB_LOGC->get: log 
record checksum mismatch
Nov 28 04:20:06 mail ctl_cyrusdb[908]: DBERROR db4: DB_LOGC->get: 
catastrophic recovery may be required
Nov 28 04:20:06 mail ctl_cyrusdb[908]: DBERROR db4: PANIC: 
DB_RUNRECOVERY: Fatal error, run database recovery
Nov 28 04:20:06 mail ctl_cyrusdb[908]: DBERROR: critical database situation
Nov 28 04:20:23 mail imapd[757]: DBERROR db4: fatal region error 
detected; runrecovery
Nov 28 04:20:23 mail imapd[757]: DBERROR: error closing: DB_RUNRECOVERY: 
Fatal error, run database recovery
Nov 28 04:20:23 mail imapd[757]: DBERROR: error closing mailboxes: 
cyrusdb error
Nov 28 04:20:23 mail imapd[757]: DBERROR db4: fatal region error 
detected; run recovery
Nov 28 04:20:23 mail imapd[757]: DBERROR: error exiting application: 
DB_RUNRECOVERY: Fatal error, run database recovery
Nov 28 04:20:23 mail imapd[757]: DBERROR: error exiting application: 
cyrusdb error
--- clip ---

Commands run as cyrus user:

# db_stat -c
db_stat: fatal region error detected; run recovery
db_stat: DB_ENV->open: DB_RUNRECOVERY: Fatal error, run database recovery

# reconstruct -f
reconstruct:
     can't initialize mboxlist environment


Ouch! Suddenly, I was in big trouble.

After some googling I tried this (even though this hint is for OSX):

http://david.codebase.ca/index.cgi/os%20x/cyrus.writeback

But in my case, it didn't start to do anything by running "reconstruct" 
command.

I ended up restoring my imap dir from last working backups and then I 
was able to run "reconstruct -f" command. Ok, got it up and running, for 
now.

Afterwards I found that using "ctl_cyrusdb -r" might have helped:

http://www.kalamazoolinux.org/pipermail/members/2004-July/011219.html

But what files exactly I need to delete before running "ctl_cyrusdb -r"?

	deliver.db      ?
	mailboxes.db    ?
	tls_sessions.db ?
	db/*            ?


I guess the cause of the problem has something to do with cyrus's weekly 
tasks to rotate/clean db files. Is that correct? Could someone please 
explaing why/what happened and how can I prevent it from happening againg?

I've read from somewhere that there is a method to dump .db files into 
textformat and then in these cases you cound recover from those. How is 
this done? I could create a hourly cron job for it, just to be more safe.

Best regards,

   Toni
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html