Problems with a crashed BDB

[Simon Matter]

> Hi,
>
> I have a big problem with my Cyrus and my BDB here. I'm using Cyrus
> 2.2.12 with BDB 4.2.52.
>
> When I'm starting Cyrus, I get this error in my imapd.log:
> -------
> Aug 30 22:38:44 alpha1 master[19525]: setrlimit: Unable to set file
> descriptors limit to -1: Operation not permitted
> Aug 30 22:38:44 alpha1 master[19525]: retrying with 1024 (current max)
> Aug 30 22:38:44 alpha1 master[19525]: process started
> Aug 30 22:38:44 alpha1 master[19526]: about to exec
> /opt/cyrus/cyrus/bin/ctl_cyrusdb
> Aug 30 22:38:44 alpha1 ctl_cyrusdb[19526]: DBERROR db4: Recovery
> function for LSN 8 7404244 failed on backward pass
> Aug 30 22:38:44 alpha1 ctl_cyrusdb[19526]: DBERROR db4: PANIC: Is a
> directory
> Aug 30 22:38:44 alpha1 ctl_cyrusdb[19526]: DBERROR: critical database
> situation
> Aug 30 22:38:44 alpha1 master[19525]: process 19526 exited, status 75
> Aug 30 22:38:44 alpha1 master[19525]: ready for work
> Aug 30 22:38:44 alpha1 master[19527]: about to exec
> /opt/cyrus/cyrus/bin/ctl_cyrusdb
> Aug 30 22:38:44 alpha1 ctl_cyrusdb[19527]: DBERROR db4: PANIC: fatal
> region error detected; run recovery
> Aug 30 22:38:44 alpha1 ctl_cyrusdb[19527]: DBERROR: critical database
> situation
> Aug 30 22:38:44 alpha1 master[19525]: process 19527 exited, status 75
> --------
>
> So I tried a reconstruct -m and a reconstruct -r user/username on a
> specific mailbox and I'm getting this:
> --------
> Aug 30 22:41:08 alpha1 reconstruct[19571]: DBERROR db4: PANIC: fatal
> region error detected; run recovery
> Aug 30 22:41:08 alpha1 reconstruct[19571]: DBERROR: critical database
> situation
> --------
>
> Running "ctl_cyrusdb -r" results in a:
> --------
> ug 30 22:41:50 alpha1 ctl_cyrusdb[19582]: DBERROR db4: Recovery function
> for LSN 8 7404244 failed on backward pass
> Aug 30 22:41:50 alpha1 ctl_cyrusdb[19582]: DBERROR db4: PANIC: Is a
> directory
> Aug 30 22:41:50 alpha1 ctl_cyrusdb[19582]: DBERROR: critical database
> situation
> --------
>
> Damn BDB, but HOW CAN I GET MY MAILS BACK? I'm at one's wits' end, so
> any help would be very great because I have some very important mails
> there!

Two things:

1) I'm usually not wasting my time anymore with BDB problems. Skiplist
does work so fine with cyrus-imapd that I see no reason to stay with BDB.
At least in my RPM packages I switched all databases to skiplist as a
default and I never heard anyone complaining about it.

2) I have the following code fragment in my db scripts to fix broken BDB
files. Maybe it can help you.

Regards,
Simon

==========================
# make sure our Berkeley databases are in a sane state
# wait for db_checkpoint to end successfully or kill it after a timeout
db_checkpoint -v -1 -h $imap_prefix/db &
DB_CHECK_PID=$!
CNT=0
while [ $CNT -lt 60 ]; do
  if ! kill -0 $DB_CHECK_PID > /dev/null 2>&1; then
    break
  fi
  sleep 1
  let CNT+=1
done
if kill -0 $DB_CHECK_PID > /dev/null 2>&1; then
  kill -USR1 $DB_CHECK_PID > /dev/null 2>&1
  sleep 1
  kill -KILL $DB_CHECK_PID > /dev/null 2>&1
  wait $DB_CHECK_PID > /dev/null 2>&1
fi

# do a normal recovery
db_recover -v -h $imap_prefix/db
RETVAL=$?
if [ $RETVAL -ne 0 ]; then
  # try a catastrophic recovery instead of normal recovery
  db_recover -v -c -h $imap_prefix/db
  RETVAL=$?
  ERRVAL=$[ $ERRVAL + $RETVAL ]
  if [ $RETVAL -ne 0 ]; then
    echo "ERROR: catastrophic recovery of Berkeley databases failed"
  fi
fi
===============================