Emergancy..... db4: PANIC: fatal region error

[eMail Lists]

On 7/27/07, Simon Matter <simon.matter at invoca.ch> wrote:
>
> > On 7/27/07, Simon Matter <simon.matter at invoca.ch> wrote:
> >>
> >> > I have been holding back updating to cyrus 2.2.13 but this afternoon
> I
> >> > went
> >> > ahead and made the major update and really tried to follow ALL the
> >> points
> >> > and rules in updating but I really think I messed up the whole system
> >> now.
> >> > I really hope I have not hosed all the email today and have to go
> back
> >> to
> >> > an
> >> > older backup :(.
> >> >
> >> > I really do need some help for I am so lost and I think everything I
> >> have
> >> > been trying has made it to worse and back again.
> >>
> >> Hi,
> >>
> >> I'm really no expert with BDB and I'm  avoiding it wherever I can. But
> >> what comes to mind is this:
> >> You didn't tell us what exactly you have updated. It's cyrus-imap from
> >> ???
> >> to 2.2.13. But what else? My guess is that you also updated BDB from
> db3
> >> to db4 and didn't manage to upgrade you BDB files, can that be? I read
> >> something like "istoric log version 3" which makes me believe you are
> >> trying to access db3 files with cyrus/db4.
> >> From what I know you have to dump the db3 db's with db3 and then import
> >> to
> >> db4 files with the db4 tools. IIRC this has been discussed on this list
> >> several times and you should be able to find the correct command
> >> sequence
> >> for it in the archives.
> >>
> >> To avoid that kind of problems in future you could convert all BDB
> >> databases to skiplist and enjoy beeing free of BDB version conflicts in
> >> the future. However to fix it now you will for sure need the db3
> >> libs/tools to get anything usefull out of your db3 files.
> >>
> >> Hope that helps at least a little bit.
> >> Simon
> >>
> >> >
> >> > Here is the output from syslog.
> >> >
> >> > Jul 26 22:37:27 jurassic cyrus/master[10684]: process started
> >> > Jul 26 22:37:31 jurassic cyrus/master[10708]: about to exec
> >> > /usr/sbin/ctl_cyrusdb
> >> > Jul 26 22:37:32 jurassic cyrus/ctl_cyrusdb[10708]: DBERROR db4:
> >> Skipping
> >> > log
> >> > file /var/lib/cyrus/db/log.00000000
> >> > 55: historic log version 3
> >> > Jul 26 22:37:33 jurassic cyrus/ctl_cyrusdb[10708]: recovering cyrus
> >> > databases
> >> > Jul 26 22:37:33 jurassic cyrus/ctl_cyrusdb[10708]: skiplist:
> recovered
> >> > /var/lib/cyrus/mailboxes.db (563 records,
> >> >  84908 bytes) in 0 seconds
> >> > Jul 26 22:37:33 jurassic cyrus/ctl_cyrusdb[10708]: skiplist:
> recovered
> >> > /var/lib/cyrus/annotations.db (0 records,
> >> >  144 bytes) in 0 seconds
> >> > Jul 26 22:37:33 jurassic cyrus/ctl_cyrusdb[10708]: DBERROR db4:
> >> > /var/lib/cyrus/db/log.0000000056: log file open
> >> > failed: No such file or directory
> >> > Jul 26 22:37:33 jurassic cyrus/ctl_cyrusdb[10708]: DBERROR db4:
> PANIC:
> >> No
> >> > such file or directory
> >> > Jul 26 22:37:33 jurassic cyrus/ctl_cyrusdb[10708]: DBERROR: critical
> >> > database situation
> >> > Jul 26 22:37:34 jurassic cyrus/master[10684]: process 10708 exited,
> >> status
> >> > 75
> >> > Jul 26 22:37:34 jurassic cyrus/master[10754]: about to exec
> >> > /usr/sbin/cyr_expire
> >> > Jul 26 22:37:34 jurassic cyrus/cyr_expire[10754]: DBERROR db4: PANIC:
> >> > fatal
> >> > region error detected; run recovery
> >> > Jul 26 22:37:34 jurassic cyrus/cyr_expire[10754]: DBERROR: critical
> >> > database
> >> > situation
> >> > Jul 26 22:37:34 jurassic cyrus/master[10684]: process 10754 exited,
> >> status
> >> > 75
> >> > Jul 26 22:37:34 jurassic cyrus/master[10755]: about to exec
> >> > /usr/sbin/tls_prune
> >> > Jul 26 22:37:34 jurassic cyrus/tls_prune[10755]: DBERROR db4: PANIC:
> >> fatal
> >> > region error detected; run recovery
> >> > Jul 26 22:37:34 jurassic cyrus/tls_prune[10755]: DBERROR: critical
> >> > database
> >> > situation
> >> > Jul 26 22:37:34 jurassic cyrus/master[10684]: process 10755 exited,
> >> status
> >> > 75
> >> > Jul 26 22:37:34 jurassic cyrus/master[10684]: ready for work
> >> > Jul 26 22:37:34 jurassic cyrus/master[10766]: about to exec
> >> > /usr/sbin/ctl_cyrusdb
> >> > Jul 26 22:37:34 jurassic cyrus/master[10767]: about to exec
> >> > /usr/lib/cyrus/bin/notifyd
> >> > Jul 26 22:37:34 jurassic cyrus/ctl_cyrusdb[10766]: DBERROR db4:
> PANIC:
> >> > fatal
> >> > region error detected; run recovery
> >> > Jul 26 22:37:34 jurassic cyrus/ctl_cyrusdb[10766]: DBERROR: critical
> >> > database situation
> >> > Jul 26 22:37:34 jurassic cyrus/master[10684]: process 10766 exited,
> >> status
> >> > 75
> >> >
> >> >
> >> >
> >> > No mater what I do I just can not get passed this error.   I have
> >> > tried...reconstructing, db4.3_recover, bringing in a backup db and
> >> each
> >> > one
> >> > just does not clear the air.  Every time I try the error still
> >> persists.
> >> > Is
> >> > this recoverable?  Everything I have read on how to fix it does not
> >> work.
> >> > I
> >> > am wondering if i really missed something here.  I really do hope to
> >> get
> >> > some feed back ASAP because I want to get this fixed within the next
> 7
> >> > hrs.
> >> >
> >> > -Adam
> >
> >
> >
> >
> > Simon,
> >
> > Thank you.  Yes.. after being very tired and warry from working on this
> > for
> > a long while I  did forget to mention a few things.
> >
> > I started out with cyrus21 and moved to cyrus22 which should have been
> an
> > easy upgrade but wow.. I really got into a mess.  This all started
> because
> > vacation scripts were not working and trying to get them to work
> properly.
> > Aslo I needed the virtual domain function in the new 2.2 which led me to
> > this fun trip.
> >
> > Yes, I did update DB to 4.3 and had thought I converted everything over
> to
> > skiplist but guess I had not.  When I used db4.3_update I did not get a
> > good
> > response and the moment I do not remember what it was.  I really tried
> to
> > do
> > as much as possible and maybe as much damage before I consulted the mail
> > list.  It looks like now the files are skiplist however the DB files in
> > the
> > db directory, do I need to do anything with those? Or, are all db files
> I
> > need to convert are in the /var/lib/cyrus/ proper directory?
> >
> > These are the files I have wich are skiplist.
> > mailboxes.db
> > annotations.db
>
> Usually there are at least four db files:
> annotations.db
> deliver.db
> mailboxes.db
> tls_sessions.db
>
> I think only mailboxes.db is really important here, deliver.db and
> tls_sessions.db can be wiped and will be created  again, annotations.db is
> empty if you don't use annotations.
> You should also have a look at the db/ directory. IIRC there can be a
> situation when you moved to skiplist but still have BDB files in db/ it
> will still disturb and should be removed while cyrus is stopped.
>
> The think with hashed directories you mentioned in another mail can be
> controlled with hashimapspool and fulldirhash in /etc/imapd.conf. I have
> no idea how the Debian packages are configured but you should find it in
> the package description somehow.
>
> Simon
>

Very nice.. Thank you.  It looks like my deliver.db is still a db(x) file.
Can the files in db/ just be erased and when cyrus starts up it creats new
files or do the __db* files ned to be converted.  I am not sure how to tell
if those are db(x) or skiplist.

I am looking up the hashimapsool and fulldrihash right now.

-Adam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20070727/44037367/attachment.html