What happened to my db/ ?

[Simon Matter]

>>> Hello All,
>>>   I recently moved my cyrus-imapd install from one machine to annother.
>>> The move went *quite* well, I had both servers stoped and rsynced the
>>> live server's data (/var/imap, /var/spool/imap) over to the new server.
>>>
>>> Everything was fine untill I started playing with SSL, at that point I
>>> started having problems getting cyrus running after a restart.
>>>
>>> Here is what I was seeing in my logs:
>>> """
>>> Jan  8 17:27:11 [tls_prune] DBERROR db4: operation not permitted during
>>> recovery.
>>> Jan  8 17:27:11 [tls_prune] DBERROR: opening /var/imap/tls_sessions.db:
>>> Invalid argument
>>> Jan  8 17:27:11 [tls_prune] DBERROR: opening /var/imap/tls_sessions.db:
>>> cyrusdb error
>>> """
>>>
>>> Somewhere I found a post from someone having the same problem. They
>>> said
>>> that they could get cyrus working again by removing all the files in
>>> /var/imap/db. So I did that. Now I can start cyrus and read my mail
>>> *PHEW*. However I now have a new raft of errors showing up in syslog.
>>> Everytime I start cyrus I get this:
>>> """
>>> Jan  8 18:01:42 [master] setrlimit: Unable to set file descriptors
>>> limit
>>> to -1: Operation not permitted
>>> Jan  8 18:01:42 [master] retrying with 1024 (current max)
>>> Jan  8 18:01:42 [master] process started
>>> Jan  8 18:01:42 [ctl_cyrusdb] recovering cyrus databases
>>> Jan  8 18:01:42 [ctl_cyrusdb] DBERROR db4: DB_ENV->log_flush: LSN past
>>> current end-of-log
>>>                 - Last output repeated twice -
>>> Jan  8 18:01:42 [ctl_cyrusdb] DBERROR db4: Recovery function for LSN 1
>>> 182
>>> failed
>>> Jan  8 18:01:42 [ctl_cyrusdb] DBERROR: dbenv->open '/var/imap/db'
>>> failed:
>>> Invalid argument
>>> Jan  8 18:01:42 [ctl_cyrusdb] DBERROR: init /var/imap/db: cyrusdb error
>>> Jan  8 18:01:42 [ctl_cyrusdb] DBERROR db4: environment not yet opened
>>> Jan  8 18:01:42 [ctl_cyrusdb] DBERROR: opening /var/imap/mailboxes.db:
>>> Invalid argument
>>> Jan  8 18:01:42 [ctl_cyrusdb] DBERROR: opening /var/imap/mailboxes.db:
>>> cyrusdb error
>>> Jan  8 18:01:42 [master] process 4105 exited, status 75_
>>> Jan  8 18:01:42 [master] ready for work
>>> Jan  8 18:01:42 [tls_prune] tls_prune: purged 0 out of 0 entries
>>> Jan  8 18:01:42 [ctl_cyrusdb] checkpointing cyrus databases
>>> Jan  8 18:01:42 [ctl_deliver] duplicate_prune: pruning back 3 days
>>> Jan  8 18:01:42 [ctl_deliver] duplicate_prune: purged 2 out of 742
>>> entries
>>> Jan  8 18:01:42 [ctl_cyrusdb] done checkpointing cyrus databases
>>> """
>>>
>>> I don't *think* it's a permissions problem...
>>> """
>>> usr # ls -la /var/imap/db/.
>>> total 12952
>>> drwxr-xr-x    2 cyrus    mail         4096 Jan  8 21:15 .
>>> drwxr-xr-x   14 cyrus    mail         4096 Jan  9 16:45 ..
>>> -rw-r--r--    1 cyrus    mail            0 Sep 23 13:31 .keep
>>> -rw-------    1 cyrus    mail         8192 Jan  8 21:15 __db.001
>>> -rw-------    1 cyrus    mail       270336 Jan  8 21:15 __db.002
>>> -rw-------    1 cyrus    mail        98304 Jan  8 21:15 __db.003
>>> -rw-------    1 cyrus    mail     18063360 Jan  8 21:15 __db.004
>>> -rw-------    1 cyrus    mail        32768 Jan  8 21:15 __db.005
>>> -rw-------    1 cyrus    mail       286763 Jan  9 17:10 log.0000000001
>>> """
>>>
>>> I'm using EXT3 so I don't think it's anything to do with the "chattr
>>> +S"
>>> stuff.
>>>
>>> I tryed recompiling cyrus.... it didn't help.
>>>
>>> The idea of cyrus dying fills me with dread. Someone please help me out
>>> here.... what is going on and how do I stop it? If someone can give me
>>> a
>>> bit of a shove in the right direction it would be MOST helpfull.....
>>> links... generall information.... even mild abuse, all gratefully
>>> accepted.
>>
>> Please search the list archives.
> I have... I used parts of the error message as keywords and didn't come up
> with much that appeared to be relevent. If you can suggest better keywords
> I will happily try again.
>
>> IIRC there have been two problems which
>> could create messages like yours. One problem is with NPTL on newer
>> RedHat
>> Linux and AMD and some other CPU's.
> I found some posts on this and did some reading. At first I got a bit
> excited becuase I have just changed from a PIII to a Athalon. Now I don't
> think that this is the problem. I compiled from source (I use Gentoo) so
> '--enable-posixmutexes' shouldn't be set. I also just tryed recompiling
> db4 with '--disable-posixmutexes'. AFAIK I didn't have to recompile cyrus
> after (please correct me if I'm wrong here). After a restart of cyrus I'm
> still getting the same messages.
>
>> The other problem is IIRC something
>> with the latest greatest DB4 release. Search for 'DB_ENV' and
>> 'dbenv->open', I'm quite sure you'll find out what's wrong.
> I can't find the second issue you reffer to.  When I search on those two
> terms alone I get a snow of documentation and when I add keywords like
> 'bug' or 'cyrus' I only find the NTPL problems.
>
> It occured to me after I posted this that I didn't give any versions of
> what I'm using....
> The system is running Gentoo and I'm using the packages marked as stable
> in the portage tree.
> DB4 = 4.0.14
> Cyrus = 2.1.15
> CPU = AMD T-bird 900
> Kernel = gs-sources 2.4.23_pre8-gss-r2
> If I've missed anything please let me know.
>
> The only other idea I've got from my hunting around is that it might be
> worth trying to manually rebuild/fix the db/ manually. That gave me nearly
> the same error:
> db_recover: DB_ENV->log_flush: LSN past current end-of-log
> db_recover: DB_ENV->log_flush: LSN past current end-of-log
> db_recover: Recovery function for LSN 1 182 failed
> db_recover: DB_ENV->open: Invalid argument
>
> Many thanks for the help but I'm afraid I need more :(

Well, since you told us you moved your db files from another server, I
suspect you have a database mismatch. Cyrus-imapd can be compiled with
many different database backends and even the ondisk format of db3 and db4
are not compatible. I'm quite sure that your database files (or one of
them) is not in the format your Cyrus-imapd expects.

Simon

>
>   Nick
>