Help moving/updating Cyrus installation

Sun Feb 15 11:36:40 EST 2004

Hi folks,

I hope this case isn't a 'told you so', but I have a bit of a problem...

Summary: I want to get imap running again, but it refuses to read my bdb
(version 3) databases. How to I recover?

I have been using Cyrus IMAPd 2.1.16 on a RH9 Linux server successfully
for some time now. I decided I should upgrade the server OS to Fedora
Core 1.  I took a binary (tar-file) backup of the imap data, which I
store on its own ext3 disk called /imap, along with other tar backups of
other parts of the system. The OS install seems to have gone well,
except that cyrus failed to start up, with errors like this:

23:09:26 master[1111]: setrlimit: Unable to set file descriptors limit to -1: Operation not permitted
23:09:26 master[1111]: retrying with 1024 (current max)
23:09:26 master[1111]: process started
23:09:26 master[1132]: about to exec /usr/cyrus/bin/ctl_cyrusdb
23:09:27 ctl_cyrusdb[1132]: recovering cyrus databases
23:09:27 ctl_cyrusdb[1132]: DBERROR db4: write: 0xbfffcf80, 8192: Invalid argument
23:09:27 ctl_cyrusdb[1132]: DBERROR: dbenv->open '/imap/conf/db' failed: Invalid argument
23:09:27 ctl_cyrusdb[1132]: DBERROR: init /imap/conf/db: cyrusdb error
23:09:27 ctl_cyrusdb[1132]: DBERROR db4: environment not yet opened
23:09:27 ctl_cyrusdb[1132]: DBERROR: opening /imap/conf/mailboxes.db: Invalid argument
23:09:27 ctl_cyrusdb[1132]: DBERROR: opening /imap/conf/mailboxes.db: cyrusdb error
23:09:27 master[1111]: process 1132 exited, status 75 
23:09:27 master[1111]: no service 'sieve' in /etc/services, disabling sieve
23:09:27 master[1111]: no service 'lmtp' in /etc/services, disabling lmtp
23:09:27 master[1111]: ready for work
23:09:27 master[1133]: about to exec /usr/cyrus/bin/tls_prune
23:09:27 master[1134]: about to exec /usr/cyrus/bin/ctl_deliver
23:09:27 master[1135]: about to exec /usr/cyrus/bin/ctl_cyrusdb
23:09:27 ctl_cyrusdb[1135]: checkpointing cyrus databases
23:09:45 ctl_cyrusdb[1135]: DBERROR db4: unable to join the environment
23:09:45 ctl_cyrusdb[1135]: DBERROR: dbenv->open '/imap/conf/db' failed: Resource temporarily unavailable
23:09:45 ctl_cyrusdb[1135]: DBERROR: init /imap/conf/db: cyrusdb error
23:09:45 ctl_cyrusdb[1135]: done checkpointing cyrus databases

So I thought "oh no"!

Several hours later and after recompiling both imapd again (no luck) and
switching from the RH kernel 2.4.22 with nptl to a standard kernel.org
2.4.24 kernel, and it still wasn't working.

Note that I hadn't changed the cyrus code at all: it was the same code
running against the same databases, and invoking the same copy of
db4 4.1.25 (self-compiled with no threading).

I eventually gave up and my email is now (i hope temporarily) back on
the old /var/mail delivery :-(

Things I can do: db_dump on the mailboxes.db file produces what
		 looks like a valid dump.

Things I can't do: any operation involving reading the database.
                 [including cvt_mmboxlist using either the old or
		 new executables]

I have now created a build of 2.2.3, partly in the hope that it works
better, and also because I want to get away from bdb: it has caused lots
of hassle in the past. Running apps from that release doesn't help me,
however. In the hope it's useful, I have included the non-data part of
the db_dump -da output here, with my /etc/imap.conf file after it:

In-memory DB structure:
btree: 0x48000 (DB->open called, read-only)
bt_meta: 0 bt_root: 1
bt_maxkey: 0 bt_minkey: 2
bt_compare: 0x400258ac bt_prefix: 0x4002590c
bt_lpgno: 0
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
page 0: btree metadata level: 0 (lsn.file: 7 lsn.offset: 7784517)
	magic: 0x53162
	version: 8
	pagesize: 4096
	type: 9
	keys: 0	records: 0
	free list: 0
	last_pgno: 3
	flags: 0
	uid: 83 fe 0 0 42 3 0 0 48 f2 52 3d 37 3d 0 0 0 0 0 0
	maxkey: 0 minkey: 2
	root: 1
page 1: btree internal level: 2 records: 0 (lsn.file: 7 lsn.offset: 7788837)
	entries:    2 offset: 4044
	[000] 4084 count:    0 pgno:    2 type:    1len:   0
	[001] 4044 count:    0 pgno:    3 type:    1len:  28 data: user.ruthc.Lists.F2S...
page 2: btree leaf level: 1 (lsn.file: 7 lsn.offset: 7849118)
	prev:    0 next:    3 entries:   70 offset: 2044
	[000] 4080 len:  11 data: Junk E-mail
	[001] 4056 len:  21 data: 0 default anyone0x09rwi...
	[002] 4044 len:   8 data: user.cbs
	[003] 4004 len:  34 data: 0 default cbs0x09lrswip...
	[004] 3984 len:  14 data: user.cbs.Admin
	[005] 3944 len:  34 data: 0 default cbs0x09lrswip...
..snip...
	[060] 2296 len:  22 data: user.ruthc.wotug-ctt...
	[061] 2264 len:  26 data: 0 default ruthc0x09lrsw...
db_dump: close: Permission denied

The /etc/imap.conf file [note I'm not using the mysql stuff - it
was an experiment]:

configdirectory: /imap/conf
sievedir: /imap/sieve
sieveusehomedir: false
sendmail: /usr/exim/bin/exim
partition-default: /imap/spool
admins: root ruthc
srvtab: /imap/srvtab
defaultacl: anyone write
allowanonymouslogin: yes
allowplaintext: yes
auxprop_plugins: mysql digest-md5 cram-md5 plain
#sasl_pwcheck_method: pwcheck
sasl_pwcheck_method: saslauthd
sasl_mech_list: mysql
sasl_mysql_user: cyrus
sasl_mysql_database: saslmail
sasl_mysql_statement: select password from users where username = %u 
#sasl_mysql_statement: select %p from users where username = %u and realm = %r
sasl_mysql_verbose:  true

---
Home Page: http://asg.web.cmu.edu/cyrus
Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html