Multiple reinstalls, couple of pigs and a few days later..cyrus testing (Long)

[Jason Williams]

Hello everyone. (Quick note...pretty long post)
Hope my subject doesn't offend anyone. It was probably a poor attempt at 
humor. :)

Recently, I took on a project of testing and installing Cyrus-IMAPD for our 
company. I liked a lot of the features that it offered such as a 'sealed 
box', sieve and just an overall outstanding imap server. It also came 
highly recommended from a few of my peers. With that, I took off with the 
project.

My ultimate goal was to install it on FreeBSD 4.9 on a nice, high end 
server. Originally, I decided to download the source and compile from hand, 
with intentions of getting a feel for it. Once that was complete, I was 
going to do the actual install through the FreeBSD ports tree.

Needless to say, I ran into quite a few interesting things that have 
happened with my testing of Cyrus. My intentions are to tell what I have 
found in hopes of getting answers to some of the problems I ran into, as 
well as to hopefully help others some day in an attempt to prevent them 
from running into the same pitfalls I ran into.
(NOTE: If it has success in parenthesis, it means it was a successful login)

TEST 1

Synced source and ports.

cd /usr/ports/mail/cyrus-imapd2

Edited the Makefile to use BerkeleyDB 41 (changed 3 to 41)

Options:
make -DWITH_SKIPLIST -DWITH_MURDER
make -DWITH_SKIPLIST -DWITH_MURDER install clean

edited /etc/services --> added sieve 2000/tcp
edited /etc/rc.conf --->  added cyrus_imapd_enable="YES" at bottom
edited /usr/local/etc/imapd.conf ---> added cyrusadm to "admins" section
executed /usr/local/cyrus/bin/mkimap

rebooted

Did the following (To remove the following entry in /var/log/auth.log and 
/var/log/messages)
OTP unavailable because can't read/write key database /etc/opiekeys: 
Permission denied

touch /etc/opiekeys
chown cyrus /etc/opiekeys


testing with imtest:

imtest -m login -a testuser -p imap localhost (success)
Jan 14 14:58:01 obsidian imapd[142]: no user in db
Jan 14 14:58:01 obsidian imapd[142]: login: localhost[::1] testuser plaintext

imtest -m DIGEST-MD5 -a testuser -p imap localhost (success)
Jan 14 14:58:13 obsidian imapd[142]: no user in db
Jan 14 14:58:13 obsidian imapd[142]: login: localhost[::1] testuser 
DIGEST-MD5 User logged in

imtest -m CRAM-MD5 -a testuser -p imap localhost (success)
Jan 14 14:58:21 obsidian imapd[142]: no user in db
Jan 14 14:58:21 obsidian imapd[142]: login: localhost[::1] testuser 
CRAM-MD5 User logged in

imtest -m NTLM -a testuser -p imap localhost (success)
Jan 14 14:58:33 obsidian imapd[142]: login: localhost[::1] testuser NTLM 
User logged in


cyradm --user cyrusadm localhost (success)

Jan 14 15:01:34 obsidian imapd[171]: no user in db
Jan 14 15:01:34 obsidian imapd[171]: login: localhost[::1] cyrusadm 
DIGEST-MD5 User logged in

saslpasswd -d testuser at obsidian.batman.com

Jan 14 15:02:20 obsidian saslpasswd2: setpass succeeded for 
testuser at obsidian.batman.com
Jan 14 15:02:20 obsidian saslpasswd2: Couldn't update db
Jan 14 15:02:20 obsidian last message repeated 2 times

Success with logging in with a imap client:

Jan 14 15:05:05 obsidian imapd[178]: no user in db
Jan 14 15:05:05 obsidian imapd[178]: login: [192.168.1.90] testuser 
CRAM-MD5 User logged in

For the most part, everything appears to work. BUT, after watching the logs 
for a bit, I get the following:

Jan 14 15:08:05 obsidian ctl_cyrusdb[180]: checkpointing cyrus databases
Jan 14 15:08:05 obsidian ctl_cyrusdb[180]: DBERROR: error listing log 
files: DB_NOTFOUND: No matching key/data pair found
Jan 14 15:08:05 obsidian ctl_cyrusdb[180]: DBERROR: archive /var/imap/db: 
cyrusdb error
Jan 14 15:08:05 obsidian ctl_cyrusdb[180]: done checkpointing cyrus databases

I restarted cyrus to see more information:

Jan 14 15:12:17 obsidian master[209]: ready for work
Jan 14 15:12:17 obsidian ctl_cyrusdb[211]: checkpointing cyrus databases
Jan 14 15:12:17 obsidian ctl_cyrusdb[211]: DBERROR: error listing log 
files: DB_NOTFOUND: No matching key/data pair found
Jan 14 15:12:17 obsidian ctl_cyrusdb[211]: DBERROR: archive /var/imap/db: 
cyrusdb error
Jan 14 15:12:17 obsidian ctl_cyrusdb[211]: done checkpointing cyrus databases

Now im getting DBerrors...
I thought i'd post my 'ver' output from within cyradm:

localhost> ver
name       : Cyrus IMAPD
version    : v2.1.16 2003/11/19 16:45:28
vendor     : Project Cyrus
support-url: http://asg.web.cmu.edu/cyrus
os         : FreeBSD
os-version : 4.9-RELEASE
environment: Cyrus SASL 2.1.17
              Sleepycat Software: Berkeley DB 4.1.25: (December 19, 2002)
              Built w/OpenSSL 0.9.7c 30 Sep 2003
              Running w/OpenSSL 0.9.7c 30 Sep 2003
              CMU Sieve 2.2
              TCP Wrappers
              mmap = shared
              lock = fcntl
              nonblock = fcntl
              auth = unix
              idle = poll
              mailboxes.db = skiplist
              seen.db = skiplist
              subs.db = flat
              deliver.db = db3-nosync
              tls_sessions.db = db3-nosync

I continued to test. This time I setup TLS as per instructions on the web page.
I was able to connect both through 'imtest' and through a Mozilla client 
with no problems at all.

What is odd though is that when I restarted the cyrus server, here are the 
entries from my messages log:

Jan 14 15:21:30 obsidian master[279]: process started
Jan 14 15:21:30 obsidian ctl_cyrusdb[280]: recovering cyrus databases
Jan 14 15:21:30 obsidian ctl_cyrusdb[280]: skiplist: recovered 
/var/imap/mailboxes.db (2 records, 488 bytes) in 0 seconds
Jan 14 15:21:30 obsidian ctl_cyrusdb[280]: done recovering cyrus databases
Jan 14 15:21:30 obsidian master[279]: ready for work
Jan 14 15:21:30 obsidian ctl_cyrusdb[281]: checkpointing cyrus databases
Jan 14 15:21:30 obsidian ctl_cyrusdb[281]: done checkpointing cyrus databases

So now, no db errors?

Looking closer at the original DB error message, I noticed it was not 
looking in the right place: /var/imap/mailboxes.db
You can see in the log above, that it is looking at the correct location 
and DB and no errors.

So my question is, why is that? Why does it first start off with an error 
message, then after some playing around, it finds it?
Is that a bug? Something on my end?

It find it very odd that this occurs.
I will continue to test this and will be more than happy to post my results 
and findings to anyone who is interested in seeing them.

I appreciate any feedback and comments.

Jason