Unifed Murder, xfer not working. DB affected.

Mon Feb 19 17:52:51 EST 2007

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Testing out a Unified Murder setup (2.3.8 vanilla)...

The setup: 1x MUPDATE master [aunty]
           2x Unified FE/BE [master & blaster]

What works:
     Mailbox creation
     Direct and proxy access to mboxes
     Communication and authentication among unfied & mupdate servers

The problem:
     Trying to xfer ("xfer user/test server") a mailbox always fails.

Sometimes (unable to reliably produce a procedure to cause the bad
failure yet) the mailbox db is corrupted with an invalid partition
listing which causes the mbox to be unreadable, un-deletable, un-"doing
any operation"-able. The only way to clear the box seem to be to delete
the db and mkimap again.

If I then "cyradm info" this (invalid partition mbox) there are repeated
(quick enough to open MAX_FILES and kill both systems eventually
[3min.]) connections from the transferer to transferee and necessitates
a restart of the master process. The 'catastrophic' failure seems to be
more common when an mbox has children.

NOTE: the sasl auth logs (at debug level) show no activity... Using
syslog-ng (lines are probably being dropped). These hosts (for testing
purposes) are VMs running on VMware ESX 3.0.1. I'll follow up with a
trace just as soon as I can get it to fail hard on the info command
again (never does die when you'd like it to...). I can provide any
additional information on request.

*What it looks like on the transferer*:
(What I see)
master:~ # !cyradm
cyradm -u cyradm master
IMAP Password:
master>
master> cm user/jim
master> xfer user/jim blaster
xfermailbox: The remote Server(s) denied the operation

(Initial transfer attempt... log view)
imap[17799]: No worthy mechs found
imap[17799]: Doing a peer verify
imap[17799]: Doing a peer verify
imap[17799]: received server certificate
imap[17799]: starttls: TLSv1 with cipher AES256-SHA (256/256 bits new)
no authentication
imap[17799]: Could not move mailbox: user.jim.old, LOCALCREATE failed
(Repeat Ad Nauseum...)

(if the partition entry is corrupted and I info the mbox = Death march
to open MAX_FD after info-ing above...)
imap[17735]: accepted connection
master[17736]: about to exec /usr/local/depot/cyrus-imapd-2.3.8/bin/imapd
Feb 19 17:12:01 blaster imap[17736]: executed
Feb 19 17:12:01 blaster imap[17735]: Doing a peer verify
Feb 19 17:12:01 blaster imap[17735]: Doing a peer verify
Feb 19 17:12:01 blaster imap[17735]: received server certificate

*What it looks like from the transferee*:
(Initially)
about to exec /usr/local/depot/cyrus-imapd-2.3.8/bin/imapd
imap[18886]: starttls: TLSv1 with cipher AES256-SHA (256/256 bits new)
no authentication
imap[18888]: executed
imap[18886]: login: blaster.ccit.Arizona.EDU [10.0.132.213] cyradm
LOGIN+TLS User logged in

(if the partition entry is borked + info command.)
master[15537]: process 17280 exited, status 0
master[15537]: process 16100 exited, status 0
master[15537]: process 15749 exited, status 0
master[15537]: process 17536 exited, status 0
master[15537]: process 16400 exited, status 0
master[15537]: process 16120 exited, status 0
master[15537]: process 15756 exited, status 0
master[15537]: process 17955 exited, status 0
master[15537]: process 16413 exited, status 0
master[15537]: process 17803 exited, status 0
master[15537]: process 17101 exited, status 0

*What the whole thing looks like to the MUPDATE server*:
aunty mupdate[3953]: accepted connection
Feb 19 18:57:58 aunty mupdate[3953]: starttls: TLSv1 with cipher
AES256-SHA (256/256 bits new) no authentication
Feb 19 18:57:59 aunty mupdate[3953]: login: blaster.ccit.Arizona.EDU
[10.0.132.213] blaster.ccit.arizona.edu LOGIN+TLS User logged in
Feb 19 18:57:59 aunty mupdate[3953]: cmd_set(fd:16, user.jim)
Feb 19 18:57:59 aunty mupdate[3953]: cmd_set(fd:16, user.jim)
Feb 19 18:57:59 aunty mupdate[3953]: cmd_find(fd:10, user.jim)
Feb 19 18:57:59 aunty mupdate[3953]: cmd_find(fd:10, user.jim)
Feb 19 18:58:00 aunty mupdate[3953]: cmd_find(fd:12, user.jim)
Feb 19 18:58:00 aunty mupdate[3953]: cmd_find(fd:12, user.jim)
Feb 19 18:58:02 aunty mupdate[3953]: cmd_find(fd:13, user.jim)
Feb 19 18:58:02 aunty mupdate[3953]: cmd_find(fd:13, user.jim)

(No deathmarch, ever... that's it).

Any help would be appreciated... the unified config documentation is
non-existant; I'd be happy to learn of any changes to the below config
files that would make this behave nicer. The real frustrator in this
situation is how poorly the system handles an "invalid partition"
response during a cyradm info command (the other commands just bail a
leave a usable system).

- ---begin FE/BE imapd.conf
configdirectory: /cyrus_config
defaultpartition: default
partition-default: /cyrus_mboxes/p0
partition-p0: /cyrus_mboxes/p0
partition-p1: /cyrus_mboxes/p1
admins: cyradm
lmtp_admins: lmtp cyradm
lmtp_overquota_perm_failure: no
sievedir: /cyrus_config/sieve
hashimapspool: true
unixhierarchysep: yes
altnamespace: yes
servername: master
tls_cert_file: /cyrus_config/email_verisign_2006.crt
tls_key_file: /cyrus_config/email_verisign_2006.key
tls_ca_file: /cyrus_config/verisign.ca.pem
tls_session_timeout: 0
imap_tls_request_cert: 0
pop3_tls_request_cert: 0
sendmail: /usr/lib/sendmail
singleinstancestore: yes
duplicatesuppression: yes
quotawarn: 85
timeout: 60
poptimeout: 10
imapidresponse: no
maxmessagesize: 52428800
postmaster: postmaster
sieve_maxscriptsize: 32
sieve_maxscripts: 1
imapidlepoll: 120
munge8bit: no
username_tolower: 1
sasl_pwcheck_method: auxprop
sasl_auxprop_plugin: sasldb
sasl_mech_list: digest-md5 plain login

# allow plain text
allowplaintext: yes

allowusermoves: 1
expunge_mode: delayed

mupdate_server: aunty
mupdate_config: unified
mupdate_authname: master
mupdate_realm: <removed>
mupdate_password: <removed>
mupdate_workers_maxspare: 5
mupdate_workers_minspare: 1
mupdate_workers_start: 2

proxy_password: <removed>
proxy_authname: cyradm
proxyservers: cyradm

- ---end imapd.conf

- ---begin (mupdate master) imapd.conf
configdirectory: /cyrus_config
partition-default: /tmp
admins: master blaster
servername: <removed>
unixhierarchysep: yes
altnamespace: yes

tls_cert_file: /cyrus_config/email_verisign_2006.crt
tls_key_file: /cyrus_config/email_verisign_2006.key
tls_ca_file: /cyrus_config/verisign.ca.pem

sasl_pwcheck_method: auxprop
sasl_auxprop_plugin: sasldb
sasl_mech_list: digest-md5 plain login
allowplaintext: yes

- ---end of (mupdate master) imapd.conf

Thanks for any input,
Shawn

- --
Shawn Nock (OpenPGP: 0xB64200E1)
Unix Systems Group; CCIT
University of Arizona
nock at email.arizona.edu
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFF2ipDbbRzLz6id34RAtmSAJ94crpw7uKd7I65cZOG5Y4wUlunNQCcCT03
dxeUFynKFKHAsZG/t4jHs4c=
=kKLj
-----END PGP SIGNATURE-----