Using xfer to migrate mailboxes to a new server

[Simon Fraser]

On Mon, 2010-02-15 at 15:25 +0200, Elver Loho wrote:

> localhost.localdomain> xfer user.elver 192.168.0.180
> xfermailbox: Server(s) unavailable to complete operation

There should be a bit more information in the log, too. While debugging
the authentication and permissions I found the error messages put into
syslog were more helpful than the program output. 

I found setting it up to be relatively easy with 2.2.12/13, needing only
changes to imapd.conf:

allowusermoves: yes
proxy_authname: cyrus 
proxy_password: cyrus's password

I have "proxyservers: cyrus" set, too, but I must admit I'm not
convinced that's necessary. The cyrus user is identical on both hosts.
TLS wasn't enabled on the old server, but is enabled on the new. 

> So I'm wondering if anyone here has any experience with getting xfer
> to work in a migration scenario that they'd be willing to share.
> Anyone?

Once the transfer of test mailboxes worked, I had exim set to defer
messages if a particular flag was set, and had a perdition proxy
checking to see if the user was on the old server, the new, or deferred,
to prevent any chance of mailbox modification attempts during the
transfer. 

Some other things I saw:  xfer will move the sieve script for you.  If
one exists for that user on the destination, the mailbox transfer will
fail, as far as the source is concerned.  The destination host will have
all the mail and consider the mailbox local.  

If the transfer fails for whatever reason, the old server will have a
record in mailboxes.db to say which host that mailbox is now on, but
will be unable to perform any operation on it.  This is particularly
annoying as xfer is recursive, so if a sub-folder fails, one can't
restart the transfer for the inbox, only the other subfolders
individually.  The solution seems to be to stop cyrus, do `ctl_mboxlist
-d > /tmp/mboxlist`, edit /tmp/mboxlist to remove the reference to the
remote host for that mailbox, `ctl_mboxlist -u < /tmp/mboxlist`, and
restart cyrus.  

The offending line will look something like the following, and you'll
want to remove "hostname!". 

user.username.somefolder        hostname!default       username
lrswipcda       

It's also worth remembering that you can run xfer jobs in parallel. 

> I'm sure I'm doing something wrong here. I might even be doing
> everything wrong here. I've googled and it seems like xfer is a common
> headache for Cyrus admins with no easy solution. However, it also
> seems to be a particularly bad headache -- most old e-mails I've found
> about this topic seem to have gone unanswered.

I don't claim to be an expert at xfer, since I've only used it to
migrate to new hardware once, but hopefully it's helpful anyway. I moved
approximately 2000 mailboxes, scripting against the Cyrus::IMAP perl
module.  About a dozen had some sort of transfer error, and these were
fixable with no loss of email.

Simon.



-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.