Upgrade and Migration

[Dave McMurtrie]

Carson Gaspar wrote:
> Ben Carter wrote:
> 
>> If you use rsync, you have to stop everything until that finishes, 
>> possibly reconstruct all mailboxes, maybe fix some other things before 
>> giving people their mail functionality back and allowing mail delivery 
>> to resume.
> 
> That's just silly. If you're going to use rsync to migrate data, you do 
> at least one rsync while the source data is live. More than one if the 
> initial sync takes a long time. Then you go offline, do a final sync 
> (which should be very fast), and bring the new data store online.
> 

Running rsync prior to shutting down so it only has to copy incremental 
changes once you shut down will be faster than not doing so, but calling 
stat() for millions of files may not be very fast.  If you're not 
worried about the duration of your downtime, this doesn't matter to you.

> You have to do the _exact_ same thing with imapsync, unless you want to 
> lose email.
> 

Not true.  Read the original response again.  No downtime was incurred 
and no mail was lost using imapsync.

>> Also, the ACL format in the mailboxes file might be different between 
>> these 2 Cyrus versions.
> 
> Might be, but I don't think it is.
> 

As of the 2.3 code, Cyrus supports rfc4314 acls.  That said, I believe 
the 2.3.14 code will do the right thing if it encounters the old-style acls.

>> If you use the protocol to move the data, you don't have to worry 
>> about any data structure differences etc.  You also can re-arrange 
>> your partitions and so on.  Plus it re-calculates all quota usage as 
>> imapsync APPENDs the messages during the migration.
> 
> All true, except to the best of my knowledge none of this (except 
> repartitioning, which the OP didn't mention) matters for cyrus imapd - 
> it will Just Work(tm) on your old data store. The only exceptions are 
> database format changes (if you use bdb and you've revved the library 
> version, for example), and sieve compiled bytecode.
> 

Standard advice, with Cyrus being a sealed-server design, is usually to 
use IMAP protocol to accomplish whatever it is you're trying to do and 
to not muck with what's in the filesystem.  In this case, imapsync would 
do everything via protocol so you don't have to muck around in the 
filesystem.

Regarding the original question, however, what you're proposing to do 
with rsync should work with some caveats.  As Carson mentioned above, if 
you have a different version of bdb on the new machine, that could give 
you headaches.  If your new machine is 64 bit and the old machine was 32 
bit, I think that could also cause you problems.  Also, check your 
imapd.conf to make sure you have all the correct database formats 
specified since you're copying over the old imapd.conf to the new server 
and you're changing formats on the new server.

Rather than trust me or anyone else who tells you this should just work, 
you should test it first.  If it causes you problems, try imapsync.

 > And why do you care about quota re-calculation? The existing quota
 > data should be correct.

Technically, quota is calculated differently in 2.3.14 than it was in 
2.1.16.  At the very least, it now ignores things that aren't in 
cyrus.index when calculating quota and it didn't used to do that.

hth,

Dave
-- 
Dave McMurtrie, SPE
Email Systems Team Leader
Carnegie Mellon University,
Computing Services