Move to another server?

[John Wade]

Hi Anders,

We just completed a migration from 2.0.16 to 2.2.10 and we ended up 
using a file system copy with NFS.  (two servers, one with 1500 users 
and 54 GB of mail, the other with 5,500 users and 15GB of mail)   This 
is on RHEL 3.0 using Simon's RPM's.   (Simon, we can't thank you 
enough!)  Old version was RedHat 7.0 with cyrus from source.

Basic Sequence is as follows:

PreConversion
1. startup NFS and mount old spool partitions on new box and old 
/var/lib/imap to new box
2.  Increase quotas on old server to match new server (and get rid of 
any mail in the queue to over quota users, flush the queue.)  This was 
OK since we were increasing quotas anyway.
3.  Stop mail services on new box
4.  Clean up new server (delete mailboxes, annotations, deliver, 
tls_sessions db's and supporting files.  Purge db, proc, quota,sieve 
user and log directories in /var/lib/imap on the new server.  Purge all 
/var/spool/imap partitions/delete all mail.
5.  reconfigure MTA on new box to block receipt of new mail until the 
process is completed.

Conversion
1.  Stop mail services on old box after confirming queue is empty
2.  Copy mail spool from old server to new server.  ( we ended up doing 
3 partitions at a time since this seemed to give the quickest copy.  
Your mileage will vary depending on the disk config you are using.)  our 
54GB copy took about 6 hours, but we had pretty fast IO hardware on the 
receiving side.
3. fix ownership/permissions of all files in new spool.  (assuming uid's 
are not the same)
4. Export mailboxes db to text on old server using ctl_mboxlist, copy 
file to new server
5.  Import mailboxes db on new server from text file with ctl_mboxlist
6. Copy sieve scripts via nfs, change ownership/permissions and run 
massievec to compile
7.  Copy seen and sub files over from /var/lib/imap/user  (if you don't 
use the same hashing scheme, you will have to rehash, we kept it the same)
8.  fix ownership and then convert seen to new format (we went from flat 
to skiplist) via cvt_cyrusdb
9.  start cyrus and check logs
10.   rerun quota setting script and run recursive quota fix on all 
quota roots.   This seemed easier than bringing the quotas over since we 
were using a standard size.
11.  Shutdown mail again on new server.
12.  We also did a IMP to squirrelmail conversion, so we had a step in 
here to export and convert the address books.
13.  unmount nfs mounts
14.  shutdown nfs permanently, disconnect old box from network
15.  change ip address and DNS name of new box to match old box.  ( we 
could have used a CNAME, but this was cleaner)
16.  Reboot new box to be safe
17.  Mail should start automatically, then Test, Test, Test
18.  Reconfigure MTA to allow mail to be delivered.  Test

This process was clean and painless, although time consuming.  Mail 
services for our larger server were down from 5:30PM Saturday night 
until aproximately 3:30AM Sunday morning.  Obviously, depending on what 
version you are coming from or going to, things may vary as far as 
hashing, DB format conversions required etc.   The documentation was 
actually extremely helpful in deciding which steps were appropriate for 
us.   Also, since we use external LDAP for authentication, we did not 
have to move any SASL related data.   

The nice thing about this process is that we were able to test the 
conversion 3 times.  (we just nfs copied the mail spool live and lived 
with the inconsistancies for the tests.)   We could then test the new 
server with a full set of users and mail and throw it to select end 
users to confirm that things look good.   We also had a backout 
strategy.  At any point in the process if things fell apart (until we 
astarted accepting mail on the new box), we could just shut off the new 
box and plug the old one back in.

Hope this helps,  I can give more detail on specific steps if you have 
any questions.   Our documentation on this is extensive but very 
specific to us and includes all sorts of special things that are not 
useful for a general audience. (like our ezmlm mailing list 
conversion)   The project leader on this for us (John Widera) did a 
great job.

One other thing to note, if you are using RHEL 3, be sure to get the 
latest kernel.   There was a bug in the file system caching code in the 
update 3 that kills any heavily used system with large amounts of memory 
(like our server) and lots of small files (like cyrus)   This showed up 
in our conversion testing and we had to backrev the kernel until the new 
one was released.

Good luck,
John Wade




Rob Tanner wrote:

>Anders,
>
>I went through a similar problem several months ago, updating from an
>ESYS server (commercial clone of a 1.x.x Cyrus server).  I had all
>kinds of incompatibility problems on some test mailboxes I moved over
>manually and also manually built the mailboxes file.  Some folders
>would be okay and other would be reported as invalid and reconstruct
>wouldn't clean it up.  I finally bit the bullet and used the perl
>program, imapsync, to move the mailboxes over.  With imapsync, since it
>actually talks to the old and new server via the imap protocol, the
>move was error free. The down side is that because it uses the imap
>protocol it's bloody slow.  I had some 5000 accounts with many
>gigabytes of mail store and the total transfer took some three and a
>half days to complete.  But then all I had to do afterwards was move
>the alias that points to the imap server and users were bust reading
>mail.  The other downside is that the SEEN flag is not a property of
>the mailbox and doesn't get transfered, so all mail showed up as
>unseen.  Since I was already aware of it, we announced it ahead of
>time.  
>
>-- Rob
>
>--On Friday, December 24, 2004 12:37:55 PM +0100 Anders Norrbring
><lists at norrbring.se> wrote:
>
>  
>
>>Hiya!
>>
>>I'm about to upgrade my "old" server to a newer platform, both
>>hardware, O/S and Cyrus itself are going to be new versions.
>>
>>So, how can I ensure that all mail is moved correctly from the old
>>platform to the new?  I have around 3000 accounts with lots of
>>e-mails stored in the IMAP folders.
>>
>>What I need is basically a HOW-TO with step-by-step instructions on
>>how to solve the move.
>>
>>Merry Christmas to you all!
>>
>>Anders Norrbring
>>Norrbring Consulting
>>
>>
>>---
>>Cyrus Home Page: http://asg.web.cmu.edu/cyrus
>>Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
>>List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>>    
>>
>
>
>
>  
>
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html