copying and renaming mailboxes to clean up directories

[Thorsten Büker]

Dear all,

For quite a few month I ran a backup-scenario set up according to [1]
including synchronisation of the mailboxes using "rsync -vaR". (In the
meantime I migrated to the usage of LVM snapshots.)

As restoring a backup became necessary due to filesystem corruption I
fetched the mailboxes from the r'synched folders as well as the mailbox 
databases and further necessary stuff. The restore went smoothly.

But wondering about the heavily increased amount of partition in use, I
noticed that my call of rsync lacked of the "--delete" attribute. In 
consequence the restored mailbox directories include not only the 
messages, which should be in the inbox, but all files beeing in the 
directories during the backup cronjobs. (My mailbox sizes ~700 MB 
instead of ~100 MB.)

With regard to backup performance I'd like to clean up the relevant
directories by ensuring uniformity of the inbox according to the Cyrus
mailbox database and the files in the mailbox directory.

In the wiki I found Andrew's collection of scripts [2] and had a try. My 
intention is/was to firstly manually select mailbox directories, which 
contain much too much unneeded files -- mainly mailboxes used via imap. 
After temporarily stopping mail delivery and mailbox access, each chosen 
mailbox might be renamed and its contents copied to a new mailbox of its 
initial name. After starting mail delivery and access again, the 
mailboxes and directories containing wrong fails might be deleted at all.

But, as usual, the idea doesn't work yet. A first try to copy the
existing mailbox "thorsten" to "thorsten_neu" leads to the following:

~~~~

mail:/var/spool/cyrus/mail/t/user#
/usr/local/sbin/cyrus/copy_user_mailbox.pl thorsten thorsten_neu
Successfully created inbox for thorsten_neu
Successfully set quota for thorsten_neu to 102400
"user.thorsten"
Copying folder user.thorsten to user.thorsten_neu
"user.thorsten.Archiv"
Copying folder user.thorsten.Archiv to user.thorsten_neu.Archiv
"user.thorsten.Drafts"
Copying folder user.thorsten.Drafts to user.thorsten_neu.Drafts
"user.thorsten.Mailinglisten"
Copying folder user.thorsten.Mailinglisten to
user.thorsten_neu.Mailinglisten
Error copying messages to user.thorsten_neu.Mailinglisten
"user.thorsten.Mailinglisten.Announces"
Not connected at /usr/local/sbin/cyrus/copy_user_mailbox.pl line 104
Error sending '24 SELECT user.thorsten.Mailinglisten.Announces' to IMAP:
  at /usr/local/sbin/cyrus/copy_user_mailbox.pl line 104
Copying folder user.thorsten.Mailinglisten.Announces to
user.thorsten_neu.Mailinglisten.Announces
Not connected at /usr/local/sbin/cyrus/copy_user_mailbox.pl line 129
Error sending '25 STATUS user.thorsten.Mailinglisten (MESSAGES)' to
IMAP:  at /usr/local/sbin/cyrus/copy_user_mailbox.pl line 129
Use of uninitialized value in numeric eq (==) at
/usr/local/sbin/cyrus/copy_user_mailbox.pl line 129.

[.. various errors at lines 104 and 129 ..]

Not connected at /usr/local/sbin/cyrus/copy_user_mailbox.pl line 152
Error sending '84 LOGOUT' to IMAP:  at
/usr/local/sbin/cyrus/copy_user_mailbox.pl line 152

~~~~

Please note, the additional output on the folder name results from
uncommenting line 92 of the script. The relevant sections look like
below [6]. Evaluating, why the connection to Cyrus gets lost while 
looping through the folders, a discovered "word too long" in 
/var/log/mail.log.

Well, in this case (my personal inbox) erasing various mails inside 
"user.thorsten.Mailinglisten" (~2200 mails) helped to make copying work. 
But regarding customer's inboxes this might not be first choice ;-)

In google I found [3], suggesting to recompile imapd. Alas, is there any 
way to modify the script instead of recompiling imapd to permit a larger 
value? Indeed I'd prefer to keep sarge's "2.1.18-1+sarge2".

Furthermore I tried to rename a mailbox via cyradm in advance, but this 
leads to the following:

~~~~

mail.myhostname.de> renm user.thorsten user.thorsten_orig
renamemailbox: Operation is not supported on mailbox

~~~~

The amount of results by google is rather poor, honestly spoken one 
result [4]. The presented workaround [5] contains quite a lot of work 
dealing manually with each subfolder. Is there any other way to rename a 
mailbox?

Or, going even further, is there any more clever attempt than the one 
above to solve the problem? Thanks in advance!

kind regards,
   Thorsten



[1] http://cyrusimap.web.cmu.edu/twiki/bin/view/Cyrus/Backup

[2] http://oregonstate.edu/~morgan/cyrus/public/copy_user_mailbox.pl

[3] http://www.irbs.net/internet/info-cyrus/0307/0439.html

[4] http://www.irbs.net/internet/info-cyrus/0502/0625.html

[5] http://www.irbs.net/internet/info-cyrus/0502/0628.html

[6] ~~~~
# Open connection for Mail::IMAPClient
my $imap1 = Mail::IMAPClient->new(Server => $prefs{'cyrushost'},
                                 User => $prefs{'cyrususer'},
                                 Password => $prefs{'cyruspass'},
                                 Uid => 1,
                                 Peek => 1,
                                 Buffer => 4096,
                                 Fast_io => 1,
                                 ) or die("Cannot connect to imap server
- $!");

my $search = $oldmailbox . ".*";
my $targetf;
my @folders;
foreach my $f (&getfolderlist($imap1, $oldmailbox)) {
print "$f\n"; # line 92

         # Strip quotes
         $f =~ s/"//g;

         # Add acl on source so we can read it
         $rv = $imap->set_acl($f, "cyrus", "lrswipcda"); # line 104
         if ($rv != 0) {
                 print "Error setting acl on $f: " . $imap->{'Error'} .
"\n";
                 next;
         }

         # Add acl on source so we can read it
         $rv = $imap->set_acl($f, "cyrus", "lrswipcda");
         if ($rv != 0) {
         # Get target folder name
         if ($f =~ /^user\.$oldusername(.*)$/) {
                 $targetf = "user.$newusername$1";
         }

         # Push all the folder names into an array so we can remove
         # the "cyrus" acls later
         push @folders, $f, $targetf;

         print "Copying folder $f to $targetf\n";

         # Create folder on target side if it does not exist
         if (! $imap->list($targetf)) {
                 # Created folders inherit acl of parent, so no need to
add cyrus acl
                 $rv = $imap->create($targetf);
                 if ($rv != 0) {
                         print "Error creating target folder $targetf: "
. $imap->{'Error'} . "\n";
                         next;
                 }
         }

         # Now copy the selected folder
         # Skip folders with zero messages or the copy will appear to error
         next if ($imap1->message_count == 0); # line 129

~~~~