sync_client bails out after 3 MAILBOXES need upgrading to USER in one run

[Bron Gondwana]

So I've got a huge log file, and it keeps bailing out trying to process it, but
at a different place each time.  Very odd.

Closer inspection shows that the do_sync code calls do_mailboxes up to 3 times
if there are errors, and if there is a mailbox that can be promoted to user
each time, does that.

This should be fine in the general case, but sort of sucks when there are a
lot of new users being created fast enough that sync_client gets more than three
events that need promoting in a single run.

I've attached my trivial solution (against CVS of last week some time), but I'm
thinking a better (as in, less wasteful) solution might be to not return an error
at all for a failed mailbox, but instead keep walking the entire tree, and then
generate a "USER" event for every mailbox that hasn't been marked yet.

Ken and David - is there a reason why you chose to pass a single "MAILBOXES"
command with multiple mailboxes to the backend rather than single mailbox
commands?  The little birdy in my head is whispering (it does that at 1am
after many hours of debugging) that it has something to do with supporting
renames.

Anyway - I think the real answer is to either have access to the user_list
deep down inside do_mailboxes so that things can be appended directly to it
at the time of error finding, or just not to mark those folders during the
mailboxes pass, so you can upgrade those users to a full USER sync.

My patch just says "if there has been any progress then that's good enough
for me - start over with the whole list of unmarked folders again without
incrementing the failure count.  It can't stave (always one more folder
marked per loop) but it sure could get slow if every folder was a subfolder
for a user that hadn't been "created" yet.

Regards,

Bron.
-- 
  Bron Gondwana
  brong at fastmail.fm

-------------- next part --------------
diff -ur --new-file cyrus-imapd-cvs.orig/imap/sync_client.c cyrus-imapd-cvs/imap/sync_client.c
--- cyrus-imapd-cvs.orig/imap/sync_client.c	2006-07-26 20:03:15.000000000 -0400
+++ cyrus-imapd-cvs/imap/sync_client.c	2006-08-26 10:45:14.000000000 -0400
@@ -2988,7 +2988,7 @@
     if (folder_list->count) {
 	int n = 0;
 	do {
-	    sleep(n*2);  /* XXX  should this be longer? */
+	    if (n) sleep(n*2);  /* XXX  should this be longer? */
 	    r = do_mailboxes(folder_list);
 	    if (r) {
 		/* promote failed personal mailboxes to USER */
@@ -3013,6 +3013,7 @@
 			       folder->name, userid);
 		    }
 		    free(userid);
+		    --n; /* we're still making progress */
 		}
 	    }
 	} while (r && (++n < SYNC_MAILBOX_RETRIES));