Geographically Redundant mail stores

[Lee]

We looked into a number of solutions to do what you're doing, and the 
best solution (within our budget) was to use block level syncing 
software like drbd (http://www.complang.tuwien.ac.at/reisner/drbd/) 
with heartbeat (linux-ha). Basically replicates a all data written to 
disc on the primary to the secondary and handles switching from primary 
to secondard when it detects that the primary is down.

L

On Tuesday, March 18, 2003, at 06:58 PM, Michael Fair wrote:

>> On Tue, 18 Mar 2003, Michael Fair wrote:
>>
>>> I'm doing some work on how to create a somewhat
>>> reliable geographically redundant mail system.
>>
>> Since I'm guessing you don't want to hear the reasons that this won't 
>> work
>> (synchronizing UIDs and flags, for example, is hard), I won't go into
>> that.
>
>
> Thanks.  I've given up on trying to provide a perfect/correct
> solution.  Instead I'm shooting for something more along the
> lines of being able to look at a live backup and then synchronizing
> any new mail that comes in.  State flags and other things above
> and beyond the email messages themselves are not a concern (but
> would be nice to have).
>
> The main problem is just that if the main server is ever unavailable
> communications come to a grinding halt.  Since we have people
> outside the office as well as in, we wanted some way for them to
> at least continue to send/receive new mail.
>
> I've been thinking about this problem for some time, and at the
> moment the best concepts I have going are:
> 1) Use Cyrus 2.2 and have the NNTP server sync the mailboxes.
>    (This does nothing for state flags and probably will not
>     help with the creation/deletion of new folders)
> 2) Create a "file locking server" that replaces the file
>    locking calls with something that is cross machine compatible
>    then use Coda, Intermezzo, or NFS to mirror the file store.
> 3) Turn Cyrus on the backup server off, use rsync to copy all
>    the files from one server to the other (making the UID/GIDs
>    match on the two servers shouldn't be a problem), then in the
>    event of a failure activate the Cyrus server, then flush the
>    MTA queue to deliver the queued mail to Cyrus (the queued mail
>    will that which has been delivered since primary failure).
>    It would look like I restored from a backup (which wouldn't
>    be too far from the truth).
>    (This is just admin intensive, and slow, and assumes that an
>     admin will always be available to manually make the changes)
> 4) Enhance Mailsync which does a good job at synchronizing
>     the mail stores for an individual user to do an entire
>     mail store.
>     (Without enhancement it needs to be setup per user.)
>     (With enhancment, by default an administrator cannot read
>      the emails within users mailboxes and therefore cannot
>      sync them)
> 5) Wait for people smarter than myself to add redundancy to
>    Cyrus directly (perhaps with a Group Communication Library
>    like Spread or something similar).
>
>
>
>
>>  Instead I'll answer your main question directly.
>
>>> My question was that the only user I know that
>>> can see the whole tree is an admin user.  But
>>> by default admin users can't select the mailboxes
>>> because they don't have the proper permissions.
>>
>> Admin users can authorize as any user they want.  So simply have the 
>> admin
>> user authorize as each user, and they can get to that mailbox with no
>> trouble.
>>
>> Note that if you SELECT a mailbox as a user, it *will* change the 
>> state of
>> \Recent flags for the user.
>
> Is there a reliable way to query the known list of users?
> I thinking of big loop:
>     foreach $user (@users) { syncMailbox($user); }
>
> I suppose I could just use the output of saldblistusers
> as STDIN input to the perl script (or the perl script
> could run it directly) since that's the backend I use.
> Or doing a List of the "user" folder one level deep.
>
>
> Any other ideas?
>
> How would you do it?
>
> The problem is:
> When the primary mail site is down, all email communication
> ceases despite the availability of other sites that could
> handle the load.
>
> In addition to allowing sending/receiving of new email,
> The system must integrate any new mail back into the main
>     site when it becomes available again.
> The system should allow people to see all their email and
>     folders older than some sane value (like 1 hour prior
>     to main site failure (shorter times preferred)).
> The system may (as added bonus points and extra special
>     kudos) preserve flag states for users email.
>
> Just as an FYI, the systems are Debian servers running
> Henrique's amazingly wonderful packages.
> The servers are Cyrus 2.1, Postfix 1.1.11, both integrated
> with sasldb for Authentication (SMTP AUTH is only allowed
> during a TLS session with Postfix - not that it matters).
> Site A has a 4MB link, Site B has 1.5MB link.
>
> -- Michael --
>