Cyrus, clusters, GFS - HA yet again

[Adam Kramer]

On 10/28/06, Simon Matter <simon.matter at invoca.ch> wrote:
> Now I think you really mix things up. 1) AFAIK quota is a per user
> database which is updated whenever there is a change to the users mailbox.
> Cyrus only scans all mail for their size with you do a "quota -f" after
> something messed with your mailspool.

Individual messages aren't even scanned during a "quota -f". Each
mailbox index has a field containing the size of the mailbox, "quota
-f" rescans each index under the quota root and adds them up to get
the total. As best I've discovered, it requires a reconstruct of the
mailbox to scan each message's size and recreate the total in the
index.

This leads to interesting behavior if you run cyrus for a while with
32 bit quota support and then upgrade to 64 bit quota support. If a
user had a mailbox bigger than 4GB before the upgrade, the size of the
mailbox in the index will have wrapped at 2^32. After the upgrade if
they delete all that mail, you can wrap below zero and end up with a
mailbox that appears to be close to 2^64th in size.



> 2) You have to consider GFS volumes
> a local storage because it is usually on SAN which is also virtually local
> storage. It really has nothing to do with networked filesystems like NFS.
> AFAIK the trick with a GFS clustered Cyrus system is that you have two or
> more independant Cyrus servers sharing the same metadata and message store
> on the block device level, and not caring about each other, which means
> they all serve tha same mailboxes/users. IIRC there are people running
> Cyrus servers that way on other systems like Tru64 or Veritas cluster.
> I think you have mixed up block device level shared filesystems with NFS
> shared systems, which for example can be used for maildir based systems.


Has anyone documented running a high volume Cyrus setup on Linux with
a clustered filesystem?


-- 
-Adam