A beginner question about Murder

[Michel Sébastien]

Hi,
we use a modified "traditional" murder, i.e. without murder daemon, to host more than 2 million mailboxes (dozen million entries in mboxlist with folders)

We started to host 200.000 mailboxes on the "traditional" murder, configured with berkeley as cyrusdb backend due to lot of corruptions with default skiplist format (cyrus 2.3.9).
However we had a lot of locks and contentions with berkeleydb, so we migrated later to cyrus 2.3.13 on a skiplist backend. Unfortunately the more we hosted mailboxes the more we had production issues. Indeed database resync is not incremental and servers restart were increasingly longer. Moreover we had instability with skiplist backend around 7.8 million entries and failed to reach 8 million entries.

Finally we chose to develop a customized mysql backend. Front-ends and back-ends are all connected to the same DB but SQL results are rewritten with plugins to make them happy (return mailbox type 1 for proxy en mailbox type 0 for backend). Now there is no more latency or issue and mysql allows us to host more several dozens million of entries.

So, we are totally murderless. FEs are scalable thanks to mysql replication and BEs are almost high available (our mysql failover is read-only and don't permit acl or folder management, but it is acceptable for us).
We developed our owns mysql backend and we can share the source code if someone is interested (perhaps need some clean-up and code rewrite to adapt to the "official" sql backend)

Sébastien

-----Message d'origine-----
De : info-cyrus-bounces+sebastien.michel=atosorigin.com at lists.andrew.cmu.edu [mailto:info-cyrus-bounces+sebastien.michel=atosorigin.com at lists.andrew.cmu.edu] De la part de Wesley Craig
Envoyé : jeudi 9 septembre 2010 20:16
À : Andrew Morgan
Cc : Shuvam Misra; info-cyrus at lists.andrew.cmu.edu
Objet : Re: A beginner question about Murder

On 08 Sep 2010, at 16:41, Andrew Morgan wrote:
> Unfortunately, I've never setup a "unified" Murder, so I don't fully
> understand what the advantages and disadvantages of it compared to a
> "traditional" Murder.  Maybe someone else can jump in here with their
> experiences.

The main advantage is that BEs can proxy for other BEs, no FEs are required.  In m observation, the FEs are typically CPU bound, while the BEs are typically IO bound.  Since the BEs have an excess of CPU, it makes some sense to combine the two roles.

Other than the bugs one finds in new code, unified is logically superior to "traditional" and in the long run traditional should be retired.

In the future, I'd like to add "unified" replication, masterless murder, and auto failover. In my ideal minimum system, there are two machines, replicating to each other, providing high availability, proxying, etc.

:wes
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/



Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité du groupe Atos Origin ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis.

This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.