Information about your Cyrus installation

[Bron Gondwana]

On Wed, Jan 27, 2010 at 06:53:47PM -0500, Dave McMurtrie wrote:
> Hello Cyrus users,
> 
> First, please allow me to apologize for this slightly off-topic message. 
>   I'll ask that all replies be sent directly to me so the list doesn't 
> become too cluttered.
> 
> I'm scheduled to do a presentation about Cyrus IMAP at the upcoming 
> Jasig conference.  You can read more information about it here if you're 
> interested: 
> http://www.ja-sig.org/jasigconf/10spring/program.jsp?conf_id=jasig17
> 
> I'm looking for some basic feedback from those of you who are running 
> Cyrus IMAP.
> 
> * What type of environment do you support (commercial, education, etc)?
> 
> * How many users do you serve?
> 
> * Describe your installation.  (Hardware, OS, murder, replication, etc)
> 
> * Anything else interesting about your use of Cyrus?
> 
> * Whether or not you mind the name of your company/institution being 
> mentioned in my presentation.
> 
> I'd like to be able to spend a few minutes during the presentation 
> talking about the varied environments that Cyrus is currently running in.
> 
> Any information you can provide would be greatly appreciated.  Again, 
> please send your replies directly to me at dave64 at andrew.cmu.edu.

You can definitely talk about FastMail!  I've described everything in various
emails to the list over time, but I'm happy to do another outline for you if
you want.

Rough dot points go something like this:

* nginx frontend proxy for POP and IMAP
* sieve updates through web interface (rule building engine or "raw") only
* LMTP delivery via custom spam scanning and filtering lmtp proxy
* multiple instances of cyrus per machine:
  - divide machine up into N x 300Gb "slots"
  - separate metadata on high speed RAID1, emails on slow big RAID5
  - replication pairs spread over multiple machines so shutting down a single
    machine doesn't cause a huge spike on one other box.
  - use Linux-HA's IPAddr2 tool to bind service IP address to current master
    for each "store", so no need for external tools to track where the master
    is.
* custom saslauthd implementation used by both nginx and cyrus.
* custom backup system that understands cyrus.index format and can fetch
  just the files which it needs to do incremental backups.
* daemon which watches syslog file for cyrus issues and emails us about them.
  Can also run incremental squatter on folders getting body searches.
* cron task which checks for sync_client processes and sync log files and can
  automatically restart replication after a failure
* cron task which emails us if the size of sync log files gets too high.  Also
  stored in a database for display by status tools if over 10kb of logs
  present.
* weekly "checkreplication" run that logs in as every single user on each
  store and runs queries to ensure that exactly the same information is
  presented via IMAP by both the master and replica.  Has been very useful
  for tracking down replication bugs, and is a good overall guarantee that
  things are consistent!

And obviously - we're very actively involved in Cyrus development, and working
on changes to make Cyrus work even better in our environment.  The next big
project involves making replication require lower bandwidth and fewer round
trips - hence more usable between geographically dispersed datacentres.

Bron.