cyrus and 500k to >1M mails per inbox

[Rob Mueller]

>> > It's quite likely, that these mailboxes will grow to 500000 or even 
>> > more then
>> >  1M mails per mailbox.
>> >
>> > Does anybody have experience with such big mailboxes?
>>
>> Is the I/O cost of message adding relative to O(n), n being the number of 
>> msgs
>> already in the mailbox, or is it independent thereof ?
>
> Stacks of stuff is O(n), sorry.  It's not really an ideal way to do 
> things.
> You'd be better off archiving per month for each mailbox and keeping the
> messages counts to a reasonable number.

Well it's not quite as bad as that.

If the mailbox is mostly just appended to, then it's not a problem. The 
append code just appends to cyrus.index and cyrus.cache I believe (Bron?), 
it doesn't have to read the cyrus.* files each time.

We have actually seen a mailbox with > 1M messages at FastMail, and I 
remember that I could view it ok. However, I'll add a few caveats around 
that.

1. Our web interface has been highly optimised over time to make sure it 
deals ok with large UID lists. I'm not sure how many email clients will 
handle viewing an IMAP folder with 1M messages
2. You need a good filesystem as well, because there will be 1M files in a 
directory. We use reiserfs3, and again, it has no problem with that many 
files in a directory, mileage with other filesystems might vary.

Rob