Tunning for large number of files in INBOX

[Andrew Morgan]

On Wed, 29 Jun 2005, Michael Loftis wrote:

> --On June 29, 2005 4:30:06 PM -0400 Joel Nimety <jnimety at perimeterusa.com> 
> wrote:
>
>> Hello,
>> 
>> I'm trying to set up cyrus-imap as a backend for an email archiving
>> solution.  I'm creating one account on the imap server for each customer
>>   domain(s) we'll be archiving mail for.  I'm concerned that the number
>> of emails that will end up in each INBOX will reach some limit (ext3 fs
>> limit, practical limit, etc.)
>
> With EXT3 there are definite limits, not hard ones, but practical ones for 
> time to traverse/read the inode and list.  Use ReiserFS.  For Mail clients 
> most can't handle big folders because many of them are just POP3/NNTP clients 
> retrofitted to squak IMAP.  Get a real IMAP client like Mulberry that takes 
> advantage of server side sorting, threading, and searching to allow for 
> (nearly) limitless mailboxes but not download each and every header.
>
> With ReiserFS and UFS+Hashdirs (Linux and FreeBSD respectively) I personally 
> have many mailboxes that are well over 20k or 30k messages, and have a few in 
> the 200k range, haven't run into any performance problems. With EXT3 I had 
> serious problems in the 5k range, or less.

In the interest of completeness, under 2.6 linux kernels you can format an 
ext3 partition using the dir_index option.  This enables a hash tree index 
for directories that supposedly improves lookups with very large 
directories.  Here is the command I use to build my mail spool filesystem:

   mkfs -t ext3 -j -m 1 -O dir_index /dev/sdb1

I have not used other filesystems such as Reiser or XFS, so I cannot offer 
any performance comparisons.

 	Andy
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html