Fwd: Re: Does anyone allow unlimited or extremely large quotas?

[Dave McMurtrie]

I didn't realize that I only responded to Rob here.  Perhaps my 
additional information will shed some light on the kind of information 
I'm looking for.

-------- Original Message --------
Subject: Re: Does anyone allow unlimited or extremely large quotas?
Date: Tue, 16 Nov 2010 07:06:53 -0500
From: Dave McMurtrie <dave64 at andrew.cmu.edu>
To: Rob Mueller <robm at fastmail.fm>

On 11/16/2010 06:45 AM, Rob Mueller wrote:
>
>> This may be slightly off-topic, so apologies in advance. Is there
>> anyone out there who allows unlimited quota for their users or provides
>> extremely large quotas when asked for?
>
> What do you consider extremely large? And what sort of problems are you
> referring to?

I don't actually know what sort of problems I'm referring to, hence the
question.  The big problem I can imagine would be opendir() and
readdir() with a huge number of files in a directory, but the cyrus code
doesn't appear to do that in a lot of places that would matter to a user
(deleting an entire folder, delete sieve scripts, etc) in the course of
normal operations.

My manager has asked me how well Cyrus will cope with large (100GB+) or
unlimited quotas.  My answer to him was that it should be okay, but I
have very little practical experience with such so I wanted to ask on
the list.

> The usual issue is just the huge number of emails and thus files that
> accumulate. Creating a fresh replica, body searching, reconstructing,
> etc all take quite a bit of time because of the large amount of random
> IOs. Apart from that, everything does actually work ok...

The only issue we ever had was with a bboard that our network group
sends automated system messages to.  Something in their environment went
haywire and we ended up with ~1.5 million messages in that bboard.  They
were unable to find a client that was willing to deal with the folder to
be able to clean it up.  I was able to connect using imtest and SELECT
and FETCH messages without any problems, though.  I also recall that
replication was broken by this folder, but I don't remember exactly why.

So basically, I have this tiny amount of practical experience that tells
me if there are 1.5 million files in a single folder, clients may be
unhappy and replication may break but the server was still generally
working.

Any anecdotal evidence I can collect in addition to this would be
helpful for me to be able to go back to my manager with.

Thanks!

Dave