ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]

[David Lang]

On Tue, 16 Nov 2010, Adam Tauno Williams wrote:

> On Tue, 2010-11-16 at 11:25 -0800, David Lang wrote:
>>> I don't actually know what sort of problems I'm referring to, hence the
>>> question.  The big problem I can imagine would be opendir() and
>>> readdir() with a huge number of files in a directory, but the cyrus code
>>> doesn't appear to do that in a lot of places that would matter to a user
>>> (deleting an entire folder, delete sieve scripts, etc) in the course of
>>> normal operations.
>> This is depends on what filesystem you are useing, I have mailboxes with hundreds
>> of thousands of messages in them on XFS and have no problems, but on ext3 I
>> start seeing slowdowns with a bit over ten thousand messages.
>
> Was dir_index enabled on that ext3 filesystem?  Prior to dir-index ext3
> was very slow for large folders. dir_index is not enabled by default in
> ext3.

yes, even with dir-index I see slowdown on large, busy folders. not as bad as 
without them, but still there.

without dir-index a folder that at one time had 10K files in it becomes 
unusuably slow forever, with dir-index the slowdown isn't as bad, but it's still 
there.

remember that dir-index only helps for the case where you are looking for a 
single file, if you are walking the entire directory it has little, if any 
effect.

XFS is slow deleting large numbers of files as noted by others, but delayed 
expunge sidesteps that issue.

David Lang