too many files in a directory

[David Lang]

On Fri, 5 Nov 2010, Ross Boylan wrote:

> My Cyrus 2.2.13 server (Debian stable) is running on an ext3
> filesystem.  One folder contains so many files that it keeps exceeding
> the limit ext3 can cope with (or at least, more than the directory index
> can handle).  I've been able to recover with e2fsck (I don't know if it
> rebalances the tree or makes more room), but clearly need to do
> something else.
>
> I have a narrow question and a broader one.  Narrowly, if I create some
> other folders and move some of the messages into them, will it help?  My
> understanding is that cyrus tries to avoid copying or moving message
> files around on disk, and so I suspect the files will continue to take
> up space in the original directory even if I move them.

if you copy them to the new folder, delete them from the old folder, and expunge 
them, then the directory slots will be freed up. I don't think that ext3 will 
actually shrink the directory, but it will re-use these available slots for new 
files.

> More broadly, any other suggestions for dealing with this situation?
> I'm also having very slow backup times, I think the result of the long
> time it takes to traverse the directory.  I might go to reiser3, which I
> used successfully on a different system.  I thought the reiser file
> system seemed like a bad long-term bet after the architect was jailed
> for murder.  But I see from the archives that people are still using it
> and having good experiences.

this is why I use XFS instead of ext3 :-)

ext4 is also better, but it's new enough that I don't use it for anything 
critical yet.

> Thanks for any advice.
> Ross
>
> P.S. For the record, the failures do not lose any messages; they result
> in messages going to my main inbox when there's an error on attempted
> delivery to the subfolder.

probably what's happening is that something is taking long enough that the 
delivery to the subfolder 'fails' and it falls back to delivering to the main 
inbox instead.

how many files do you have in the problem folders?

David Lang