improving concurrency/performance

[David Lang]

On Mon, 7 Nov 2005, Sergio Devojno Bruder wrote:

> David Lang wrote:
>> (..)
>> I was recently doing some testing of lots of small files on the various 
>> filesystems, and I ran into a huge difference (8x) depending on what 
>> allocator was used for ext*. the default allocator changed between ext2 and 
>> ext3 (you can override it as a mount option) and when reading 1M files (10 
>> dirs of 10 dirs of 10 dirs of 1000 1K files) the time to read them went 
>> from ~5 min with the old allocator useed in ext2 to 40 min for the one 
>> that's the default for ext3.
>> 
>> David Lang
>> 
> (!!) Interesting. You said mount options? man mount man page only show me 
> data=journal, data=ordered, data=writeback, etcetera.
>
> How can I change that?

I found more things listed under /usr/src/linux/Documentation/filesystems

there are ext2.txt and ext3.txt files that list all the options available.

note that with my test all the files were created in order, it may be that 
if the files are created in a random order things would be different, so 
further testing is warrented.

I was doing tests to find how long it took to tar/untar these files  (with 
the tarball on a different drive).

here are the notes I made at the time. this was either 2.6.8.1 or 2.6.13.4 
(I upgraded about that time, but I'm not sure what the exact timeing was

note that on my cyrus server I actually use XFS with very large folders 
(20,000 mails in one folder) and it seems lightning fast. I haven't 
reconciled that observed bahavior with the tests listed below

the fact that on ext* filesystems I had tests range from 5 min to 80 min 
is somewhat scary. I did make sure to clear memory (by reading a file 
larger then available ram and doing a sync) between tests

David Lang

------------------------------------------------

on ext2 reading the tarball takes 53 seconds, createing the tar takes 10m, 
untarring it takes 4 min, copying it between drives on different 
controllers takes 62 seconds.

XFS looks bad for small files (13 min to untar, 9:41 to tar), but good for 
large files (47 sec to read)

reiserfs: reading the tar 43 sec 4:50 to tar 2:06 to untar (it was 
designed for tiny files and it appears to do that well)

   a couple tests I ran on reiserfs that I hadn't thought to run on the 
others, untaring on top of an existing directory took 7m, ls -lR took 
2:40, ls -flR (unsorted) took 2:40, find . -print took 21sec, rm -r took 
3m

jfs: 57 sec to read, untar 15:30, no other tests run

ext3: untar 3:30, read 64sec, tar 5:46, untarring on top of an existing 
directory 5:20, ls -lR 53 sec, ls -flR 47 sec, find . -print 7 sec

enabling dir_hash changed the read (36 sec) ls -flr (57 sec), ls -lR 61 
sec, find (25 sec), tar 81m!!!!!!!

turning off dir_hash and removing the journal (effectivly turning it into 
ext2 again) and mounting noatime
the tar goes to  34 min

mounting with oldalloc,noatime untar is 4:45, tar is 5:51.


-- 
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.
  -- C.A.R. Hoare