De-duping attachments

[Simon Matter]

> On Wed, September 15, 2010 10:01 am, Simon Matter wrote:
>
>> I guess much more efficient than a compressing filesystem would be a
>> compressing and de-duping filesystem or disk storage in this case. Has
>> anyone
>> tried this with a Cyrus message store with lots of "corporate message
>> data"
>> stored on it?
>
>
> Simon,
>
>
> The Cyrus server I hope to get online tomorrow evening holds 4.2 TB of
> mail
> and uses ZFS with maximal compression (gzip9) for the message files.
> (OS : Solaris 10)
>
> ZFS reports a compressratio of between 1.95 and 1.97 (we have nine
> partitions)
>
> A series of tests revealed our metadata can actually be compressed by a
> factor
> of 3.76 (!)
>
> Perhaps a two-university environment with 60,000+ users doesn't quite
> qualify
> as "corporate" enough but here you have our figures :-)

Eric, that looks of course interesting. With more "corporate" style I
means much less users but much bigger mailboxes. "Enforcing" quota in the
mulit GB range seems quite common these days. In such environment I expect
the compression ratio to increase.
But, the big question for me is how much filesystem / block level deduping
is going to shrink it? You said ZFS, did you consider testing its built in
deduping? (If its even there in Solaris 10?)

Simon