Quotas and disk usage

[Michael Menge]

Quoting Eric Luyten <Eric.Luyten at vub.ac.be>:

> On Thu, January 14, 2010 8:26 pm, Wil Cooley wrote:
>> Adam Tauno Williams wrote:
>>
>>
>>> Some earlier versions of Cyrus had quota calculation issues with big
>>> quotas (2GB? 4GB?) or big mailboxes.  I think there is a blurb in WMOGAG
>>> about that, and what version fixed it.
>>
>> It happens with 2.2 crossing the 4GiB range, but not related to what he's
>> seeing.
>>
>>>> bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user1 Quota   % Used     Used
>>>> Root
>>>> 5242880       63  3303494 user.user1
>>>> bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user2 Quota   % Used     Used
>>>> Root
>>>> 41943040       93 39410366 user.user2
>>>> bash-3.00$ du -sh user1 7.6G   user1
>>>> bash-3.00$ du -sh user2 6.4G   user2
>>>>
>>>
>>> Do you have delayed expunge enabled?  If so actual disk size may very
>>> well exceed quota values - because the  
>>> expunged-but-not-yet-expired messages
>>> are still on the disk.
>>
>> Other possible contributors:
>> o Squatter indexes (my estimation is that they add 50%) o Directory sizes
>> themselves o Other cyrus.* metadata
>>
>> You can use something like this to calculate just the total (sorry, you need
>> GNU
>> du, find and xargs):
>>
>> find `mbpath user.user1` -type f -name \*. -print0 \ |xargs -0 du -cm|awk
>> '/total/ { tot=tot+$1 } END { print tot "MiB" }'
>>
>>
>> You need the awk if there are more files than possible for a single
>> invocation of "du". And you can calculate the metadata overhead by negating
>> the "-name" parameter:
>>
>>
>> find `mbpath user.user1` -type f \! -name \*. -print0 \ |xargs -0 du -cm|awk
>> '/total/ { tot=tot+$1 } END { print tot "MiB" }'
>
>
>
> On our previous Cyrus server (2.2 on Solaris 9 with UFS) I detected several
> multiply-linked message files (sitting in different folders/directories).
>
> Those will be counted twice/thrice/... towards the Cyrus quota but not 'du'.
> We were unable to find out how some IMAP clients were capable of having the
> server create them.
> (This being said, we never invested much effort into it :-)
>
>    % find `mbpath user/NNNNNN` -type f \! -links 1
>

It's not the client but the Server doing this. See man imapd.conf

singleinstancestore: 1
             If  enabled, imapd, lmtpd and nntpd attempt to only write one copy
             of a message per partition and create hard links, resulting  in  a
             potentially large disk savings.


>
> On our new server (2.3 on Solaris 10 with ZFS and basic filesystem  
> compression
> enabled) there is a ("zfs get compressratio") difference of 27 to 28 %
>
>
> Eric Luyten, Computing Centre VUB/ULB, postmaster.
>
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>



--------------------------------------------------------------------------------
M.Menge                                Tel.: (49) 7071/29-70316
Universität Tübingen                   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung          mail:  
michael.menge at zdv.uni-tuebingen.de
Wächterstraße 76
72074 Tübingen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5339 bytes
Desc: S/MIME Signatur
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20100115/b560e137/attachment-0001.bin