Quotas and disk usage

[Eric Luyten]

On Thu, January 14, 2010 8:26 pm, Wil Cooley wrote:
> Adam Tauno Williams wrote:
>
>
>> Some earlier versions of Cyrus had quota calculation issues with big
>> quotas (2GB? 4GB?) or big mailboxes.  I think there is a blurb in WMOGAG
>> about that, and what version fixed it.
>
> It happens with 2.2 crossing the 4GiB range, but not related to what he's
> seeing.
>
>>> bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user1 Quota   % Used     Used
>>> Root
>>> 5242880       63  3303494 user.user1
>>> bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user2 Quota   % Used     Used
>>> Root
>>> 41943040       93 39410366 user.user2
>>> bash-3.00$ du -sh user1 7.6G   user1
>>> bash-3.00$ du -sh user2 6.4G   user2
>>>
>>
>> Do you have delayed expunge enabled?  If so actual disk size may very
>> well exceed quota values - because the expunged-but-not-yet-expired messages
>> are still on the disk.
>
> Other possible contributors:
> o Squatter indexes (my estimation is that they add 50%) o Directory sizes
> themselves o Other cyrus.* metadata
>
> You can use something like this to calculate just the total (sorry, you need
> GNU
> du, find and xargs):
>
> find `mbpath user.user1` -type f -name \*. -print0 \ |xargs -0 du -cm|awk
> '/total/ { tot=tot+$1 } END { print tot "MiB" }'
>
>
> You need the awk if there are more files than possible for a single
> invocation of "du". And you can calculate the metadata overhead by negating
> the "-name" parameter:
>
>
> find `mbpath user.user1` -type f \! -name \*. -print0 \ |xargs -0 du -cm|awk
> '/total/ { tot=tot+$1 } END { print tot "MiB" }'



On our previous Cyrus server (2.2 on Solaris 9 with UFS) I detected several
multiply-linked message files (sitting in different folders/directories).

Those will be counted twice/thrice/... towards the Cyrus quota but not 'du'.
We were unable to find out how some IMAP clients were capable of having the
server create them.
(This being said, we never invested much effort into it :-)

   % find `mbpath user/NNNNNN` -type f \! -links 1


On our new server (2.3 on Solaris 10 with ZFS and basic filesystem compression
enabled) there is a ("zfs get compressratio") difference of 27 to 28 %


Eric Luyten, Computing Centre VUB/ULB, postmaster.