Vanishing quota information

[Ken Murchison]

Simon Matter wrote:

>>Simon Matter wrote:
>>
>>
>>>>Simon Matter wrote:
>>>>
>>>>
>>>>
>>>>>>Do you still have the user folder with their emails?
>>>>>>
>>>>>>I have seen this as well on 2.1.15, although not as frequently as your
>>>>>>seeing.  We have about 800 users and every so often a client does not
>>>>>>have
>>>>>>any quota information when they view the account in our Client
>>>>>>Interface.
>>>>>>When we check further their email directory is just gone.  We just
>>>>>>wrote
>>>>>>it
>>>>>>off as user error because there was not rhyme or reason for it
>>>>>>happening
>>>>>>(that we could see).
>>>>>>
>>>>>>Is that similar to what you are seeing?
>>>>>
>>>>>
>>>>>No, everything else is normal. And AFAIK we didn't have this problem
>>>>>until
>>>>>recent 2.2.x versions.
>>>>
>>>>Unfortunately, you're the only one to report this (yet) and I can't
>>>>reproduce this, so you'll have to try to narrow down what is going on
>>>>when this happens.
>>>>
>>>>Is it always the same mailboxes, or completely random?
>>>
>>>
>>>Seems completely random. I just changed it to dump core but none have
>>>shown up until now.
>>>BTW, could if have something to do with runnung quota -f against the
>>>runnung server?
>>
>>It shouldn't.  If it does, then something is broken.
> 
> 
> Unfortunately something is really broken and it reminds me the bug in
> 2.2.4 in hash.c. There was also a fildescriptor leak which should be fixed
> in 2.2.6, which is what I'm running.
> Unfortunately I can't seem to find what's going on but I know that quota
> -f still doesn't work as expected. I tried to run it today with the
> following result:
> 
> Have a look at user.442vk which is a used mailbox with a freshly created
> quota.
> 
> [root at imap01 root]# su - cyrus -c "/usr/lib/cyrus-imapd/quota -f"
> user.442vk: quota root user.442vk --> (none)
> user.442vk.Drafts: quota root user.442vk --> (none)
> user.442vk.Sent: quota root user.442vk --> (none)
> user.442vk.Trash: quota root user.442vk --> (none)
> user.freybmic: usage was 62407657, now 62410867
> user.graensam: usage was 423950920, now 423955183
> user.guethher: usage was 391475709, now 391477185
> user.hellerog: usage was 71319450, now 71325146
> user.hohmpet: usage was 489757782, now 489801516
> user.hugenjoe: usage was 158136268, now 158140557
> user.hummeman: usage was 368907340, now 368919134
> user.johomar: usage was 39755627, now 39757080
> user.kriegtho: usage was 474422484, now 474466229
> user.liebiute: usage was 2837508, now 2842548
> user.lopezmar: usage was 31271607, now 31291165
> user.maretala: usage was 6685492, now 6687794
> user.meyermau: usage was 284419148, now 284422339
> user.ottilwer: usage was 236995916, now 237014000
> user.442vk: removed
> user.raffafab: usage was 101802957, now 101969240
> user.rotolsyl: usage was 72383042, now 72392335
> user.rubinser: usage was 12245471, now 12251694
> 
> :::::::
> 
> user.gianeang: usage was 224158041, now 448316082
> user.gibbosim: usage was 93957125, now 187914250
> user.gigerkur: usage was 19138930, now 38277860
> user.gillearn: usage was 6575736, now 13151472
> quota: System I/O error Too many open files
> 
> The last lines are also interesting, because it seems that many quota are
> wrong. Unfortunately it seems that usually quota -f doesn't finish. It
> sometimes leaves lot's of *.NEW files around, the next run will remove
> them again.
> An strace of quota shows that there is no user.442vk.NEW file created and
> later removed but instead, the user.442vk file is removed:
> 
> q2:open("/var/lib/imap/quota/m/user.mattesim", O_RDWR) = 3
> q2:open("/var/lib/imap/quota/m/user.mattesim.NEW", O_RDWR) = 3
> q2:open("/var/spool/imap/m/user/mattesim/Trash/cyrus.header", O_RDWR) = 612
> q2:open("/var/spool/imap/m/user/mattesim/Trash/cyrus.index", O_RDWR) = 613
> q2:open("/var/spool/imap/m/user/mattesim/Trash/cyrus.cache", O_RDWR) = 614
> q2:open("/var/lib/imap/quota/m/user.mattesim", O_RDWR) = 612
> q2:stat64("/var/lib/imap/quota/m/user.mattesim", {st_mode=S_IFREG|0600,
> st_size=19, ...}) = 0
> q2:unlink("/var/lib/imap/quota/m/user.mattesim.NEW") = -1 ENOENT (No such
> file or directory)
> q2:open("/var/lib/imap/quota/m/user.mattesim.NEW", O_RDWR|O_CREAT|O_TRUNC,
> 0666) = 613
> q2:open("/var/lib/imap/quota/m/user.mattesim.NEW", O_RDWR) = 614
> q2:stat64("/var/lib/imap/quota/m/user.mattesim.NEW",
> {st_mode=S_IFREG|0600, st_size=19, ...}) = 0
> q2:unlink("/var/lib/imap/quota/m/user.mattesim.NEW") = 0
> q2:rename("/var/lib/imap/quota/m/user.mattesim.NEW",
> "/var/lib/imap/quota/m/user.mattesim") = -1 ENOENT (No such file or
> directory)
> 
> q2:open("/var/lib/imap/quota/q/user.442vk", O_RDWR) = 3
> q2:open("/var/spool/imap/q/user/442vk/cyrus.header", O_RDWR) = 4
> q2:open("/var/spool/imap/q/user/442vk/Drafts/cyrus.header", O_RDWR) = 4
> q2:open("/var/spool/imap/q/user/442vk/Sent/cyrus.header", O_RDWR) = 4
> q2:open("/var/spool/imap/q/user/442vk/Trash/cyrus.header", O_RDWR) = 4
> q2:open("/var/lib/imap/quota/q/user.442vk", O_RDWR) = 736
> q2:stat64("/var/lib/imap/quota/q/user.442vk", {st_mode=S_IFREG|0600,
> st_size=15, ...}) = 0
> q2:unlink("/var/lib/imap/quota/q/user.442vk") = 0
> 
> Any ideas?


I believe we have fixed both issues in CVS.  You need to fetch the 
latest lib/cyrusdb_quotalegacy.c and imap/quota.c

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html