unexpunge segfaults with -l on some mailboxes

[Patrick Boutilier]

Patrick Boutilier wrote:
> Bron Gondwana wrote:
> 
> <snip>
>> Not so good.  Crap.  And this is on 2.3.13?  I don't see any changes
>> touching that code in the post 2.3.13 changelogs...
> 
> 
> Correct. We have been running 2.3.13 pretty much since it was released, 
> 2.3.12 before that, and 2.3.11 before that.
> 
>>>> I'm tempted to protect the code from crashing though... we don't
>>>> use unexpunge at FastMail, which is probably why I haven't already
>>>> done so.
>>>>
>>>> Something like the attached should do it.  I'll test it more
>>>> completely and commit it to CVS for 2.3.14 (since Ken hasn't
>>>> cut a release candidate yet!)
>>> Thanks for the patch.
>> Hey, don't use it though - it doesn't even compile!  It was a first
>> draft.  I'll give you a real patch soon... been working on doing it
>> _properly_ :)
> 
> Ok. :-)
> 
>>> Would ipurge be causing the corruption? We currently purge e-mails older  
>>> than 31 days on a weekly basis. I will turn that off for a bit (since  
>>> disk space is not as much of an issue as it used to be) and see if the  
>>> corruption returns.
>> Oooh... maybe.  I don't use ipurge.  Let me know what you find with
>> turning it off.  I've never even _looked_ at that code.
>>
> 
> The more I think of it, the more I believe that ipurge will be the 
> source of the problem. When I was manually checking for corruption (and 
> reconstructing the mailboxes that had problems) there would be no 
> corruption for 6 days. On the 7th day (always Sunday morning if I recall 
> correctly) corruption would reappear in many mailboxes. As it turns out, 
> our weekly ipurge ran on Saturday morning.
> 
> This will be real easy to test though. I will just run ipurge on a 
> subfolder of my mailbox and see if it corrupts it. :-)

I can duplicate the corruption by following steps similar to this report:

http://archives.devshed.com/forums/networking-100/cyr-expire-sigsegv-2146878.html


Current directory here is /var/imap/metadata/G/user/boutilpj/Testmail


    Clean mailbox:


[cyrus at student2 Testmail]$ ll
total 60
-rw------- 1 cyrus cyrus 49612 Mar 13 10:03 cyrus.cache
-rw------- 1 cyrus cyrus   189 Mar 13 10:03 cyrus.header
-rw------- 1 cyrus cyrus  3792 Mar 13 10:03 cyrus.index


    Quick way to show number of messages in the folder:



[cyrus at student2 Testmail]$ cat cyrus.cache |grep -a Prec|wc -l
42


    Delete 11 messages through IMAP client:

[cyrus at student2 Testmail]$ ll
total 64
-rw------- 1 cyrus cyrus 49612 Mar 13 10:03 cyrus.cache
-rw------- 1 cyrus cyrus  1064 Mar 13 10:05 cyrus.expunge
-rw------- 1 cyrus cyrus   189 Mar 13 10:03 cyrus.header
-rw------- 1 cyrus cyrus  2824 Mar 13 10:05 cyrus.index


    Run ipurge with a number high enough not to delete anything. I purge 
     only sees 31 messages and removes them from cyrus.cache:


[cyrus at student2 Testmail]$ /usr/local/cyrus/bin/ipurge -f -d 2333 
user.boutilpj.Testmail
Working on user.boutilpj.Testmail...
total messages                   31
total bytes                      539400
Deleted messages                 0
Deleted bytes                    0
Remaining messages               31
Remaining bytes                  539400

[cyrus at student2 Testmail]$ ll
total 48
-rw------- 1 cyrus cyrus 36652 Mar 13 10:08 cyrus.cache
-rw------- 1 cyrus cyrus  1064 Mar 13 10:05 cyrus.expunge
-rw------- 1 cyrus cyrus   189 Mar 13 10:03 cyrus.header
-rw------- 1 cyrus cyrus  2824 Mar 13 10:08 cyrus.index

[cyrus at student2 Testmail]$ cat cyrus.cache |grep -a Prec|wc -l
31


    Run cyr_expire and corruption occurs:

[cyrus at student2 Testmail]$ /usr/local/cyrus/bin/cyr_expire -E 333 -X 7 
-p user.boutilpj.Testmail -v
Expunging deleted messages in mailboxes older than 7 days

Expunged 0 out of 33 messages from 1 mailboxes
[cyrus at student2 Testmail]$ ll
total 96
-rw------- 1 cyrus cyrus 36652 Mar 13 10:08 cyrus.cache
-rw------- 1 cyrus cyrus 37852 Mar 13 10:15 cyrus.cache.NEW
-rw------- 1 cyrus cyrus  1064 Mar 13 10:05 cyrus.expunge
-rw------- 1 cyrus cyrus   272 Mar 13 10:15 cyrus.expunge.NEW
-rw------- 1 cyrus cyrus   189 Mar 13 10:03 cyrus.header
-rw------- 1 cyrus cyrus  2824 Mar 13 10:08 cyrus.index
-rw------- 1 cyrus cyrus  2824 Mar 13 10:15 cyrus.index.NEW


Mar 13 10:15:40 student2 cyr_expire[8725]: IOERROR: reading cache record 
for user.boutilpj.Testmail: item 1 has bogus offset 576450680 of 36652 
for 2/11; mailbox needs a reconstruct




> 
>> Bron.
> 
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

-------------- next part --------------
A non-text attachment was scrubbed...
Name: boutilpj.vcf
Type: text/x-vcard
Size: 286 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090313/13e7c468/attachment.vcf