Delayed delete oddities

[David Mayo]

David Mayo wrote:

> We have just started transferring the first mailboxes onto our new Cyrus 
> IMAP 2.3.13 server running on Solaris 10. Delayed expunge seems to work 
> fine but I've noticed a strange phenomenon with delayed delete on mailboxes.
> 
> We have set up cyrus to delete expunged emails and deleted mailboxes 
> after 7 days:
> 
>    delprune      cmd="cyr_expire -E 3 -D 7 -X 7" at=0400
> 
> It appears to be deleting emails 7 days after they have been expunged, 
> so this looks fine:
>
> ...
> 
> It's just the first entry I'm concerned by:
> 
> Dec 11 04:00:00 imap-1.bath.ac.uk cyr_expire[29491]: [ID 956378 
> mail.notice] Expunged 4662 messages from DELETED.user.usera.BUCS.IT 
> discussion list.4B2123D3
> 
> This mailbox was only deleted yesterday:
> 
> Dec 10 16:37:39 sauber.bath.ac.uk imap[11321]: [ID 630590 mail.notice] 
> Deleted mailbox user.usera.BUCS.IT discussion list
> 
> The mailbox was moved to the right area of the IMAP tree but all the 
> messages were removed after just one day. The messages in the mailbox 
> were not marked for deletion after being moved. Worse still, it hasn't 
> even kept the messages on the filesystem:
> 
> imap-1 $ ls -la /mail/02/DELETED/user/usera/BUCS/IT\ discussion\ 
> list/4B2123D3/
> total 21441
> drwx------   2 cyrus    cyrus         12 Dec 11 04:00 .
> drwx------   3 cyrus    cyrus          3 Dec 10 16:37 ..
> -rw-------   1 cyrus    cyrus      27366 Feb 17  2009 4861.
> -rw-------   1 cyrus    cyrus       3151 May 31  2009 5007.
> -rw-------   1 cyrus    cyrus       2543 Jun 15 10:04 5025.
> -rw-------   1 cyrus    cyrus       2396 Jun 15 11:45 5026.
> -rw-------   1 cyrus    cyrus       2366 Jul 29 09:46 5084.
> -rw-------   1 cyrus    cyrus          4 Dec 11 04:00 cyrus.cache
> -rw-------   1 cyrus    cyrus        536 Dec 10 16:37 cyrus.expunge
> -rw-------   1 cyrus    cyrus        175 Dec 10 16:37 cyrus.header
> -rw-------   1 cyrus    cyrus         96 Dec 11 04:00 cyrus.index
> -rw-------   1 cyrus    cyrus    10734041 Dec 11 02:00 cyrus.squat
> 
> (squatter is run at 2am, cyr_expire at 4am, which explains the size of 
> cyrus.squat)
> 
> There were emails newer and older than those ones there, so I'm not sure 
> why those 5 were kept. Running unexpunge on the mailbox resulted in this:
> 
> imap-1 $ unexpunge -l "DELETED.user.usera.BUCS.IT discussion list.4B2123D3"
> UID: 4861
>          Size: 27366
>          Sent: Tue Feb 17 12:00:00 2009
>          Recv: Tue Feb 17 10:47:12 2009
>          Expg: Thu Dec 10 16:37:39 2009
> Segmentation Fault (core dumped)
> 
> [imap-1][/] # pstack /var/tmp/core.unexpunge
> core '/var/tmp/core.unexpunge' of 516:  unexpunge -l 
> DELETED.user.usera.BUCS.IT discussion list.4B2123D3
>   08058317 list_expunged (8046734, 8134898, 5, fea50000) + 186
>   080599df main     (3, 8047a5c, 8047a6c) + 941
>   08057f84 _start   (3, 8047b58, 8047b62, 8047b65, 0, 8047b9a) + 80
> 
> Is the deleted mailbox code considered reliable? Should we be pruning 
> these using a different method? I'm happy to supply more information if 
> it would lead to this being better understood and/or fixed.

I have been doing more testing on this by deleting some of my own 
mailboxes. Each time, all the messages are removed from the mailbox and 
most or all of them are deleted from the filesystem too by cyr_expire 
the first time it runs after the mailbox has been deleted.

I tried setting the 'expire' annotation to '99' on the DELETED hierarchy 
yesterday but the emails in the mailbox were still all deleted. Tonight 
I have removed the -D 7 switch from cyr_expire to see if that fixes things.

Is anyone else using delayed delete for mailboxes? Does cyr_expire leave 
the mailboxes untouched after a mailbox has been deleted? Are other 
people running cyr_expire from within cyrus.conf ? It would be nice to 
know if other people are seeing this or if it's something unique to our 
setup.

Regards,


Dave.

David Mayo
Networks/Systems Administrator
University of Bath Computing Services, UK

Tel: +44 1225 38 6046
Email: D.J.Mayo at bath.ac.uk