cyr_expire does not delete files on disk with expunge_mode:delayed?

[Kenneth Marshall]

We hit the same issue. You will need to run cyr_expire twice to
have items be removed correctly. Once as you are currently doing
and then a second time ignoring the mailbox annotations (the "-a"
option). This will actually run the delete.

Regards,
Ken

On Mon, Nov 22, 2010 at 09:55:46PM +0100, Andreas Piesk wrote:
> Hello,
> 
> i just noticed a strange problem with expunge_mode:delayed which i cannot explain. it's probably a
> lack of understanding on my side since i'm no cyrus-imap expert not even an email expert :)
> 
> OK, let me explain my problem. recently i discovered that my spam mailbox had 14.000 messages in it,
> which surprised me necause i thought, cyr_expire deletes messages every day. i checked the mailbox
> with cyrdump and according to cyrdump it was empty. so i ran some tests to understand what went wrong:
> 
> this is the mailbox for the test:
> 
> localhost.localdomain> info user.me.Junk
> {user.me.Junk}:
>   condstore: false
>   duplicatedeliver: false
>   expire: 7
>   lastpop:
>   lastupdate: 21-Nov-2010 01:10:29 +0100
>   partition: default
>   pop3newuidl: true
>   sharedseen: false
>   size: 0
>   squat: false
> 
> first test with expunge_mode:delayed
> 
> $ ls -l /var/spool/imap/user/me/Junk/
> total 24
> -rw------- 1 cyrus mail   4 Nov 21 00:51 cyrus.cache
> -rw------- 1 cyrus mail 153 Nov 21 00:51 cyrus.header
> -rw------- 1 cyrus mail  96 Nov 21 00:51 cyrus.index
> 
> nothing in there. let's move a message in there:
> 
> $ ls -l /var/spool/imap/user/me/Junk/
> total 32
> -rw------- 2 cyrus mail 2952 Oct 25  2005 172936.
> -rw------- 1 cyrus mail  836 Nov 21 00:58 cyrus.cache
> -rw------- 1 cyrus mail  153 Nov 21 00:51 cyrus.header
> -rw------- 1 cyrus mail  184 Nov 21 00:58 cyrus.index
> 
> and delete the message. status after deletion:
> 
> $ ls -l /var/spool/imap/user/me/Junk/
> total 32
> -rw------- 3 cyrus mail 2952 Oct 25  2005 172936.
> -rw------- 1 cyrus mail  836 Nov 21 00:58 cyrus.cache
> -rw------- 1 cyrus mail  153 Nov 21 00:51 cyrus.header
> -rw------- 1 cyrus mail  184 Nov 21 01:01 cyrus.index
> 
> $ /usr/lib/cyrus-imapd/cyrdump user.me.Junk
> 
> <imapdump uniqueid="372a2ba243fb45a1">
>   <mailbox-url>imap://pyramid/user.me.Junk</mailbox-url>
>   <incremental-uid>0</incremental-uid>
>   <nextuid>172937</nextuid>
> 
>   <uidlist>172936 </uidlist>
> 
>   <flags>
>   <flag name="\Answered" user="*"></flag>
>   <flag name="\Deleted" user="*">172936 </flag>
>   <flag name="\Draft" user="*"></flag>
>   <flag name="\Flagged" user="*"></flag>
>   </flags>
> </imapdump>
> 
> still there but flagged as deleted, which is OK, cyr_expire should take care:
> 
> $ /usr/lib/cyrus-imapd/cyr_expire -v -E 90 -X 0
> Expunging deleted messages in mailboxes older than 0 days
> expiring messages in user.me.Junk older than 7 days
> expiring messages in user.me.VIRUS older than 7 days
> 
> but it is still there:
> 
> $ ls -l /var/spool/imap/user/me/Junk/
> total 40
> -rw------- 3 cyrus mail 2952 Oct 25  2005 172936.
> -rw------- 1 cyrus mail  836 Nov 21 00:58 cyrus.cache
> -rw------- 1 cyrus mail  184 Nov 21 01:04 cyrus.expunge
> -rw------- 1 cyrus mail  153 Nov 21 00:51 cyrus.header
> -rw------- 1 cyrus mail   96 Nov 21 01:04 cyrus.index
> 
> but cyrus thinks it's really gone:
> 
> $ /usr/lib/cyrus-imapd/cyrdump user.me.Junk
> 
> <imapdump uniqueid="372a2ba243fb45a1">
>   <mailbox-url>imap://pyramid/user.me.Junk</mailbox-url>
>   <incremental-uid>0</incremental-uid>
>   <nextuid>172937</nextuid>
> 
>   <uidlist></uidlist>
> 
>   <flags>
>   <flag name="\Answered" user="*"></flag>
>   <flag name="\Deleted" user="*"></flag>
>   <flag name="\Draft" user="*"></flag>
>   <flag name="\Flagged" user="*"></flag>
>   </flags>
> </imapdump>
> 
> no messages, not even deleted ones.
> 
> 
> now with expunge_mode_immediate:
> 
> $ ls -l /var/spool/imap/user/me/Junk/
> total 32
> -rw------- 2 cyrus mail 2952 Oct 24  2005 172937.
> -rw------- 1 cyrus mail  836 Nov 21 01:07 cyrus.cache
> -rw------- 1 cyrus mail  153 Nov 21 01:07 cyrus.header
> -rw------- 1 cyrus mail  184 Nov 21 01:07 cyrus.index
> 
> delete message and check flags:
> 
> $ ls -l /var/spool/imap/user/me/Junk/
> total 32
> -rw------- 3 cyrus mail 2952 Oct 24  2005 172937.
> -rw------- 1 cyrus mail  836 Nov 21 01:07 cyrus.cache
> -rw------- 1 cyrus mail  153 Nov 21 01:07 cyrus.header
> -rw------- 1 cyrus mail  184 Nov 21 01:08 cyrus.index
> 
> $ /usr/lib/cyrus-imapd/cyrdump user.me.Junk
> 
> <imapdump uniqueid="372a2ba243fb45a1">
>   <mailbox-url>imap://pyramid/user.me.Junk</mailbox-url>
>   <incremental-uid>0</incremental-uid>
>   <nextuid>172938</nextuid>
> 
>   <uidlist>172937 </uidlist>
> 
>   <flags>
>   <flag name="\Answered" user="*"></flag>
>   <flag name="\Deleted" user="*">172937 </flag>
>   <flag name="\Draft" user="*"></flag>
>   <flag name="\Flagged" user="*"></flag>
>   </flags>
> </imapdump>
> 
> message still on disk but correctly flagged as deleted. after running cyr_expire:
> 
> $ ls -l /var/spool/imap/user/me/Junk/
> total 24
> -rw------- 1 cyrus mail   4 Nov 21 01:10 cyrus.cache
> -rw------- 1 cyrus mail 153 Nov 21 01:07 cyrus.header
> -rw------- 1 cyrus mail  96 Nov 21 01:10 cyrus.index
> 
> message was deleted.
> 
> it seems that with expunge_mode:delayed running cyr_expire does not delete the message files, why
> and how can i change that? for now my workaround is to set expunge_mode:immediate.
> 
> if i miss something or my test is bogus, please let me know, i'm willing to learn :)
> 
> Thanks.
> 
> regards,
> -ap
> ----
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>