cyr_expire does not delete files on disk with expunge_mode:delayed?

[Andreas Piesk]

Hello,

i just noticed a strange problem with expunge_mode:delayed which i cannot explain. it's probably a
lack of understanding on my side since i'm no cyrus-imap expert not even an email expert :)

OK, let me explain my problem. recently i discovered that my spam mailbox had 14.000 messages in it,
which surprised me necause i thought, cyr_expire deletes messages every day. i checked the mailbox
with cyrdump and according to cyrdump it was empty. so i ran some tests to understand what went wrong:

this is the mailbox for the test:

localhost.localdomain> info user.me.Junk
{user.me.Junk}:
  condstore: false
  duplicatedeliver: false
  expire: 7
  lastpop:
  lastupdate: 21-Nov-2010 01:10:29 +0100
  partition: default
  pop3newuidl: true
  sharedseen: false
  size: 0
  squat: false

first test with expunge_mode:delayed

$ ls -l /var/spool/imap/user/me/Junk/
total 24
-rw------- 1 cyrus mail   4 Nov 21 00:51 cyrus.cache
-rw------- 1 cyrus mail 153 Nov 21 00:51 cyrus.header
-rw------- 1 cyrus mail  96 Nov 21 00:51 cyrus.index

nothing in there. let's move a message in there:

$ ls -l /var/spool/imap/user/me/Junk/
total 32
-rw------- 2 cyrus mail 2952 Oct 25  2005 172936.
-rw------- 1 cyrus mail  836 Nov 21 00:58 cyrus.cache
-rw------- 1 cyrus mail  153 Nov 21 00:51 cyrus.header
-rw------- 1 cyrus mail  184 Nov 21 00:58 cyrus.index

and delete the message. status after deletion:

$ ls -l /var/spool/imap/user/me/Junk/
total 32
-rw------- 3 cyrus mail 2952 Oct 25  2005 172936.
-rw------- 1 cyrus mail  836 Nov 21 00:58 cyrus.cache
-rw------- 1 cyrus mail  153 Nov 21 00:51 cyrus.header
-rw------- 1 cyrus mail  184 Nov 21 01:01 cyrus.index

$ /usr/lib/cyrus-imapd/cyrdump user.me.Junk

<imapdump uniqueid="372a2ba243fb45a1">
  <mailbox-url>imap://pyramid/user.me.Junk</mailbox-url>
  <incremental-uid>0</incremental-uid>
  <nextuid>172937</nextuid>

  <uidlist>172936 </uidlist>

  <flags>
  <flag name="\Answered" user="*"></flag>
  <flag name="\Deleted" user="*">172936 </flag>
  <flag name="\Draft" user="*"></flag>
  <flag name="\Flagged" user="*"></flag>
  </flags>
</imapdump>

still there but flagged as deleted, which is OK, cyr_expire should take care:

$ /usr/lib/cyrus-imapd/cyr_expire -v -E 90 -X 0
Expunging deleted messages in mailboxes older than 0 days
expiring messages in user.me.Junk older than 7 days
expiring messages in user.me.VIRUS older than 7 days

but it is still there:

$ ls -l /var/spool/imap/user/me/Junk/
total 40
-rw------- 3 cyrus mail 2952 Oct 25  2005 172936.
-rw------- 1 cyrus mail  836 Nov 21 00:58 cyrus.cache
-rw------- 1 cyrus mail  184 Nov 21 01:04 cyrus.expunge
-rw------- 1 cyrus mail  153 Nov 21 00:51 cyrus.header
-rw------- 1 cyrus mail   96 Nov 21 01:04 cyrus.index

but cyrus thinks it's really gone:

$ /usr/lib/cyrus-imapd/cyrdump user.me.Junk

<imapdump uniqueid="372a2ba243fb45a1">
  <mailbox-url>imap://pyramid/user.me.Junk</mailbox-url>
  <incremental-uid>0</incremental-uid>
  <nextuid>172937</nextuid>

  <uidlist></uidlist>

  <flags>
  <flag name="\Answered" user="*"></flag>
  <flag name="\Deleted" user="*"></flag>
  <flag name="\Draft" user="*"></flag>
  <flag name="\Flagged" user="*"></flag>
  </flags>
</imapdump>

no messages, not even deleted ones.


now with expunge_mode_immediate:

$ ls -l /var/spool/imap/user/me/Junk/
total 32
-rw------- 2 cyrus mail 2952 Oct 24  2005 172937.
-rw------- 1 cyrus mail  836 Nov 21 01:07 cyrus.cache
-rw------- 1 cyrus mail  153 Nov 21 01:07 cyrus.header
-rw------- 1 cyrus mail  184 Nov 21 01:07 cyrus.index

delete message and check flags:

$ ls -l /var/spool/imap/user/me/Junk/
total 32
-rw------- 3 cyrus mail 2952 Oct 24  2005 172937.
-rw------- 1 cyrus mail  836 Nov 21 01:07 cyrus.cache
-rw------- 1 cyrus mail  153 Nov 21 01:07 cyrus.header
-rw------- 1 cyrus mail  184 Nov 21 01:08 cyrus.index

$ /usr/lib/cyrus-imapd/cyrdump user.me.Junk

<imapdump uniqueid="372a2ba243fb45a1">
  <mailbox-url>imap://pyramid/user.me.Junk</mailbox-url>
  <incremental-uid>0</incremental-uid>
  <nextuid>172938</nextuid>

  <uidlist>172937 </uidlist>

  <flags>
  <flag name="\Answered" user="*"></flag>
  <flag name="\Deleted" user="*">172937 </flag>
  <flag name="\Draft" user="*"></flag>
  <flag name="\Flagged" user="*"></flag>
  </flags>
</imapdump>

message still on disk but correctly flagged as deleted. after running cyr_expire:

$ ls -l /var/spool/imap/user/me/Junk/
total 24
-rw------- 1 cyrus mail   4 Nov 21 01:10 cyrus.cache
-rw------- 1 cyrus mail 153 Nov 21 01:07 cyrus.header
-rw------- 1 cyrus mail  96 Nov 21 01:10 cyrus.index

message was deleted.

it seems that with expunge_mode:delayed running cyr_expire does not delete the message files, why
and how can i change that? for now my workaround is to set expunge_mode:immediate.

if i miss something or my test is bogus, please let me know, i'm willing to learn :)

Thanks.

regards,
-ap