Expunged emails after upgrade from 2.3.16 to 2.4.5

[Stephen Carr]

Dear Bron

Yes ant  = any  - see comments below as well.

Any chance the reconstruct runs could have had an impact ?

I have restored all the emails on one account and have run reconstruct - 
r -f -G on that account that had about 70,000 emails rediscovered.

Will watch what happens overnight.

PS Most users POP theie emails off the server only a few use pure IMAP.

Thanks again.
Stephen Carr


Bron Gondwana wrote:
> On Mon, Dec 06, 2010 at 08:22:57AM +1030, Stephen Carr wrote:
>   
>> After fixing sync problems with the help of Bron this morning I
>> discovered that ant email older that 150 days has been expunged.
>> What alerted me was the size of the Full backup done one day after
>> the upgrade.
>>     
>
> "ant email" - do you mean "any email"?
>  
>   
>> Here is part of the log
>>
>> Dec  5 04:00:02 brooks cyr_expire[23141]: Expunged 0 out of 208877
>> messages from 1453 mailboxes
>>     
>
> No messages expired.
>   

I did an ls -lR of the users directory and this morning got 77,179 where 
as before it was around  230,000

I restored the maildata from the backup prior to the upgrade to another 
system as a check and ran the command below result was 235,362

The command is

ls -lR /maildata/cyrus/user/ | grep " [1-9]*\." | wc -l
>  
>   
>> Dec  5 21:00:00 brooks ipurge[4410]: Expunged 2 messages from
>> user.user1.Trash
>> Dec  5 21:00:00 brooks ipurge[4410]: Expunged 1 messages from
>> user.user2.Trash
>> Dec  5 21:00:01 brooks ipurge[4410]: Expunged 25 messages from
>> user.user3.Trash
>> ...
>> Dec  6 03:00:00 brooks ipurge[18145]: Expunged 1 messages from
>> user.user4.quarantine
>> Dec  6 03:00:00 brooks ipurge[18145]: Expunged 5 messages from
>> user.user5.quarantine
>> ....
>> Dec  6 04:00:00 brooks ipurge[20370]: Expunged 95 messages from
>> user.support.Bacula
>>     
>
> Lots of messages cleaned up by ipurge.
>   

Some of these emails are temporary especially from Bacula where I get 
150+ emails a day as I backup clients and servers.
>   
>> Dec  6 04:00:02 brooks cyr_expire[20368]: Expunged 0 out of 113053
>> messages from 1453 mailboxes
>>     
>
> Still no messages expired.
>
>   
>> Notice the 208877 is now 113053 and I suspect it now more like 77000
>> since it is 2 days after the upgrade.
>>
>> Is there a default Expunge ?
>>     
>
> It's looking to me more like ipurge used to be broken, and now it's
> working!
>
>   
>> expunge_mode: immediate
>> delete_mode: immediate
>>     
>
> I strongly recommend removing the "expunge_mode: immediate" and either
> setting it explicitly to "expunge_mode: default" or even just not
> specifying it.  "expunge_mode: immediate" doesn't work nicely with
> replication - you wind up having to run inefficient sync all the time
> because it cleaned out the expunge information too quickly.
>
>   
Change ran kill -HUP of master process ID.

>> sync_password: 30cyrus51
>>     
>
>   

Opps will change

> In general I wouldn't recommend posting this bit to a public
> mailing list...
>
>   
>>   # this is only necessary if using duplicate delivery suppression,
>>   # Sieve or NNTP
>>   delprune      cmd="cyr_expire -E 3" at=0400
>>     
>
> You don't have any "-X" in there, which means that delayed expunge
> will be running immediately for all mailboxes.  Again, -X 1 along
> with the expunge_mode changes above will make your replication
> experience nicer.
>   
Updated  cyrus config
>  
>   
>>   trashprune1	cmd="ipurge -f -d 21 user.%.Trash" at=2100
>>   trashprune2	cmd="ipurge -f -d 21 user.%.trash" at=2130
>>
>>   # Purge quarantine mailboxes older than 14 days at 3 am
>>   purgespam	cmd="ipurge -f -d 14 user.%.quarantine" at=0300
>>   purgespam2	cmd="ipurge -f -d 2 user.%.spam" at=0330
>>   purgebacula	cmd="ipurge -f -d 28 user.support.Bacula" at=0400
>>     
>
> Yeah, it's those ipurge commands that are doing it.  I suspect
> there used to be bugs in ipurge with immediate expunge, and
> you just didn't notice that it wasn't actually being cleaned up
> because you didn't have a -X specified in you cyr_expire...
>
> Bron.
>
>   

-- 
Stephen Carr
School of Civil, Environmental & Mining Engineering
The University of Adelaide
Adelaide, South Australia, 5005
Australia
Tel +618-8303-4313
Fax +618-8303-4359

Email sgcarr at civeng.adelaide.edu.au

CRICOS Provider Number 00123M
----------------------------------------------------------- 
This email message is intended only for the addressee(s) and
contains information that may be confidential and/or copyright.
If you are not the intended recipient please notify the sender 
by reply email and immediately delete this email. Use, disclosure 
or reproduction of this email by anyone other than the intended 
recipient(s) is strictly prohibited. No representation is made 
that this email or any attachments are free of viruses. Virus 
scanning is recommended and is the responsibility of the recipient.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20101206/b58951af/attachment.html