<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Dear All<br>
<br>
Mea Culpa !!<br>
<br>
I think I was assuming that replication would work the same in 2.4.5
as it was in 2.3.16 - but it has had a major rewrite.<br>
<br>
I think the problem is I had this in the replica's cyrus.conf<br>
<br>
purgeall cmd="ipurge -f -d 150" at=2200<br>
<br>
So it impacted on the client / master but the replica the logs report<br>
<br>
Dec 5 01:00:02 proxy cyr_expire[25067]: Expunged 0 out of 640465
messages from 1479 mailboxes<br>
Dec 6 01:01:27 proxy cyr_expire[29917]: Expunged 0 out of 588837
messages from 1431 mailboxes<br>
<br>
The reason I suspect this has an impact is the -d 150 and I now suspect
what happens on the replica is reflected on the master.<br>
I had this parameter as I did not want the amount of emails to get
massive - if an email was more than 150 days old it was lost.<br>
<br>
Now I know I was not using the replica as a true "image" of the master
in that I have in the replica's imapd.conf the following<br>
<br>
expunge_mode: delayed<br>
delete_mode: delayed<br>
<br>
The reason for this is that I have had instances of users
misconfiguring POP or deleting emails and then emptying Trash and
asking where is the email from John Citizen.<br>
I have had to restore whole mail folders for a few users.<br>
<br>
So I was using the replica as a emergency backup store of emails.<br>
<br>
What do you recommend ?<br>
<br>
I have disabled the purgeall command.<br>
<br>
Thanks<br>
Stephen Carr<br>
<br>
<br>
<br>
Stephen Carr wrote:
<blockquote cite="mid:4CFC17BB.3030404@civeng.adelaide.edu.au"
type="cite">
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
Dear Bron<br>
<br>
Yes ant = any - see comments below as well.<br>
<br>
Any chance the reconstruct runs could have had an impact ?<br>
<br>
I have restored all the emails on one account and have run reconstruct
- r -f -G on that account that had about 70,000 emails rediscovered.<br>
<br>
Will watch what happens overnight.<br>
<br>
PS Most users POP theie emails off the server only a few use pure IMAP.<br>
<br>
Thanks again.<br>
Stephen Carr<br>
<br>
<br>
Bron Gondwana wrote:
<blockquote cite="mid:20101205222110.GA5592@brong.net" type="cite">
<pre wrap="">On Mon, Dec 06, 2010 at 08:22:57AM +1030, Stephen Carr wrote:
</pre>
<blockquote type="cite">
<pre wrap="">After fixing sync problems with the help of Bron this morning I
discovered that ant email older that 150 days has been expunged.
What alerted me was the size of the Full backup done one day after
the upgrade.
</pre>
</blockquote>
<pre wrap=""><!---->
"ant email" - do you mean "any email"?
</pre>
<blockquote type="cite">
<pre wrap="">Here is part of the log
Dec 5 04:00:02 brooks cyr_expire[23141]: Expunged 0 out of 208877
messages from 1453 mailboxes
</pre>
</blockquote>
<pre wrap=""><!---->
No messages expired.
</pre>
</blockquote>
<br>
I did an ls -lR of the users directory and this morning got 77,179
where as before it was around 230,000<br>
<br>
I restored the maildata from the backup prior to the upgrade to another
system as a check and ran the command below result was 235,362<br>
<br>
The command is <br>
<br>
ls -lR /maildata/cyrus/user/ | grep " [1-9]*\." | wc -l<br>
<blockquote cite="mid:20101205222110.GA5592@brong.net" type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">Dec 5 21:00:00 brooks ipurge[4410]: Expunged 2 messages from
user.user1.Trash
Dec 5 21:00:00 brooks ipurge[4410]: Expunged 1 messages from
user.user2.Trash
Dec 5 21:00:01 brooks ipurge[4410]: Expunged 25 messages from
user.user3.Trash
...
Dec 6 03:00:00 brooks ipurge[18145]: Expunged 1 messages from
user.user4.quarantine
Dec 6 03:00:00 brooks ipurge[18145]: Expunged 5 messages from
user.user5.quarantine
....
Dec 6 04:00:00 brooks ipurge[20370]: Expunged 95 messages from
user.support.Bacula
</pre>
</blockquote>
<pre wrap=""><!---->
Lots of messages cleaned up by ipurge.
</pre>
</blockquote>
<br>
Some of these emails are temporary especially from Bacula where I get
150+ emails a day as I backup clients and servers.<br>
<blockquote cite="mid:20101205222110.GA5592@brong.net" type="cite">
<pre wrap=""> </pre>
<blockquote type="cite">
<pre wrap="">Dec 6 04:00:02 brooks cyr_expire[20368]: Expunged 0 out of 113053
messages from 1453 mailboxes
</pre>
</blockquote>
<pre wrap=""><!---->
Still no messages expired.
</pre>
<blockquote type="cite">
<pre wrap="">Notice the 208877 is now 113053 and I suspect it now more like 77000
since it is 2 days after the upgrade.
Is there a default Expunge ?
</pre>
</blockquote>
<pre wrap=""><!---->
It's looking to me more like ipurge used to be broken, and now it's
working!
</pre>
<blockquote type="cite">
<pre wrap="">expunge_mode: immediate
delete_mode: immediate
</pre>
</blockquote>
<pre wrap=""><!---->
I strongly recommend removing the "expunge_mode: immediate" and either
setting it explicitly to "expunge_mode: default" or even just not
specifying it. "expunge_mode: immediate" doesn't work nicely with
replication - you wind up having to run inefficient sync all the time
because it cleaned out the expunge information too quickly.
</pre>
</blockquote>
Change ran kill -HUP of master process ID.<br>
<br>
<blockquote cite="mid:20101205222110.GA5592@brong.net" type="cite">
<blockquote type="cite">
<pre wrap="">sync_password: 30cyrus51
</pre>
</blockquote>
<pre wrap=""><!---->
</pre>
</blockquote>
<br>
Opps will change<br>
<br>
<blockquote cite="mid:20101205222110.GA5592@brong.net" type="cite">
<pre wrap="">In general I wouldn't recommend posting this bit to a public
mailing list...
</pre>
<blockquote type="cite">
<pre wrap=""> # this is only necessary if using duplicate delivery suppression,
# Sieve or NNTP
delprune cmd="cyr_expire -E 3" at=0400
</pre>
</blockquote>
<pre wrap=""><!---->
You don't have any "-X" in there, which means that delayed expunge
will be running immediately for all mailboxes. Again, -X 1 along
with the expunge_mode changes above will make your replication
experience nicer.
</pre>
</blockquote>
Updated cyrus config<br>
<blockquote cite="mid:20101205222110.GA5592@brong.net" type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap=""> trashprune1 cmd="ipurge -f -d 21 user.%.Trash" at=2100
trashprune2 cmd="ipurge -f -d 21 user.%.trash" at=2130
# Purge quarantine mailboxes older than 14 days at 3 am
purgespam cmd="ipurge -f -d 14 user.%.quarantine" at=0300
purgespam2 cmd="ipurge -f -d 2 user.%.spam" at=0330
purgebacula cmd="ipurge -f -d 28 user.support.Bacula" at=0400
</pre>
</blockquote>
<pre wrap=""><!---->
Yeah, it's those ipurge commands that are doing it. I suspect
there used to be bugs in ipurge with immediate expunge, and
you just didn't notice that it wasn't actually being cleaned up
because you didn't have a -X specified in you cyr_expire...
Bron.
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Stephen Carr
School of Civil, Environmental & Mining Engineering
The University of Adelaide
Adelaide, South Australia, 5005
Australia
Tel +618-8303-4313
Fax +618-8303-4359
Email <a moz-do-not-send="true" class="moz-txt-link-abbreviated"
href="mailto:sgcarr@civeng.adelaide.edu.au">sgcarr@civeng.adelaide.edu.au</a>
CRICOS Provider Number 00123M
-----------------------------------------------------------
This email message is intended only for the addressee(s) and
contains information that may be confidential and/or copyright.
If you are not the intended recipient please notify the sender
by reply email and immediately delete this email. Use, disclosure
or reproduction of this email by anyone other than the intended
recipient(s) is strictly prohibited. No representation is made
that this email or any attachments are free of viruses. Virus
scanning is recommended and is the responsibility of the recipient.
</pre>
<pre wrap="">
<hr size="4" width="90%">
----
Cyrus Home Page: <a class="moz-txt-link-freetext" href="http://www.cyrusimap.org/">http://www.cyrusimap.org/</a>
List Archives/Info: <a class="moz-txt-link-freetext" href="http://lists.andrew.cmu.edu/pipermail/info-cyrus/">http://lists.andrew.cmu.edu/pipermail/info-cyrus/</a></pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Stephen Carr
School of Civil, Environmental & Mining Engineering
The University of Adelaide
Adelaide, South Australia, 5005
Australia
Tel +618-8303-4313
Fax +618-8303-4359
Email <a class="moz-txt-link-abbreviated" href="mailto:sgcarr@civeng.adelaide.edu.au">sgcarr@civeng.adelaide.edu.au</a>
CRICOS Provider Number 00123M
-----------------------------------------------------------
This email message is intended only for the addressee(s) and
contains information that may be confidential and/or copyright.
If you are not the intended recipient please notify the sender
by reply email and immediately delete this email. Use, disclosure
or reproduction of this email by anyone other than the intended
recipient(s) is strictly prohibited. No representation is made
that this email or any attachments are free of viruses. Virus
scanning is recommended and is the responsibility of the recipient.
</pre>
</body>
</html>