backup rsync

[Bron Gondwana]

We have an option called 'auditlog' now, which logs every append, regardless of where it came from, along with the sha1 of the spool file, and of course the mailbox name and UID.  From that, you could calculate exactly which emails were new.



I keep talking of writing a real incremental dump mode for Cyrus, but I haven't had a chance yet :(



Bron.





On Mon, Sep 8, 2014, at 02:28 PM, Stephen Ulmer wrote:

I think at that point we may not have cared… We did still run TSM incrementals, basically back-to-back, but those didn’t finish in 24 hours. We needed a better-effort way to get most message for a disaster.



I looked for the patch a bit ago (not that it would apply, but I would at least see what I had changed!) but couldn’t find it. I haven’t been employed by that organization for just over 8 years now, so it’s nowhere in my cache.

The general point of my comment was: If you’re backing up files, looking for them is expensive. If you can get Cyrus to tell you what it changed, then you don’t have to ask the filesystem — which usually involves asking every file. The other option would be to use a filesystem that supports DMAPI, and write a DMAPI application that kept track of changed files and added them to a backup queue. It seems like having Cyrus log the delivery would have other benefits, though, and would work on any filesystem. I’m not currently a Cyrus admin, though, so it could already do that for all I know. :)

Liberty,

--
Stephen


On Sep 7, 2014, at 10:03 PM, Bron Gondwana <[1]brong at fastmail.fm> wrote:

Did you back up Sent folders too?


On Mon, Sep 8, 2014, at 11:48 AM, Stephen Ulmer wrote:

A long time ago for a much older version of Cyrus, we hacked lmtpd (I think, it’s been years) to log the messages as it wrote them down. Then we just processed the log every hour or so to backup only those files. That saved having to traverse the entire in ode tree most of the time.


--
Stephen


On Sep 6, 2014, at 7:32 PM, Bron Gondwana <[2]brong at fastmail.fm> wrote:

No, no - we do replication.  Replication rocks.

You could easily stop the replica and take a snapshot of that, but our real backup solution is much more evil.  I've posted it to this list before, but it's basically a perl daemon which knows far too much about how Cyrus locks its data files.  It actually reads and parses cyrus.index files to work out what it needs to do.

Bron.

On Sun, Sep 7, 2014, at 04:50 AM, Marcus Schopen wrote:

Hi Bron,



Am Samstag, den 06.09.2014, 22:17 +1000 schrieb Bron Gondwana:

  That's what we do :)

Thanks for your feedbeek. What's your workaround for not stopping cyrus

before taking a lvm snapshot and run rsnapshot?



Ciao

Marcus











--
 Bron Gondwana
[3]brong at fastmail.fm
----
Cyrus Home Page:[4]http://www.cyrusimap.org/
List Archives/Info:[5]http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
[6]https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus



--
Bron Gondwana
[7]brong at fastmail.fm





--
Bron Gondwana
brong at fastmail.fm

References

1. mailto:brong at fastmail.fm
2. mailto:brong at fastmail.fm
3. mailto:brong at fastmail.fm
4. http://www.cyrusimap.org/
5. http://lists.andrew.cmu.edu/pipermail/info-cyrus/
6. https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
7. mailto:brong at fastmail.fm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20140908/9a1b010f/attachment.html