<div dir="ltr"><div><div>Thank you for your help, I managed to write a cyrus.index parser that checks all the effective mail UIDs (=filenames) and erases all the unnecessary files (previously deleted and thus not referenced in the index file).<br>
<br></div>Just in case someone faces the same issue someday, here is the script :<br>(instead of directly deleting the emails, it creates a bash file with all the rm commands, so it can be reviewed before, just in case something goes wrong...)<br>
<br></div>In my case, I used the script (cleanMailbox.sh) this way (to clean only the INBOXes) :<br><br>for userdir in `find /data/mail/imap/spool/*/user/ -maxdepth 1 -mindepth 1 -name "*"`; do echo $userdir; ~/adminScripts/cleanMailbox.sh "$userdir"; done;<br>
<div><div><br>and the script itself:<br>*************************************************************************************<br>#!/bin/bash<br></div><div># cleanMailbox.sh : remove all mailfiles not referenced in the cyrus.index file<br>
<br></div><div>if [ ! -d "$1" ]; then<br> echo "USAGE: argument must be a valid cyrus user mail directory"<br> exit 0<br>fi<br><br>INDEXPATH="$1/cyrus.index"<br><br>if [ ! -f "$INDEXPATH" ]; then<br>
echo "No cyrus.index in $1. ABORT"<br> exit 0<br>fi<br><br>#1- read start offset in cyrus.index at position 0x0c ==12 (4 bytes)<br>STARTOFFSET=`od --skip-bytes=12 --read-bytes=4 -An -t x1 "$INDEXPATH" | tr -d ' '`<br>
STARTOFFSET_10=`printf '%d\n' 0x"$STARTOFFSET"`<br>echo "messages start offset =0x$STARTOFFSET (=$STARTOFFSET_10)"<br><br>#2- read record size in cyrus.index at position 0x10 ==16 (4 bytes)<br>
RECORDSIZE=`od --skip-bytes=16 --read-bytes=4 -An -t x1 "$INDEXPATH" | tr -d ' '`<br>RECORDSIZE_10=`printf '%d\n' 0x"$RECORDSIZE"`<br>echo "message record size =0x$RECORDSIZE (=$RECORDSIZE_10)"<br>
<br>#3- read number of emails in cyrus.index at position 0x14 ==20 (4 bytes)<br>NBMESSAGES=`od --skip-bytes=20 --read-bytes=4 -An -t x1 "$INDEXPATH" | tr -d ' '`<br>NBMESSAGES_10=`printf '%d\n' 0x"$NBMESSAGES"`<br>
echo "number of emails in metadata=0x$NBMESSAGES (=$NBMESSAGES_10)"<br><br># ALL message UID (=filename) will be in cyrus.index at STARTOFFSET + i * RECORDSIZE <br># with i integer in [1..NBMESSAGES]<br># Build a reference file to keep track of mail filenames (UID.) that will NOT be deleted <br>
for i in `seq 0 $((NBMESSAGES_10 - 1))`;<br> do<br> MESSAGEOFFSET=$((STARTOFFSET_10 + i * RECORDSIZE_10))<br> UIDMESSAGE=`od --skip-bytes=$MESSAGEOFFSET --read-bytes=4 -An -t x1 "$INDEXPATH" | tr -d ' '`<br>
UIDMESSAGE_10=`printf '%d\n' 0x"$UIDMESSAGE"`<br> echo "$UIDMESSAGE_10". >> "$1"/KEEP_THIS.tmp<br>done<br><br># instead of deleting directly the undesirable files, build a command file with all the rm commands:<br>
echo cd "$1" >> /tmp/eraseOldMails.sh<br><br>cd "$1"<br># for all email files <br>for mailfile in `ls *\.`;<br> do<br> # if mailfile not found in KEEP_THIS list, delete it:<br> grep "^$mailfile$" "$1"/KEEP_THIS.tmp >/dev/null || echo rm "$mailfile" >> /tmp/eraseOldMails.sh<br>
done<br><br>chmod +x /tmp/eraseOldMails.sh<br>rm "$1"/KEEP_THIS.tmp<br>*************************************************************************************<br><br></div><div>Yannick Barbeaux<br></div><div><br></div>
</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-04-23 17:06 GMT+02:00 Michael Menge <span dir="ltr"><<a href="mailto:michael.menge@zdv.uni-tuebingen.de" target="_blank">michael.menge@zdv.uni-tuebingen.de</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi<div class=""><br>
<br>
Quoting Y B <<a href="mailto:ybpubs@gmail.com" target="_blank">ybpubs@gmail.com</a>>:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
OK, I managed to reconstruct my test mailbox. The files that were in my<br>
directory structure but not visible in the mailbox are now all visible in<br>
my email client.<br>
Unfortunately, the cyrreconstruct command is not verbose at all and I<br>
cannot see the filenames corresponding to rediscovered emails (so I cannot<br>
delete them...).<br>
Any clue?<br>
<br>
<br>
</blockquote></div>
...<div class=""><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
My version of cyrus : 2.2.13<br>
</blockquote></blockquote>
<br>
<br>
<br></div>
Cyrus 2.2.13 is very old. AFSIR the -n feature was added in 2.4.x.<br>
Updating in this situation might do harm, as the metadata format<br>
changed in 2.4.x<br>
<br>
I think there are no reconstruct options or other tools, to remove the<br>
deleted mails out of the box, but the information which mails where in<br>
the mailbox are stored in the cyrus.index file.<br>
See <a href="http://git.cyrusimap.org/cyrus-imapd/tree/doc/internal/mailbox-format.html?h=cyrus-imapd-2.2" target="_blank">http://git.cyrusimap.org/<u></u>cyrus-imapd/tree/doc/internal/<u></u>mailbox-format.html?h=cyrus-<u></u>imapd-2.2</a><br>
<br>
So you may be able to write a tool to parse the file.<br>
<br>
Regards<br>
<br>
Michael Menge<br>
<br>
------------------------------<u></u>------------------------------<u></u>--------------------<br>
M.Menge Tel.: (49) 7071/29-70316<br>
Universität Tübingen Fax.: (49) 7071/29-5912<br>
Zentrum für Datenverarbeitung mail: <a href="mailto:michael.menge@zdv.uni-tuebingen.de" target="_blank">michael.menge@zdv.uni-<u></u>tuebingen.de</a><br>
Wächterstraße 76<br>
72074 Tübingen<br>----<br>
Cyrus Home Page: <a href="http://www.cyrusimap.org/" target="_blank">http://www.cyrusimap.org/</a><br>
List Archives/Info: <a href="http://lists.andrew.cmu.edu/pipermail/info-cyrus/" target="_blank">http://lists.andrew.cmu.edu/pipermail/info-cyrus/</a><br>
To Unsubscribe:<br>
<a href="https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus" target="_blank">https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus</a><br></blockquote></div><br></div>