another problem with conversations db

[Bron Gondwana]

Yes, updating reconstruct is on our roadmap. The biggest problem is that there are multiple file types across mailboxes and conversations and the changes can't be atomic, so if reconstruct discovers something partially done, it can't tell if the conversations were correct.

The longer term grand plan is to reduce the number of db types and make conversations updates able to be atomic.

Cheers,

Bron.

On Fri, Jan 25, 2019, at 22:30, Michael Menge wrote:
> Hi,
> 
> 
> Quoting Bron Gondwana <brong at fastmail.fm>:
> 
> > On Fri, Jan 25, 2019, at 20:00, Michael Menge wrote:
> >> Hi Bron
> >>
> >>
> >> Quoting Bron Gondwana <brong at fastmail.fm>:
> >>
> >> > Sorry I haven't been following along with this earlier. Can you post
> >> > your imapd.conf and cyrus.conf as well as let her know if you run
> >> > anything which directly messes with files on disk.
> >>
> >> Thanks for looking into it. I have attached the backend and replica
> >> configs. There should be nothing messing with the files on disk
> >> directly. Some times we need to restore mails from file based
> >> backup (bacula) followed by a reconstruct but this was not the
> >> case this time.
> >
> > Do you always rebuild the conversations db after doing the 
> > reconstruct? That will be necessary now. We switched to doing all 
> > our restores using IMAP append a while back so we're never fiddling 
> > the file system under Cyrus.
> >
> 
> We didn't have to restore any mails from backup since we enabled 
> conversation db
> a few weeks ago. It is good to know that the rebuild is necessary, but 
> shouldn't
> reconstruct also update the conversation db if it re-appends the message?
> At least a hint should be put in the reconstruct man page, like the 
> one for "quota -f"
> 
> 
> >> >
> >> > Also, what operating system is this on and what Cyrus version?
> >> >
> >>
> >> We are running a cyrus 3.0.8 compiled with the following Options
> >> (./configure --enable-murder --enable-http --enable-calalarmd
> >> --enable-replication --enable-backup --enable-idled
> >> --enable-autocreate CFLAGS="-fPIC -g")
> >> in a murder configuration on a RHEL 7.5 System. As filesystem we use xfs.
> >
> > They should be fine. I'll have a read of the config when I'm at a 
> > real computer.
> >
> >>
> >> > Bron.
> >> >
> >> > On Fri, Jan 25, 2019, at 04:08, Michael Menge wrote:
> >> >> Hi,
> >> >>
> >> >> I have discovered an other problem with the conversations db:
> >> >>
> >> >> Thousends of lines with "IOERROR: conversations_audit on load:" and
> >> >> "IOERROR: conversations_audit on store:"
> >> >> A look at the source code shows that these errors are logged after
> >> >> "_sanity_check_counts" is called.
> >> >> The log level LOG_ERR and the prefix IOERROR indicate that I have a
> >> >> serious problem. Do I?
> >> >>
> >> >> This problem occurred for accounts where the rebuild of the
> >> >> conversations db was successful.
> >> >>
> >> >> I don't want to rebuild the conversations db every few days.
> >> >>
> >> >> Any help is appreciated.
> >> >>
> >> >> Kind regards
> >> >>
> >> >>
> >> >> Michael Menge
> 
> 
> 
> --------------------------------------------------------------------------------
> M.Menge Tel.: (49) 7071/29-70316
> Universität Tübingen Fax.: (49) 7071/29-5912
> Zentrum für Datenverarbeitung mail: 
> michael.menge at zdv.uni-tuebingen.de
> Wächterstraße 76
> 72074 Tübingen
> 
> ----
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-- 
 Bron Gondwana
 brong at fastmail.fm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20190125/27dfd8a9/attachment.html>