<!DOCTYPE html><html><head><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div style="font-family:Arial;">Thanks Michael - I don't have much time to give a longer response right now, but this is really valuable feedback into what we focus on over the next few months and into the 3.2 release plans. I really appreciate you taking the time to spell this out in detail.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">Cheers,<br></div><div style="font-family:Arial;"><br>Bron.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">On Tue, Mar 5, 2019, at 02:17, Michael Menge wrote:<br></div><blockquote type="cite" id="fastmail-quoted"><div style="font-family:Arial;">Hi Ellie and Bron,<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">first thank you for your ideas for the workaround and for opening the issue.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">I have to apologize about the original subject, but I was a bit <br></div><div style="font-family:Arial;">frustrated that<br></div><div style="font-family:Arial;">I have encountered again a problem with the conversation db.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">As I have discovered jet another, not jet reported problem with <br></div><div style="font-family:Arial;">conversation db<br></div><div style="font-family:Arial;">(I am unsure if it's a problem with the conversation db or if only <br></div><div style="font-family:Arial;">shows an other<br></div><div style="font-family:Arial;">problem some where else), I have decided to deactivate the conversation db for<br></div><div style="font-family:Arial;">the moment.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">I think I should elaborate the general problem I have with this <br></div><div style="font-family:Arial;">feature and the<br></div><div style="font-family:Arial;">cyrus development as i have observed it.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">I am an experienced linux and cyrus administrator. I am not an <br></div><div style="font-family:Arial;">software developer,<br></div><div style="font-family:Arial;">or programmer, but I understand enough about programming to fix small problems<br></div><div style="font-family:Arial;">and narrow down problems.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">Cyrus code quality has grown in the last 14 years since i first set up <br></div><div style="font-family:Arial;">our first<br></div><div style="font-family:Arial;">cyrus imap (2.3) server. Especially since Fastmail dedicated personal for the<br></div><div style="font-family:Arial;">cyrus project. But also automatic testing and other design decisions helped to<br></div><div style="font-family:Arial;">bring the project a big step forward.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">I know we have a complex setup (murder, replication, meta- and archive <br></div><div style="font-family:Arial;">partition,<br></div><div style="font-family:Arial;">delayed delete, delayed expunge) so that we use combinations of <br></div><div style="font-family:Arial;">features that are not<br></div><div style="font-family:Arial;">that common. So fare we are happy with our cyrus setup.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">We did not encounter any data loss, and where able to fix most <br></div><div style="font-family:Arial;">problems in short time.<br></div><div style="font-family:Arial;">The system is very stable and expect for the slow search can't <br></div><div style="font-family:Arial;">complain about the performance.<br></div><div style="font-family:Arial;">So thanks again to the devs for the work and also for the community <br></div><div style="font-family:Arial;">the help I received<br></div><div style="font-family:Arial;">in the last 14 years.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">I like new features, but I have always to balance the advantages of <br></div><div style="font-family:Arial;">these new features<br></div><div style="font-family:Arial;">with the impact on stability, performance and administration overhead. <br></div><div style="font-family:Arial;">In that regard I<br></div><div style="font-family:Arial;">tend to exercise caution.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">While testing cyrus 3.0, on one hand neither conversation db nor <br></div><div style="font-family:Arial;">sphinx or xapian search<br></div><div style="font-family:Arial;">engines looked particular interesting, as I had no need to improve <br></div><div style="font-family:Arial;">search speed in<br></div><div style="font-family:Arial;">cyrus 2.4 with squatter ("If it isn't broken don't fix it"). On the <br></div><div style="font-family:Arial;">other hand i didn't<br></div><div style="font-family:Arial;">have time to rigorous test these features, so I decided not to use <br></div><div style="font-family:Arial;">them as new feature<br></div><div style="font-family:Arial;">have a tendency to contain more new bugs.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">After upgrading the production system to cyrus 3.0 I discovered that <br></div><div style="font-family:Arial;">search was slow with<br></div><div style="font-family:Arial;">squatter, conversation db did more than the documentation suggested <br></div><div style="font-family:Arial;">that it does.<br></div><div style="font-family:Arial;">I don't know if it was intended that conversation db was required for <br></div><div style="font-family:Arial;">squatter to work<br></div><div style="font-family:Arial;">or if it broke by one change to support multiple search engines but <br></div><div style="font-family:Arial;">the requirement<br></div><div style="font-family:Arial;">was surprising. I did try to find the commit that did break the <br></div><div style="font-family:Arial;">squatter search but<br></div><div style="font-family:Arial;">failed, as was unable to compile most commits git bisect suggested.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">The problems I encountered with conversation db, confirmed my initial <br></div><div style="font-family:Arial;">caution not to enable<br></div><div style="font-family:Arial;">conversation db without testing. But it would be unfair to blame only <br></div><div style="font-family:Arial;">the implementation of<br></div><div style="font-family:Arial;">conversation db.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">One problem was caused by "reconstruct -V max" not upgrading all <br></div><div style="font-family:Arial;">mailboxes, which i did<br></div><div style="font-family:Arial;">miss in the logs. It would be nice if ctl_conversationsdb would check <br></div><div style="font-family:Arial;">mailbox version<br></div><div style="font-family:Arial;">before creating a huge conversation db file in an endless loop.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">The problem with "IOERROR: conversations_audit on load:" and "IOERROR: <br></div><div style="font-family:Arial;">conversations_audit on store:"<br></div><div style="font-family:Arial;">is still a mystery to so it is unclear if it is really a bug in the <br></div><div style="font-family:Arial;">conversation db of if it shows<br></div><div style="font-family:Arial;">an other problem with my installation.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">The same is with the new problem i had no time to report jet. Deleting <br></div><div style="font-family:Arial;">users i see the following error<br></div><div style="font-family:Arial;">popping up for some accounts.<br></div><div style="font-family:Arial;">"Fatal error: Internal error: assertion failed: imap/conversations.c: <br></div><div style="font-family:Arial;">2205: !status.exists"<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">But breaking an common use case of an other feature like "delete_mode: <br></div><div style="font-family:Arial;">delayed" is<br></div><div style="font-family:Arial;">an other case. This should have been fixed before it was released in <br></div><div style="font-family:Arial;">the stable<br></div><div style="font-family:Arial;">cyrus version. At least a WARNING in the documentation is required.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">I would like to help to improve the documentation, but there are some <br></div><div style="font-family:Arial;">questions that need to be answered:<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">1. Which search engines and combinations are currently supported?<br></div><div style="font-family:Arial;"> Is a stand alone squatter still supported?<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">2. What are to pros and cons for the supported search engines?<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">3. Should, or to which extend should, the search engines work without <br></div><div style="font-family:Arial;">conversations db?<br></div><div style="font-family:Arial;"> Or is enabling the conversations db a new requirement for some/all <br></div><div style="font-family:Arial;">search engines?<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">4. Is the conversations db murder aware? And how do shared folders <br></div><div style="font-family:Arial;">(one user shared one<br></div><div style="font-family:Arial;"> of his folders with other users) on the same server/cross server<br></div><div style="font-family:Arial;"> affect search results and performances<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">5. What is stored in the conversations db?<br></div><div style="font-family:Arial;"> <br></div><div style="font-family:Arial;">https://www.cyrusimap.org/dev/imap/concepts/deployment/databases.html#conversations-userid-conversations<br></div><div style="font-family:Arial;"> is incomplete as conversations db also contains hashes of mime parts.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">6. Which Information is affected by conversations_expire_days<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">TLDR; I like cyrus, but there is some work to do regarding to <br></div><div style="font-family:Arial;">conversation db and search engines,<br></div><div style="font-family:Arial;">in the field of documentation, code testing and feature interaction<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">--------------------------------------------------------------------------------<br></div><div style="font-family:Arial;">M.Menge Tel.: (49) 7071/29-70316<br></div><div style="font-family:Arial;">Universität Tübingen Fax.: (49) 7071/29-5912<br></div><div style="font-family:Arial;">Zentrum für Datenverarbeitung mail: <br></div><div style="font-family:Arial;">michael.menge@zdv.uni-tuebingen.de<br></div><div style="font-family:Arial;">Wächterstraße 76<br></div><div style="font-family:Arial;">72074 Tübingen<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">----<br></div><div style="font-family:Arial;">Cyrus Home Page: http://www.cyrusimap.org/<br></div><div style="font-family:Arial;">List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/<br></div><div style="font-family:Arial;">To Unsubscribe:<br></div><div style="font-family:Arial;">https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus<br></div></blockquote><div style="font-family:Arial;"><br></div><div id="sig56629417"><div class="signature">--<br></div><div class="signature"> Bron Gondwana, CEO, FastMail Pty Ltd<br></div><div class="signature"> brong@fastmailteam.com<br></div><div class="signature"><br></div></div><div style="font-family:Arial;"><br></div></body></html>