<!DOCTYPE html>
<html>
<head>
<title></title>
<style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style>
</head>
<body><div style="font-family:Arial;">On Sat, Aug 18, 2018, at 00:08, Albert Shih wrote:<br></div>
<blockquote type="cite"><div>Le 16/08/2018 à 14:04:22+0200, Sebastian Hagedorn a écrit<br></div>
<div>Hi<br></div>
<div><br></div>
<blockquote><div><br></div>
<div>I have a question regarding the conversations db and how it affects (Xapian)<br></div>
<div>search. In this GitHub issue<br></div>
<div>(<<a href="https://github.com/cyrusimap/cyrus-imapd/issues/2376">https://github.com/cyrusimap/cyrus-imapd/issues/2376</a>>) I was dealing with<br></div>
<div>Xapian search always failing. The underlying reason turned out to be an<br></div>
<div>empty conversations db. When I regenerate that user's conversations db,<br></div>
<div>Xapian search works fine. So far, so good. But then I noticed this piece of<br></div>
<div>documentation:<br></div>
<div><br></div>
<div><<a href="https://www.cyrusimap.org/imap/concepts/deployment/databases.html?highlight=conversations#conversations-userid-conversations">https://www.cyrusimap.org/imap/concepts/deployment/databases.html?highlight=conversations#conversations-userid-conversations</a>><br></div>
<div><br></div>
<div>Quote: "This file contains all the message-id fields from every email that<br></div>
<div>has been seen in the ***past three months***, mapping to the conversation<br></div>
<div>IDs in which this message ID has been seen, and the timestamp when it was<br></div>
<div>last seen."<br></div>
<div><br></div>
<div>This raises a number of questions. What does "has been seen" mean in this<br></div>
<div>context? And does it mean that Xapian search will always fail to find emails<br></div>
<div>that arrived more than three months ago?? That doesn't sound very useful ...<br></div>
</blockquote><div><br></div>
<div>If I'm correct (I'm new with cyrus too), the conversations DB as nothing to<br></div>
<div>do with the database of xapian.<br></div>
<div><br></div>
<div>The xapian database is some file with .glass extension and contain all the<br></div>
<div>index of all your mail. Those files stand inside<br></div>
<div><br></div>
<div> t1searchpartition-default/FIRST_LETTER_OF_LOGIN/user/LOGIN/<br></div>
<div><br></div>
<div>The conversation database is some cyrus internal database use by cyrus to<br></div>
<div><b>*create*</b> the xapian index.<br></div>
<div><br></div>
<div>In other word the three months means, if you stop the xapian index now, you<br></div>
<div>have three months to restart it or you're going to need to regenerated the<br></div>
<div>xapian database from the beginning (and that take very loooong time).<br></div>
<div><br></div>
<div>Please if it's incorrect please someone correct me.<br></div>
</blockquote><div style="font-family:Arial;"><br></div>
<div style="font-family:Arial;">That's incorrect - the messageids are actually only used for thread calculations - so if you get a new messages more than 3 months later, it won't be threaded with related messages (conversations.db threads only, aka: JMAP and XCONV commands).<br></div>
<div style="font-family:Arial;"><br></div>
<div style="font-family:Arial;">The G keys used for xapian are kept forever (at least: until the message is deleted from the index - which is not when it's expunged, but when cyr_expire cleans up the expunged record).<br></div>
<div style="font-family:Arial;"><br></div>
<div style="font-family:Arial;">Bron.<br></div>
<div style="font-family:Arial;"><br></div>
<div id="sig56629417"><div class="signature">--<br></div>
<div class="signature"> Bron Gondwana, CEO, FastMail Pty Ltd<br></div>
<div class="signature"> brong@fastmailteam.com<br></div>
<div class="signature"><br></div>
</div>
<div style="font-family:Arial;"><br></div>
</body>
</html>