Conversations and search

[Bron Gondwana]

On Sat, Aug 18, 2018, at 00:08, Albert Shih wrote:
> Le 16/08/2018 à 14:04:22+0200, Sebastian Hagedorn a écrit
> Hi
> 
>> 
>> I have a question regarding the conversations db and how it affects
>> (Xapian)>> search. In this GitHub issue
>> (<https://github.com/cyrusimap/cyrus-imapd/issues/2376>) I was
>> dealing with>> Xapian search always failing. The underlying reason turned out
>> to be an>> empty conversations db. When I regenerate that user's
>> conversations db,>> Xapian search works fine. So far, so good. But then I noticed this
>> piece of>> documentation:
>> 
>> <https://www.cyrusimap.org/imap/concepts/deployment/databases.html?highlight=conversations#conversations-userid-conversations>>> 
>> Quote: "This file contains all the message-id fields from every
>> email that>> has been seen in the ***past three months***, mapping to the
>> conversation>> IDs in which this message ID has been seen, and the timestamp
>> when it was>> last seen."
>> 
>> This raises a number of questions. What does "has been seen" mean
>> in this>> context? And does it mean that Xapian search will always fail to
>> find emails>> that arrived more than three months ago?? That doesn't sound very
>> useful ...> 
> If I'm correct (I'm new with cyrus too), the conversations DB as
> nothing to> do with the database of xapian.
> 
> The xapian database is some file with .glass extension and
> contain all the> index of all your mail. Those files stand inside
> 
>   t1searchpartition-default/FIRST_LETTER_OF_LOGIN/user/LOGIN/
> 
> The conversation database is some cyrus internal database use by
> cyrus to> **create** the xapian index.
> 
> In other word the three months means, if you stop the xapian index
> now, you> have three months to restart it or you're going to need to
> regenerated the> xapian database from the beginning (and that take very loooong time).> 
> Please if it's incorrect please someone correct me.

That's incorrect - the messageids are actually only used for thread
calculations - so if you get a new messages more than 3 months later, it
won't be threaded with related messages (conversations.db threads only,
aka: JMAP and XCONV commands).
The G keys used for xapian are kept forever (at least: until the message
is deleted from the index - which is not when it's expunged, but when
cyr_expire cleans up the expunged record).
Bron.

--
  Bron Gondwana, CEO, FastMail Pty Ltd
  brong at fastmailteam.com


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20180818/adbfc0ce/attachment.html>