thread=refs
Ken Murchison
murch at andrew.cmu.edu
Tue Jul 19 09:02:36 EDT 2016
I don't think much has changed in the threading code for a while. I
would expect that the patch would apply pretty cleanly to 2.4.x.
On 07/19/2016 02:58 AM, Gabriele Bulfon wrote:
> Wow! This is really interesting!
> What minimum version of cyrus sources can I use for these changes?
> At the moment I'm running servers with 2.4.12, on our illumos based
> distro XStreamOS.
>
> ----------------------------------------------------------------------------------------
> *Sonicle S.r.l. *: http://www.sonicle.com <http://www.sonicle.com/>
> *Music: *http://www.gabrielebulfon.com <http://www.gabrielebulfon.com/>
> *Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon
>
> ------------------------------------------------------------------------
>
>
> *Da:* Ken Murchison <murch at andrew.cmu.edu>
> *A:* gbulfon at sonicle.com info-cyrus at lists.andrew.cmu.edu
> *Data:* 14 luglio 2016 17.14.18 CEST
> *Oggetto:* Re: thread=refs
>
>
> I went ahead and committed an implementation of THREAD-REFS:
> https://github.com/cyrusimap/cyrus-imapd/commit/16747e608f32f9dd9348988d3f83cb8f1b037ff6
>
> Per the draft, grouping by subject is skipped and threads
> (toplevel messages) are sorted by INTERNALDATE, while the messages
> within the threads are still sorted by SENTDATE.
>
> I confirmed that THREAD=REFERENCES is still correct, but I have
> nothing to compare THREAD=REFS results to. The current threading
> in Thunderbird is close, but it might be using INTERNALDATE
> throughout.
>
>
> On 07/12/2016 04:44 PM, Ken Murchison via Info-cyrus wrote:
>> Gabriele,
>>
>> The attached patch is what I was thinking in terms of
>> implementation. It skips the grouping by subject for REFS, but I
>> didn't do the REFS-specific date handling. Contrary to what the
>> THREAD=REFS draft says, I'm not sure if the special date handling
>> should be done in step 4 or 6. I would have to did deeper into
>> the code to see where is belongs.
>>
>>
>> On 07/08/2016 11:36 AM, Gabriele Bulfon via Info-cyrus wrote:
>>> Mainly web clients, installed clients usually implements
>>> threading internally to overcome problems with the original
>>> references algorithm that is often confusing.
>>>
>>> The problem with references is that it includes subject
>>> grouping, that is an old netscape model of the 90s: today, we
>>> just need references within the headers ids, or we may take a
>>> wrong message in the thread just because it has a similar
>>> subject (for example automatic mails with same subjects would be
>>> treated as a thread, which is wrong).
>>>
>>> Now, we're staring to implement threading view on our web
>>> collaboration software, running on cyrus.
>>> So we are investigating how RoundCube is doing threading on a
>>> dovecot installation, and we found it to be the same as the
>>> client algrithm of Thunderbird, which is fine. Looking at the
>>> protocol, it uses REFS first, probably because it has no subject
>>> grouping by definition, and it should have a better date
>>> sorting. Should, because I found that Dovecot does not sort it
>>> reversed...
>>>
>>> Maybe I will ask Dovecot guys why they choose to keep sort same
>>> as references: I suspect that claim to support refs, but
>>> actually the do the same references functions, but never do
>>> subject grouping.
>>>
>>> ----------------------------------------------------------------------------------------
>>> *Sonicle S.r.l. *: http://www.sonicle.com
>>> *Music: *http://www.gabrielebulfon.com
>>> *Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> *Da:* Ken Murchison <murch at andrew.cmu.edu>
>>> *A:* gbulfon at sonicle.com info-cyrus at lists.andrew.cmu.edu
>>> *Data:* 8 luglio 2016 17.17.32 CEST
>>> *Oggetto:* Re: thread=refs
>>>
>>>
>>>
>>>
>>> On 07/08/2016 11:08 AM, Gabriele Bulfon wrote:
>>>> Ok, sure, but still two issues remain other than the draft:
>>>> - we need to get rid of subject grouping in REFS, it only
>>>> brings disorder, merging stuff that is not related
>>>
>>> I believe that the parameterization of the core functions
>>> should be able to handle this.
>>>
>>>
>>>> - I would try to guess why dovecot does not change sorting
>>>> in REFS, keeping it same as REFERENCES
>>>
>>> I would contact that Dovecot authors and find out which
>>> version of the THREAD=REFS draft they based their work on.
>>>
>>> BTW, which clients use THREAD=REFS?
>>>
>>>
>>>>
>>>> ----------------------------------------------------------------------------------------
>>>> *Sonicle S.r.l. *: http://www.sonicle.com
>>>> *Music: *http://www.gabrielebulfon.com
>>>> *Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>> *Da:* Ken Murchison <murch at andrew.cmu.edu>
>>>> *A:* gbulfon at sonicle.com info-cyrus at lists.andrew.cmu.edu
>>>> *Data:* 8 luglio 2016 16.39.17 CEST
>>>> *Oggetto:* Re: thread=refs
>>>>
>>>>
>>>> Is there an actual RFC? All I find is
>>>> draft-ietf-morg-inthread-01. Looking at that draft, the
>>>> only difference between REFS ad REFERENCES is this:
>>>>
>>>> THREAD=REFS sorts threads by the most recent INTERNALDATE in each
>>>> thread, replacing THREAD=REFERENCES step (4). This means that when a
>>>> new message arrives, its thread becomes the latest thread. (Note
>>>> that while threads are sorted by arrival date, messages within a
>>>> thread are sorted by sent date, just as for THREAD=REFERENCES.)
>>>>
>>>>
>>>> This being the case, I don't think we need two copies
>>>> of the threading functions. I'd modify the exiting
>>>> functions to take an additional parameter to specify
>>>> whether we're doing REFS or REFERENCES and then have 2
>>>> wrapper functions which call the main function with the
>>>> parameter set appropriately for the given algorithm.
>>>>
>>>>
>>>> On 07/08/2016 10:03 AM, Gabriele Bulfon wrote:
>>>>> Ok, it works :) but checking against dovecot
>>>>> implementation, it looks like they have refs order
>>>>> same as references, but without subject grouping.
>>>>> AFAIK the RFC on refs says ordering of dates within
>>>>> the group should be reversed. Am I wrong?
>>>>>
>>>>> ----------------------------------------------------------------------------------------
>>>>> *Sonicle S.r.l. *: http://www.sonicle.com
>>>>> *Music: *http://www.gabrielebulfon.com
>>>>> *Quantum Mechanics :
>>>>> *http://www.cdbaby.com/cd/gabrielebulfon
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>> *Da:* Gabriele Bulfon via Info-cyrus
>>>>> <info-cyrus at lists.andrew.cmu.edu>
>>>>> *A:* Ken Murchison <murch at andrew.cmu.edu>
>>>>> info-cyrus at lists.andrew.cmu.edu
>>>>> *Data:* 8 luglio 2016 15.22.56 CEST
>>>>> *Oggetto:* Re: thread=refs
>>>>>
>>>>>
>>>>> Testing ;) and checking against a dovecot machine
>>>>> with refs and same messages.
>>>>> Will let you know
>>>>>
>>>>> ----------------------------------------------------------------------------------------
>>>>> *Sonicle S.r.l. *: http://www.sonicle.com
>>>>> <http://www.sonicle.com/>
>>>>> *Music: *http://www.gabrielebulfon.com
>>>>> *Quantum Mechanics :
>>>>> *http://www.cdbaby.com/cd/gabrielebulfon
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>> *Da:* Ken Murchison <murch at andrew.cmu.edu>
>>>>> *A:* gbulfon at sonicle.com
>>>>> info-cyrus at lists.andrew.cmu.edu
>>>>> *Data:* 8 luglio 2016 15.02.38 CEST
>>>>> *Oggetto:* Re: thread=refs
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 07/07/2016 02:03 PM, Gabriele Bulfon via
>>>>> Info-cyrus wrote:
>>>>>> I can finally get back to this after so many
>>>>>> months!
>>>>>> I checked the sources, and I actually see it
>>>>>> doesn't look very hard.
>>>>>>
>>>>>> Looks like:
>>>>>> - renaming all functions like
>>>>>> "index_thread_ref" into "index_thread_references"
>>>>>> - duplicate them as "index_thread_refs"
>>>>>> - let "references" alg call the "references"
>>>>>> funcs
>>>>>> - add support for "refs" in thread_algs and
>>>>>> let them call the "refs" funcs
>>>>>
>>>>> Makes sense.
>>>>>
>>>>>
>>>>>>
>>>>>> then:
>>>>>> - completely remove the call to
>>>>>> "ref_group_subjects", we don't want it at all
>>>>>> in refs
>>>>>> - change the sortcrit to use the SORT_REVERSE
>>>>>> modifier
>>>>>
>>>>> As long as you mean making these changes for
>>>>> just the "refs" variant and not both.
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> what do you think? may be fine?
>>>>>>
>>>>>> ----------------------------------------------------------------------------------------
>>>>>> *Sonicle S.r.l. *: http://www.sonicle.com
>>>>>> *Music: *http://www.gabrielebulfon.com
>>>>>> *Quantum Mechanics :
>>>>>> *http://www.cdbaby.com/cd/gabrielebulfon
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>> *Da:* Ken Murchison <murch at andrew.cmu.edu>
>>>>>> *A:* info-cyrus at lists.andrew.cmu.edu
>>>>>> *Data:* 5 ottobre 2015 14.04.02 CEST
>>>>>> *Oggetto:* Re: thread=refs
>>>>>>
>>>>>>
>>>>>> As far as I can tell, the last
>>>>>> specification for thread=refs was
>>>>>> here:https://tools.ietf.org/html/draft-ietf-morg-inthread-01
>>>>>>
>>>>>> To implement this you want to look at
>>>>>> index.c in the Cyrus source and add
>>>>>> another entry to the thread_algs[] array.
>>>>>> I'm guessing that you can reuse a lot of
>>>>>> the existing index_thread_ref() code
>>>>>> (which is probably needs to be renamed to
>>>>>> index_thread_references()).
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 10/05/2015 06:07 AM, Gabriele Bulfon
>>>>>> wrote:
>>>>>>> Great, Ken. Can you give me some advice
>>>>>>> / pointer to the sources I should look at?
>>>>>>>
>>>>>>> Gabriele
>>>>>>>
>>>>>>> ------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>> *Da:* Ken Murchison <murch at andrew.cmu.edu>
>>>>>>> *A:* info-cyrus at lists.andrew.cmu.edu
>>>>>>> *Data:* 2 ottobre 2015 19.08.04 CEST
>>>>>>> *Oggetto:* Re: thread=refs
>>>>>>>
>>>>>>>
>>>>>>> On 10/02/2015 10:53 AM, Gabriele
>>>>>>> Bulfon wrote:
>>>>>>>> Nice, it's not a big deal for us to
>>>>>>>> upgrade to new versions, surely
>>>>>>>> easier than porting to Dovecot! ;)
>>>>>>>>
>>>>>>>> So, maybe we can help with the
>>>>>>>> implementation.
>>>>>>>> In my mind, it's almost about
>>>>>>>> changing the "thread=reference" and
>>>>>>>> let it omit the subject matching,
>>>>>>>> change sorting
>>>>>>>> and...maybe just this? How much
>>>>>>>> hard do you think it is?
>>>>>>>>
>>>>>>>
>>>>>>> That sounds about right from what I
>>>>>>> remember of THREAD=REFERENCES (which
>>>>>>> I co-authored and implemented) and
>>>>>>> THREAD=REFS (which I think was last
>>>>>>> documented in 2010).
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>> *Da:* Bron Gondwana <brong at fastmail.fm>
>>>>>>>> *A:* info-cyrus at lists.andrew.cmu.edu
>>>>>>>> *Data:* 2 ottobre 2015 12.59.08 CEST
>>>>>>>> *Oggetto:* Re: thread=refs
>>>>>>>>
>>>>>>>>
>>>>>>>> No, there isn't. The
>>>>>>>> conversations work in 3.0 beta
>>>>>>>> contains a lot of what would be
>>>>>>>> required to efficiently
>>>>>>>> implement THREAD=REFS, but
>>>>>>>> nobody has done the work to
>>>>>>>> implement it.
>>>>>>>> It certainly will never be
>>>>>>>> backported to the 2.4 series,
>>>>>>>> which is only getting security
>>>>>>>> updates and fixes for major
>>>>>>>> bugs now.
>>>>>>>> Regards,
>>>>>>>> Bron.
>>>>>>>> On Fri, Oct 2, 2015, at 18:40,
>>>>>>>> Gabriele Bulfon wrote:
>>>>>>>>> Hi,
>>>>>>>>> we have systems running cyrus
>>>>>>>>> 2.4.12, where thread
>>>>>>>>> algorithms are only references
>>>>>>>>> and orderedsubject.
>>>>>>>>> Is there support for the
>>>>>>>>> thread=refs algorithm?
>>>>>>>>> Thanks
>>>>>>>>> Gabriele
>>>>>>>>> ----
>>>>>>>>> Cyrus Home Page:
>>>>>>>>> http://www.cyrusimap.org/
>>>>>>>>> List Archives/Info:
>>>>>>>>> http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>>>>>>>> To Unsubscribe:
>>>>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>>>>>> --
>>>>>>>> Bron Gondwana
>>>>>>>> brong at fastmail.fm
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----
>>>>>>>> Cyrus Home Page:http://www.cyrusimap.org/
>>>>>>>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>>>>>>> To Unsubscribe:
>>>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Kenneth Murchison
>>>>>>> Principal Systems Software Engineer
>>>>>>> Carnegie Mellon University
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ----
>>>>>>> Cyrus Home Page:http://www.cyrusimap.org/
>>>>>>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>>>>>> To Unsubscribe:
>>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>>>>
>>>>>> --
>>>>>> Kenneth Murchison
>>>>>> Principal Systems Software Engineer
>>>>>> Carnegie Mellon University
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----
>>>>>> Cyrus Home Page:http://www.cyrusimap.org/
>>>>>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>>>>> To Unsubscribe:
>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>>>
>>>>> --
>>>>> Kenneth Murchison
>>>>> Principal Systems Software Engineer
>>>>> Carnegie Mellon University
>>>>>
>>>>> ----
>>>>> Cyrus Home Page:http://www.cyrusimap.org/
>>>>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>>>> To Unsubscribe:
>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>>>
>>>>
>>>> --
>>>> Kenneth Murchison
>>>> Principal Systems Software Engineer
>>>> Carnegie Mellon University
>>>>
>>>
>>> --
>>> Kenneth Murchison
>>> Principal Systems Software Engineer
>>> Carnegie Mellon University
>>>
>>>
>>>
>>> ----
>>> Cyrus Home Page:http://www.cyrusimap.org/
>>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>
>> --
>> Kenneth Murchison
>> Principal Systems Software Engineer
>> Carnegie Mellon University
>>
>>
>> ----
>> Cyrus Home Page:http://www.cyrusimap.org/
>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>
> --
> Kenneth Murchison
> Principal Systems Software Engineer
> Carnegie Mellon University
>
--
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20160719/4aa974b8/attachment-0001.html>
More information about the Info-cyrus
mailing list