thread=refs
Ken Murchison
murch at andrew.cmu.edu
Thu Jul 14 11:14:18 EDT 2016
I went ahead and committed an implementation of THREAD-REFS:
https://github.com/cyrusimap/cyrus-imapd/commit/16747e608f32f9dd9348988d3f83cb8f1b037ff6
Per the draft, grouping by subject is skipped and threads (toplevel
messages) are sorted by INTERNALDATE, while the messages within the
threads are still sorted by SENTDATE.
I confirmed that THREAD=REFERENCES is still correct, but I have nothing
to compare THREAD=REFS results to. The current threading in Thunderbird
is close, but it might be using INTERNALDATE throughout.
On 07/12/2016 04:44 PM, Ken Murchison via Info-cyrus wrote:
> Gabriele,
>
> The attached patch is what I was thinking in terms of implementation.
> It skips the grouping by subject for REFS, but I didn't do the
> REFS-specific date handling. Contrary to what the THREAD=REFS draft
> says, I'm not sure if the special date handling should be done in step
> 4 or 6. I would have to did deeper into the code to see where is belongs.
>
>
> On 07/08/2016 11:36 AM, Gabriele Bulfon via Info-cyrus wrote:
>> Mainly web clients, installed clients usually implements threading
>> internally to overcome problems with the original references
>> algorithm that is often confusing.
>>
>> The problem with references is that it includes subject grouping,
>> that is an old netscape model of the 90s: today, we just need
>> references within the headers ids, or we may take a wrong message in
>> the thread just because it has a similar subject (for example
>> automatic mails with same subjects would be treated as a thread,
>> which is wrong).
>>
>> Now, we're staring to implement threading view on our web
>> collaboration software, running on cyrus.
>> So we are investigating how RoundCube is doing threading on a dovecot
>> installation, and we found it to be the same as the client algrithm
>> of Thunderbird, which is fine. Looking at the protocol, it uses REFS
>> first, probably because it has no subject grouping by definition, and
>> it should have a better date sorting. Should, because I found that
>> Dovecot does not sort it reversed...
>>
>> Maybe I will ask Dovecot guys why they choose to keep sort same as
>> references: I suspect that claim to support refs, but actually the do
>> the same references functions, but never do subject grouping.
>>
>> ----------------------------------------------------------------------------------------
>> *Sonicle S.r.l. *: http://www.sonicle.com
>> *Music: *http://www.gabrielebulfon.com
>> *Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon
>>
>> ------------------------------------------------------------------------
>>
>>
>> *Da:* Ken Murchison <murch at andrew.cmu.edu>
>> *A:* gbulfon at sonicle.com info-cyrus at lists.andrew.cmu.edu
>> *Data:* 8 luglio 2016 17.17.32 CEST
>> *Oggetto:* Re: thread=refs
>>
>>
>>
>>
>> On 07/08/2016 11:08 AM, Gabriele Bulfon wrote:
>>> Ok, sure, but still two issues remain other than the draft:
>>> - we need to get rid of subject grouping in REFS, it only brings
>>> disorder, merging stuff that is not related
>>
>> I believe that the parameterization of the core functions should
>> be able to handle this.
>>
>>
>>> - I would try to guess why dovecot does not change sorting in
>>> REFS, keeping it same as REFERENCES
>>
>> I would contact that Dovecot authors and find out which version
>> of the THREAD=REFS draft they based their work on.
>>
>> BTW, which clients use THREAD=REFS?
>>
>>
>>>
>>> ----------------------------------------------------------------------------------------
>>> *Sonicle S.r.l. *: http://www.sonicle.com
>>> *Music: *http://www.gabrielebulfon.com
>>> *Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> *Da:* Ken Murchison <murch at andrew.cmu.edu>
>>> *A:* gbulfon at sonicle.com info-cyrus at lists.andrew.cmu.edu
>>> *Data:* 8 luglio 2016 16.39.17 CEST
>>> *Oggetto:* Re: thread=refs
>>>
>>>
>>> Is there an actual RFC? All I find is
>>> draft-ietf-morg-inthread-01. Looking at that draft, the only
>>> difference between REFS ad REFERENCES is this:
>>>
>>> THREAD=REFS sorts threads by the most recent INTERNALDATE in each
>>> thread, replacing THREAD=REFERENCES step (4). This means that when a
>>> new message arrives, its thread becomes the latest thread. (Note
>>> that while threads are sorted by arrival date, messages within a
>>> thread are sorted by sent date, just as for THREAD=REFERENCES.)
>>>
>>>
>>> This being the case, I don't think we need two copies of the
>>> threading functions. I'd modify the exiting functions to
>>> take an additional parameter to specify whether we're doing
>>> REFS or REFERENCES and then have 2 wrapper functions which
>>> call the main function with the parameter set appropriately
>>> for the given algorithm.
>>>
>>>
>>> On 07/08/2016 10:03 AM, Gabriele Bulfon wrote:
>>>> Ok, it works :) but checking against dovecot
>>>> implementation, it looks like they have refs order same as
>>>> references, but without subject grouping. AFAIK the RFC on
>>>> refs says ordering of dates within the group should be
>>>> reversed. Am I wrong?
>>>>
>>>> ----------------------------------------------------------------------------------------
>>>> *Sonicle S.r.l. *: http://www.sonicle.com
>>>> *Music: *http://www.gabrielebulfon.com
>>>> *Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>> *Da:* Gabriele Bulfon via Info-cyrus
>>>> <info-cyrus at lists.andrew.cmu.edu>
>>>> *A:* Ken Murchison <murch at andrew.cmu.edu>
>>>> info-cyrus at lists.andrew.cmu.edu
>>>> *Data:* 8 luglio 2016 15.22.56 CEST
>>>> *Oggetto:* Re: thread=refs
>>>>
>>>>
>>>> Testing ;) and checking against a dovecot machine with
>>>> refs and same messages.
>>>> Will let you know
>>>>
>>>> ----------------------------------------------------------------------------------------
>>>> *Sonicle S.r.l. *: http://www.sonicle.com
>>>> <http://www.sonicle.com/>
>>>> *Music: *http://www.gabrielebulfon.com
>>>> *Quantum Mechanics :
>>>> *http://www.cdbaby.com/cd/gabrielebulfon
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>> *Da:* Ken Murchison <murch at andrew.cmu.edu>
>>>> *A:* gbulfon at sonicle.com info-cyrus at lists.andrew.cmu.edu
>>>> *Data:* 8 luglio 2016 15.02.38 CEST
>>>> *Oggetto:* Re: thread=refs
>>>>
>>>>
>>>>
>>>>
>>>> On 07/07/2016 02:03 PM, Gabriele Bulfon via
>>>> Info-cyrus wrote:
>>>>> I can finally get back to this after so many months!
>>>>> I checked the sources, and I actually see it
>>>>> doesn't look very hard.
>>>>>
>>>>> Looks like:
>>>>> - renaming all functions like "index_thread_ref"
>>>>> into "index_thread_references"
>>>>> - duplicate them as "index_thread_refs"
>>>>> - let "references" alg call the "references" funcs
>>>>> - add support for "refs" in thread_algs and let
>>>>> them call the "refs" funcs
>>>>
>>>> Makes sense.
>>>>
>>>>
>>>>>
>>>>> then:
>>>>> - completely remove the call to
>>>>> "ref_group_subjects", we don't want it at all in refs
>>>>> - change the sortcrit to use the SORT_REVERSE modifier
>>>>
>>>> As long as you mean making these changes for just
>>>> the "refs" variant and not both.
>>>>
>>>>
>>>>
>>>>>
>>>>> what do you think? may be fine?
>>>>>
>>>>> ----------------------------------------------------------------------------------------
>>>>> *Sonicle S.r.l. *: http://www.sonicle.com
>>>>> *Music: *http://www.gabrielebulfon.com
>>>>> *Quantum Mechanics :
>>>>> *http://www.cdbaby.com/cd/gabrielebulfon
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>> *Da:* Ken Murchison <murch at andrew.cmu.edu>
>>>>> *A:* info-cyrus at lists.andrew.cmu.edu
>>>>> *Data:* 5 ottobre 2015 14.04.02 CEST
>>>>> *Oggetto:* Re: thread=refs
>>>>>
>>>>>
>>>>> As far as I can tell, the last specification
>>>>> for thread=refs was
>>>>> here:https://tools.ietf.org/html/draft-ietf-morg-inthread-01
>>>>>
>>>>> To implement this you want to look at index.c
>>>>> in the Cyrus source and add another entry to
>>>>> the thread_algs[] array. I'm guessing that you
>>>>> can reuse a lot of the existing
>>>>> index_thread_ref() code (which is probably
>>>>> needs to be renamed to index_thread_references()).
>>>>>
>>>>>
>>>>>
>>>>> On 10/05/2015 06:07 AM, Gabriele Bulfon wrote:
>>>>>> Great, Ken. Can you give me some advice /
>>>>>> pointer to the sources I should look at?
>>>>>>
>>>>>> Gabriele
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>> *Da:* Ken Murchison <murch at andrew.cmu.edu>
>>>>>> *A:* info-cyrus at lists.andrew.cmu.edu
>>>>>> *Data:* 2 ottobre 2015 19.08.04 CEST
>>>>>> *Oggetto:* Re: thread=refs
>>>>>>
>>>>>>
>>>>>> On 10/02/2015 10:53 AM, Gabriele Bulfon
>>>>>> wrote:
>>>>>>> Nice, it's not a big deal for us to
>>>>>>> upgrade to new versions, surely easier
>>>>>>> than porting to Dovecot! ;)
>>>>>>>
>>>>>>> So, maybe we can help with the
>>>>>>> implementation.
>>>>>>> In my mind, it's almost about changing
>>>>>>> the "thread=reference" and let it omit
>>>>>>> the subject matching, change sorting
>>>>>>> and...maybe just this? How much hard do
>>>>>>> you think it is?
>>>>>>>
>>>>>>
>>>>>> That sounds about right from what I
>>>>>> remember of THREAD=REFERENCES (which I
>>>>>> co-authored and implemented) and
>>>>>> THREAD=REFS (which I think was last
>>>>>> documented in 2010).
>>>>>>
>>>>>>
>>>>>>
>>>>>>> ------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>> *Da:* Bron Gondwana <brong at fastmail.fm>
>>>>>>> *A:* info-cyrus at lists.andrew.cmu.edu
>>>>>>> *Data:* 2 ottobre 2015 12.59.08 CEST
>>>>>>> *Oggetto:* Re: thread=refs
>>>>>>>
>>>>>>>
>>>>>>> No, there isn't. The conversations
>>>>>>> work in 3.0 beta contains a lot of
>>>>>>> what would be required to
>>>>>>> efficiently implement THREAD=REFS,
>>>>>>> but nobody has done the work to
>>>>>>> implement it.
>>>>>>> It certainly will never be
>>>>>>> backported to the 2.4 series, which
>>>>>>> is only getting security updates and
>>>>>>> fixes for major bugs now.
>>>>>>> Regards,
>>>>>>> Bron.
>>>>>>> On Fri, Oct 2, 2015, at 18:40,
>>>>>>> Gabriele Bulfon wrote:
>>>>>>>> Hi,
>>>>>>>> we have systems running cyrus
>>>>>>>> 2.4.12, where thread algorithms are
>>>>>>>> only references and orderedsubject.
>>>>>>>> Is there support for the
>>>>>>>> thread=refs algorithm?
>>>>>>>> Thanks
>>>>>>>> Gabriele
>>>>>>>> ----
>>>>>>>> Cyrus Home Page:
>>>>>>>> http://www.cyrusimap.org/
>>>>>>>> List Archives/Info:
>>>>>>>> http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>>>>>>> To Unsubscribe:
>>>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>>>>> --
>>>>>>> Bron Gondwana
>>>>>>> brong at fastmail.fm
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ----
>>>>>>> Cyrus Home Page:http://www.cyrusimap.org/
>>>>>>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>>>>>> To Unsubscribe:
>>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Kenneth Murchison
>>>>>> Principal Systems Software Engineer
>>>>>> Carnegie Mellon University
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----
>>>>>> Cyrus Home Page:http://www.cyrusimap.org/
>>>>>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>>>>> To Unsubscribe:
>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>>>
>>>>> --
>>>>> Kenneth Murchison
>>>>> Principal Systems Software Engineer
>>>>> Carnegie Mellon University
>>>>>
>>>>>
>>>>>
>>>>> ----
>>>>> Cyrus Home Page:http://www.cyrusimap.org/
>>>>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>>>> To Unsubscribe:
>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>>
>>>> --
>>>> Kenneth Murchison
>>>> Principal Systems Software Engineer
>>>> Carnegie Mellon University
>>>>
>>>> ----
>>>> Cyrus Home Page:http://www.cyrusimap.org/
>>>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>>> To Unsubscribe:
>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>>>
>>>
>>> --
>>> Kenneth Murchison
>>> Principal Systems Software Engineer
>>> Carnegie Mellon University
>>>
>>
>> --
>> Kenneth Murchison
>> Principal Systems Software Engineer
>> Carnegie Mellon University
>>
>>
>>
>> ----
>> Cyrus Home Page:http://www.cyrusimap.org/
>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>
> --
> Kenneth Murchison
> Principal Systems Software Engineer
> Carnegie Mellon University
>
>
> ----
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
--
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20160714/490fe8b4/attachment-0001.html>
More information about the Info-cyrus
mailing list