Folder subscription issue
Egoitz Aurrekoetxea
egoitz at sarenet.es
Tue Jul 16 14:20:49 EDT 2019
Hi!
Mmm perhaps I’m telling this excessively sure but…
In sync_do_user() function, when your scripts syncs user by user instead of using MODE_ALLUSER (which perhaps is more normal… but our scripts are done with a for loop with each user) shouldn’t the userid be converted :
/* we don't hold locks while sending commands */
mailbox_close(&mailbox);
r = do_user_main(userid, topart, replica_folders, replica_quota,
sync_be, channelp, flags);
if (r) goto done;
/* COMMENTED CHANGE */
char *user_without_dot = mbname_userid(userid)
r = sync_do_user_sub(user_without_dot, replica_subs, sync_be, flags);
if (r) goto done;
r = sync_do_user_sieve(user_without_dot, replica_sieve, sync_be, flags);
if (r) goto done;
r = sync_do_user_seen(user_without_dot, replica_seen, sync_be, flags);
/* END COMMENTED CHANGE */
I got tons of cases like this that could are explained this way. :
It’s like if sync_do_user wan’t giving the proper user for later creating the file and so to sync_do_user_sub , sync_do_user_seen… etc etc…
Perhaps I have not reproduced (I realized now, although not tested) this because I didn’t create folders in my testing env (the one for reproducing this)… not at least this way…
I think fix this issue…. What do you think about it?. At least that, shouldn’t damage… I mean at least the fact of replacing the ^ with a dot… and subscripciones would work…. Seen seems to be handled now perhaps in conversations database or similar… but subscriptions… the problem I’m seeing is this “^” in the file name….
What do you think mates?
Egoitz Aurrekoetxea
Dpto. de sistemas
944 209 470
Parque Tecnológico. Edificio 103
48170 Zamudio (Bizkaia)
egoitz at sarenet.es <mailto:undefined>
www.sarenet.es <http://www.sarenet.es/>
Antes de imprimir este correo electrónico piense si es necesario hacerlo.
> El 16 jul 2019, a las 18:41, Egoitz Aurrekoetxea <egoitz at sarenet.es> escribió:
>
> Hi!
>
> One question are this records in the mailbox database (seen with a ctl_mboxlist -d) normal ?
>
> xxxxxxxxxx.ccc!user.ama.COMUNICACIONES CIAS-OTROS.COMUNICACIONES CIAS 2017. NOVIEMBRE 2017 16 (null)
> xxxxxxxxxx.ccc!user.ama.COMUNICACIONES CIAS-OTROS.COMUNICACIONES CIAS 2017. OCTUBRE 2017 16 (null)
> xxxxxxxxxx.ccc!user.ama.COMUNICACIONES CIAS-OTROS.COMUNICACIONES CIAS 2017.AGOSTO 2017 16 (null)
> xxxxxxxxxx.ccc!user.ama.COMUNICACIONES CIAS-OTROS.COMUNICACIONES CIAS 2017.DICIEMBRE 2017 16 (null)
> xxxxxxxxxx.ccc!user.ama.COMUNICACIONES CIAS-OTROS.COMUNICACIONES CIAS 2017.SEPTIEMBRE 2017 16 (null)
> xxxxxxxxxx.ccc!user.ama.COMUNICACIONES CIAS-OTROS.COMUNICACIONES CIAS 2017.SEPTIEMBRE 2017.JULIO 2017 16 (null)
> xxxxxxxxxx.ccc!user.ama.COMUNICACIONES CIAS-OTROS.COMUNICACIONES CIAS 2017.SEPTIEMBRE 2017.JUNIO 2017 16 (null)
> xxxxxxxxxx.ccc!user.ama.COMUNICACIONES CIAS-OTROS.COMUNICACIONES CIAS 2017.SEPTIEMBRE 2017.MAYO 2017 16 (null)
>
> Cheers!
>
>
>
> Egoitz Aurrekoetxea
> Dpto. de sistemas
> 944 209 470
> Parque Tecnológico. Edificio 103
> 48170 Zamudio (Bizkaia)
> egoitz at sarenet.es <mailto:undefined>
> www.sarenet.es <http://www.sarenet.es/>
> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
>
>> El 16 jul 2019, a las 10:10, Egoitz Aurrekoetxea <egoitz at sarenet.es <mailto:egoitz at sarenet.es>> escribió:
>>
>> No problem Bron!!
>>
>> Very thankful for your time and your help!!.
>>
>> I have some ideas/questions about the synchronous replication too for commenting you… I have never heard about Cassandane (only to Ellie in this list or Github commit comments) :) but will check it :) :) although I’d need to wait the holidays to come, in order to be more free for all these :)
>>
>> Cheers!
>>
>>
>>
>> Egoitz Aurrekoetxea
>> Dpto. de sistemas
>> 944 209 470
>> Parque Tecnológico. Edificio 103
>> 48170 Zamudio (Bizkaia)
>> egoitz at sarenet.es <mailto:undefined>
>> www.sarenet.es <http://www.sarenet.es/>
>> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
>>
>>> El 16 jul 2019, a las 2:04, Bron Gondwana <brong at fastmail.fm <mailto:brong at fastmail.fm>> escribió:
>>>
>>> Sorry for not getting back to you yesterday. I was on my way back from vacation and had family commitments all last night.
>>>
>>> Regarding contribution - the very best thing to do for a case like this is to build Cassandane tests which isolate the issue :) I'll see if I can get that done today.
>>>
>>> Cheers,
>>>
>>> Bron.
>>>
>>> On Tue, Jul 16, 2019, at 02:34, Egoitz Aurrekoetxea wrote:
>>>> Hi Bron!
>>>>
>>>> Sorry for answering so late I had the thought I answered you before…. I’m slightly stressed these days...
>>>>
>>>> I answer below in green bold for instance…..
>>>>
>>>>
>>>>
>>>> Egoitz Aurrekoetxea
>>>> Dpto. de sistemas
>>>> 944 209 470
>>>> Parque Tecnológico. Edificio 103
>>>> 48170 Zamudio (Bizkaia)
>>>> egoitz at sarenet.es <mailto:egoitz at sarenet.es>
>>>> www.sarenet.es <http://www.sarenet.es/>
>>>>
>>>> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
>>>>
>>>>> El 14 jul 2019, a las 14:36, Bron Gondwana <brong at fastmailteam.com <mailto:brong at fastmailteam.com>> escribió:
>>>>>
>>>>> On Wed, Jul 10, 2019, at 19:11, Egoitz Aurrekoetxea wrote:
>>>>>> Good morning,
>>>>>>
>>>>>> In previous thread (with the title slightly incorrect) I talk about an issue suffered some day with Sieve script and folder subscriptions. The Sieve part, was related to the migration, so for the moment let’s forget about it (for me at least) as it has never reproduced again since then.
>>>>>
>>>>> Sorry, I missed the previous thread. Did you mention which version of Cyrus you are running? There's clearly a bug and it would be good to know which version(s) it affects.
>>>>
>>>>
>>>> I think I have specified all these details (including what the previous thread was saying) in the answered mail this morning… I think it's all there… the version was 3.0.8…. If is not all or you need something, please tell me and I’ll try my bests for providing you all the info you need..
>>>>
>>>>
>>>>>
>>>>>> About the folder subscription issue, I think I got something, at least a close approximation... When a user causes in mua a rename mailbox (a rename for a folder caused by a folder move in hierarchy), after the own rename, if folders were subscribed “should” (for the "plain user" at least) become subscribed in the new path. It seems that after a user rename in Cyrus the new folder is automatically subscribed (even if no subscribe command is sent by the mua). But this, does not cause in the replica (in the slave, if SUB is not sent by the client) a sync_apply_changesub() or something like entering in the “ move_subscription” condition in mboxlist_renamemailbox(), and then, the folder is properly renamed but not subscribed in the slave. I think this is what I’m suffering. Obviously, if after a rename the mua sends a subscribe too, no issue is seen. I think the problem happens when a mailbox rename happens and a SUB is not send later.
>>>>>
>>>>> Subscriptions aren't automatically renamed when a folder is renamed, but they should be automatically renamed for a user rename. Intermediate replicated states can be bit messy due to race conditions with replication, but I believe it should always wind up in the final correct state. If not, that's a bug for sure!
>>>>
>>>> This tests are some days ago… I’m slightly confused now because I don’t remember it exactly…
>>>>
>>>>>
>>>>>> An example :
>>>>>>
>>>>>> The folder domain.com <http://domain.com/>!user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG is moved (renamed) to trash.
>>>>>>
>>>>>> May 16 16:16:50 mx6c imap[83976]: conversations_rename_folder: renamed domain.com <http://domain.com/>!user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG to domain.com <http://domain.com/>!user.parta^partb.Trash.XINGANG
>>>>>> May 16 16:16:50 mx6c imap[83976]: Rename: domain.com <http://domain.com/>!user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG -> domain.com <http://domain.com/>!user.parta^partb.Trash.XINGANG
>>>>>> May 16 16:16:50 mx6c imap[83976]: Deleted mailbox domain.com <http://domain.com/>!user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG
>>>>>>
>>>>>> In the master (sync), the folder in Trash is subscribed but in the slave it is not, and remains subscribed the folder in the original location in the “____.sub” file.
>>>>>
>>>>> This might just be a matter of failing to sync_log the subscription in that codepath. Hmm...
>>>>>
>>>>> But there is a question of whether we should even be renaming the subscription - RFC3501 is a little silent on this issue, but it does say in a couple of places that the server MUST NOT remove a subscription even if the mailbox with that name doesn't exist, which makes renaming subscriptions a bit unclear. I'll check in with the authors of draft-ietf-extra-imap4rev2 and ask for this to be clarified next time around!
>>>>
>>>>
>>>> That’s it Bron.. it was strange… anyway… you know, when you try to debug an issue like the commented you do a closer inspection to all this behaviour, you know…. Although as said before, perhaps it’s better to talk about today's examples… I got them more recent….
>>>>
>>>>
>>>>>
>>>>>> A diff between the master (replication client) .sub file and the slave’s one is (mx5 is the master, the client and 6 the slave):
>>>>>>
>>>>>> --- subscripciones-mx5c/parta.partb.sub 2019-07-10 08:47:29.000000000 +0200
>>>>>> +++ subscripciones-mx6c/parta.partb.sub 2019-07-10 08:48:08.000000000 +0200
>>>>>>
>>>>>> +domain.com <http://domain.com/>!user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG
>>>>>> -domain.com <http://domain.com/>!user.parta^partb.Trash.XINGANG
>>>>>>
>>>>>> Perhaps a move_subscription to one or sync_apply_changesub should be forced in order to fix it?. I have seen issues with Outlook 2016 and Thunderbird… both mua… perhaps by RFC they should send the SUB command but… it’s the only theory I could arrive to… I have a ton more cases like this one.. Data gets properly handled but subscriptions have this issue (perhaps we could say is a mua issue but….)..
>>>>>
>>>>> Bron.
>>>>
>>>>
>>>> Thanks a lot!
>>>>> --
>>>>> Bron Gondwana, CEO, Fastmail Pty Ltd
>>>>> brong at fastmailteam.com <mailto:brong at fastmailteam.com>
>>>>>
>>>>>
>>>>> ----
>>>>> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
>>>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
>>>>> To Unsubscribe:
>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
>>>>
>>>> ----
>>>> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
>>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
>>>> To Unsubscribe:
>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
>>>
>>> --
>>> Bron Gondwana
>>> brong at fastmail.fm <mailto:brong at fastmail.fm>
>>>
>>>
>>> ----
>>> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
>> ----
>> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
> ----
> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20190716/93b62a66/attachment-0001.html>
More information about the Info-cyrus
mailing list