cvt_xlist_specialuse does not work with subfolders

[Michael Menge]

Quoting Dilyan Palauzov <Dilyan.Palauzov at aegee.org>:

> Hello,
>
> you can use cyradm with setmetadata:  
> https://www.cyrusimap.org/dev/imap/reference/manpages/systemcommands/cyradm.html#setmetadata .  It works the sameway on 3.0, despite the fact the 3.0's documentation is silent on  
> this.
>
> When making chnages on master, e.g. updating the documengation,  
> please verify if they are relevant for 3.0 and in such case backport  
> them.
>
> There is no \spam special use, it is \junk .
>

Thanks for the hint to the documentation.
I did read a bit more about XLIST, SPECIAL-USE and Annotations and
I found some inconsistent behavior, between IMAP setmetadata command (imtest),
cvt_xlist_specialuse and cyradm setmetadata

Here the setup for the account user/zrstes1

mamx02.mail.localhost> cm user/zrstes3
mamx02.mail.localhost> cm user/zrstes3/Mail
mamx02.mail.localhost> cm user/zrstes3/Mail/v-spam
mamx02.mail.localhost> cm user/zrstes3/Mail/trash
mamx02.mail.localhost> cm user/zrstes3/Mail/sent
mamx02.mail.localhost> cm user/zrstes3/Mail/drafts
mamx02.mail.localhost> cm user/zrstes3/newdrafts
mamx02.mail.localhost> cm user/zrstes3/NewDrafts
mamx02.mail.localhost> getmetadata user/zrstes3*
mamx02.mail.localhost> lm user/zrstes3*
user/zrstes3 (\HasChildren)
user/zrstes3/Mail (\HasChildren)
user/zrstes3/Mail/drafts (\HasNoChildren)
user/zrstes3/Mail/sent (\HasNoChildren)
user/zrstes3/Mail/trash (\HasNoChildren)
user/zrstes3/Mail/v-spam (\HasNoChildren)
user/zrstes3/NewDrafts (\HasNoChildren)
user/zrstes3/newdrafts (\HasNoChildren)
mamx02.mail.localhost> setmetadata user/zrstes3/Mail/sent specialuse "\\Sent"
mamx02.mail.localhost> getmetadata user/zrstes3*
{user/zrstes3/Mail/sent}:
   private:
     specialuse: \Sent
mamx02.mail.localhost>


Switching to imtest

a LIST "" "*"
* LIST (\HasNoChildren) "/" INBOX
* LIST (\HasChildren) "/" Mail
* LIST (\HasNoChildren) "/" Mail/drafts
* LIST (\HasNoChildren) "/" Mail/sent
* LIST (\HasNoChildren) "/" Mail/trash
* LIST (\HasNoChildren) "/" Mail/v-spam
* LIST (\HasNoChildren) "/" NewDrafts
* LIST (\HasNoChildren) "/" newdrafts
a OK Completed (0.000 secs 8 calls)
a XLIST "" "Mail/sent"
* XLIST (\HasNoChildren) "/" Mail/sent
a OK Completed (0.000 secs 1 calls)
b LIST (SPECIAL-USE) "" "*"
b OK Completed (0.000 secs 8 calls)
c LIST "" "Mail/sent" RETURN (SPECIAL-USE)
* LIST (\HasNoChildren) "/" Mail/sent
c OK Completed (0.000 secs 1 calls)


No Special use visible, lets set it via IMAP for an other folder.


d SETMETADATA "Mail/trash" (/private/specialuse "\\Trash")
d OK Completed
e XLIST "" "Mail/trash"
* XLIST (\HasNoChildren \Trash) "/" Mail/trash
e OK Completed (0.000 secs 1 calls)
f LIST (SPECIAL-USE) "" "*"
* LIST (\HasNoChildren \Trash) "/" Mail/trash
f OK Completed (0.000 secs 8 calls)

Lets check in cyradm again

mamx02.mail.localhost> getmetadata user/zrstes3*
{user/zrstes3/Mail/sent}:
   private:
     specialuse: \Sent
mamx02.mail.localhost>

Still only the sent folder we configured via cyradm before.

So lets try cvt_xlist_specialuse running with "xlist-drafts: newdrafts"

cvt_xlist_specialuse -C /etc/imapd_be.conf -v user/zrstes3*
will set \drafts for folders named newdrafts
will set \sent for folders named Mail.sent
will set \trash for folders named Mail.trash
will set \junk for folders named Mail.s-spam
will set \spam for folders named Mail.v-spam
be/cvt_xlist_specialuse[3847]: couldn't authenticate to backend  
server: SASL library is not initialized
be/cvt_xlist_specialuse[3847]: mupdate_connect failed: no auth status
be/cvt_xlist_specialuse[3847]: cannot connect to mupdate server for  
update of 'user.zrstes3.newdrafts'
be/cvt_xlist_specialuse[3847]: auditlog: modseq  
sessionid=<be-3847-1534177056-1-8638186448626841993>  
mailbox=<user.zrstes3.newdrafts>  
uniqueid=<b297865b-9d2a-422d-95e2-16805e1f4678> highestmodseq=<2>
set specialuse \drafts for user.zrstes3.newdrafts

Checking in imtest.

g LIST (SPECIAL-USE) "" "*"
* LIST (\HasNoChildren \Trash) "/" Mail/trash
* LIST (\HasNoChildren \drafts) "/" newdrafts
g OK Completed (0.000 secs 8 calls)

and in cyradm

mamx02.mail.localhost> getmetadata user/zrstes3*
{user/zrstes3/Mail/sent}:
   private:
     specialuse: \Sent
mamx02.mail.localhost>

I was wondering if the Annotation set by cvt_xlist_specialuse should  
be \\ instead of \drafts
so i tried again with "/" as delimiter for xlist-sent, xlist-trash,  
xlist-junk, xlist-spam and with
xlist-Drafts: NewDrafts


> cvt_xlist_specialuse -C /etc/imapd_be.conf -v user/zrstes3*
will set \drafts for folders named NewDrafts
will set \sent for folders named Mail/sent
will set \trash for folders named Mail/trash
will set \junk for folders named Mail/s-spam
will set \spam for folders named Mail/v-spam
not setting specialuse \drafts for user.zrstes3.NewDrafts, already  
exists as user.zrstes3.newdrafts

So what is happening here?

Is cyradm updating the annotation for the admin user cyrus, and not in the
name of the user zrstes3?

Is the special-uses annotation case-sensitive? And should  
cvt_xlist_specialuse use \Drafts?



> Regards
>   Дилян
>
> On August 13, 2018 6:53:06 AM PDT, Michael Menge  
> <michael.menge at zdv.uni-tuebingen.de> wrote:
>> Hi,
>>
>> We are testing the migration from Cyrus 2.4.20 to 3.0.7
>> and have discovered a problem with cvt_xlist_specialuse
>> (migration of the xlist-* options to specialuse annotations).
>>
>> Below are the important options from the imapd.conf of the current
>> backends
>>
>> altnamespace: 1
>> unixhierarchysep: 1
>> xlist-sent: Mail.sent
>> xlist-trash: Mail.trash
>> xlist-drafts: Mail.drafts
>> xlist-junk: Mail.v-spam
>> xlist-spam: Mail.v-spam
>> specialusealways: 1
>>
>> --- default ----
>> virtdomains: off
>>
>>
>> cvt_xlist_specialuse did only output
>>
>> will set \drafts for folders named Mail.drafts
>> will set \sent for folders named Mail.sent
>> will set \trash for folders named Mail.trash
>> will set \junk for folders named Mail.s-spam
>> will set \spam for folders named Mail.v-spam
>>
>> but was not able to find any mailboxes to update the annotation.
>>
>> First we suspected that with the changed default setting for
>> unixhierarchysep
>> the format of xlist- Options might have changed to unixhierarchysep as
>> well.
>>
>> But setting
>>
>> xlist-sent: Mail/sent
>> xlist-trash: Mail/trash
>> xlist-drafts: Mail/drafts
>> xlist-junk: Mail/v-spam
>> xlist-spam: Mail/v-spam
>>
>> did only change the output of cvt_xlist_specialuse to
>>
>> will set \drafts for folders named Mail/drafts
>> will set \sent for folders named Mail/sent
>> will set \trash for folders named Mail/trash
>> will set \junk for folders named Mail/s-spam
>> will set \spam for folders named Mail/v-spam
>>
>> but it still didn't find any mailboxes to update the annotations.
>>
>> After changing the xlist- option to a direct folder of the INBOX e.g.
>> Mail
>> cvt_xlist_specialuse was able to find the folder and set the
>> annotation.
>>
>> Is there a way to set the specialuse annotations with cyradm or perl
>> script?
>>
>> Regards
>>
>>     Michael Menge
>>
>>
>>
>>
>> --------------------------------------------------------------------------------
>> M.Menge                                Tel.: (49) 7071/29-70316
>> Universität Tübingen                   Fax.: (49) 7071/29-5912
>> Zentrum für Datenverarbeitung          mail:
>> michael.menge at zdv.uni-tuebingen.de
>> Wächterstraße 76
>> 72074 Tübingen



--------------------------------------------------------------------------------
M.Menge                                Tel.: (49) 7071/29-70316
Universität Tübingen                   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung          mail:  
michael.menge at zdv.uni-tuebingen.de
Wächterstraße 76
72074 Tübingen