cvt_xlist_specialuse does not work with subfolders

[Dilyan Palauzov]

Hello,

my understanding is that imap attributes are case-sensitive, so  
\drafts is not the same as \Drafts.

In cyrus 3.0 normal LIST does not return the special-use flags, unless  
they are explicitly requested, or specialusealways: on is explicitly  
set - https://github.com/cyrusimap/cyrus-imapd/issues/2410 .

In cyrus imap only folders immediately below INBOX can have special  
use flags, and there are more restrictions on setting special use  
since  
https://github.com/cyrusimap/cyrus-imapd/commit/657b4ce15f160dd831d03070f5c31b290fdaf905  .  For this likely haven't filled a ticket on updating the documentation, probably because I estimated that opening the ticket will not lead anywhere and is therefore lost  
time.

I haven't used cvt_xlist_specialuse, however yours prints \spam and as  
I already wrote, there is special use flag with that name.

Enjoy!

----- Message from Michael Menge <michael.menge at zdv.uni-tuebingen.de>  
---------
    Date: Mon, 13 Aug 2018 18:33:50 +0200
    From: Michael Menge <michael.menge at zdv.uni-tuebingen.de>
Subject: Re: cvt_xlist_specialuse does not work with subfolders
      To: Dilyan Palauzov <Dilyan.Palauzov at aegee.org>
      Cc: cyrus-devel at lists.andrew.cmu.edu


> Quoting Dilyan Palauzov <Dilyan.Palauzov at aegee.org>:
>
>> Hello,
>>
>> you can use cyradm with setmetadata:  
>> https://www.cyrusimap.org/dev/imap/reference/manpages/systemcommands/cyradm.html#setmetadata .  It works the sameway on 3.0, despite the fact the 3.0's documentation is silent on  
>> this.
>>
>> When making chnages on master, e.g. updating the documengation,  
>> please verify if they are relevant for 3.0 and in such case  
>> backport them.
>>
>> There is no \spam special use, it is \junk .
>>
>
> Thanks for the hint to the documentation.
> I did read a bit more about XLIST, SPECIAL-USE and Annotations and
> I found some inconsistent behavior, between IMAP setmetadata command  
> (imtest),
> cvt_xlist_specialuse and cyradm setmetadata
>
> Here the setup for the account user/zrstes1
>
> mamx02.mail.localhost> cm user/zrstes3
> mamx02.mail.localhost> cm user/zrstes3/Mail
> mamx02.mail.localhost> cm user/zrstes3/Mail/v-spam
> mamx02.mail.localhost> cm user/zrstes3/Mail/trash
> mamx02.mail.localhost> cm user/zrstes3/Mail/sent
> mamx02.mail.localhost> cm user/zrstes3/Mail/drafts
> mamx02.mail.localhost> cm user/zrstes3/newdrafts
> mamx02.mail.localhost> cm user/zrstes3/NewDrafts
> mamx02.mail.localhost> getmetadata user/zrstes3*
> mamx02.mail.localhost> lm user/zrstes3*
> user/zrstes3 (\HasChildren)
> user/zrstes3/Mail (\HasChildren)
> user/zrstes3/Mail/drafts (\HasNoChildren)
> user/zrstes3/Mail/sent (\HasNoChildren)
> user/zrstes3/Mail/trash (\HasNoChildren)
> user/zrstes3/Mail/v-spam (\HasNoChildren)
> user/zrstes3/NewDrafts (\HasNoChildren)
> user/zrstes3/newdrafts (\HasNoChildren)
> mamx02.mail.localhost> setmetadata user/zrstes3/Mail/sent specialuse "\\Sent"
> mamx02.mail.localhost> getmetadata user/zrstes3*
> {user/zrstes3/Mail/sent}:
>   private:
>     specialuse: \Sent
> mamx02.mail.localhost>
>
>
> Switching to imtest
>
> a LIST "" "*"
> * LIST (\HasNoChildren) "/" INBOX
> * LIST (\HasChildren) "/" Mail
> * LIST (\HasNoChildren) "/" Mail/drafts
> * LIST (\HasNoChildren) "/" Mail/sent
> * LIST (\HasNoChildren) "/" Mail/trash
> * LIST (\HasNoChildren) "/" Mail/v-spam
> * LIST (\HasNoChildren) "/" NewDrafts
> * LIST (\HasNoChildren) "/" newdrafts
> a OK Completed (0.000 secs 8 calls)
> a XLIST "" "Mail/sent"
> * XLIST (\HasNoChildren) "/" Mail/sent
> a OK Completed (0.000 secs 1 calls)
> b LIST (SPECIAL-USE) "" "*"
> b OK Completed (0.000 secs 8 calls)
> c LIST "" "Mail/sent" RETURN (SPECIAL-USE)
> * LIST (\HasNoChildren) "/" Mail/sent
> c OK Completed (0.000 secs 1 calls)
>
>
> No Special use visible, lets set it via IMAP for an other folder.
>
>
> d SETMETADATA "Mail/trash" (/private/specialuse "\\Trash")
> d OK Completed
> e XLIST "" "Mail/trash"
> * XLIST (\HasNoChildren \Trash) "/" Mail/trash
> e OK Completed (0.000 secs 1 calls)
> f LIST (SPECIAL-USE) "" "*"
> * LIST (\HasNoChildren \Trash) "/" Mail/trash
> f OK Completed (0.000 secs 8 calls)
>
> Lets check in cyradm again
>
> mamx02.mail.localhost> getmetadata user/zrstes3*
> {user/zrstes3/Mail/sent}:
>   private:
>     specialuse: \Sent
> mamx02.mail.localhost>
>
> Still only the sent folder we configured via cyradm before.
>
> So lets try cvt_xlist_specialuse running with "xlist-drafts: newdrafts"
>
> cvt_xlist_specialuse -C /etc/imapd_be.conf -v user/zrstes3*
> will set \drafts for folders named newdrafts
> will set \sent for folders named Mail.sent
> will set \trash for folders named Mail.trash
> will set \junk for folders named Mail.s-spam
> will set \spam for folders named Mail.v-spam
> be/cvt_xlist_specialuse[3847]: couldn't authenticate to backend  
> server: SASL library is not initialized
> be/cvt_xlist_specialuse[3847]: mupdate_connect failed: no auth status
> be/cvt_xlist_specialuse[3847]: cannot connect to mupdate server for  
> update of 'user.zrstes3.newdrafts'
> be/cvt_xlist_specialuse[3847]: auditlog: modseq  
> sessionid=<be-3847-1534177056-1-8638186448626841993>  
> mailbox=<user.zrstes3.newdrafts>  
> uniqueid=<b297865b-9d2a-422d-95e2-16805e1f4678> highestmodseq=<2>
> set specialuse \drafts for user.zrstes3.newdrafts
>
> Checking in imtest.
>
> g LIST (SPECIAL-USE) "" "*"
> * LIST (\HasNoChildren \Trash) "/" Mail/trash
> * LIST (\HasNoChildren \drafts) "/" newdrafts
> g OK Completed (0.000 secs 8 calls)
>
> and in cyradm
>
> mamx02.mail.localhost> getmetadata user/zrstes3*
> {user/zrstes3/Mail/sent}:
>   private:
>     specialuse: \Sent
> mamx02.mail.localhost>
>
> I was wondering if the Annotation set by cvt_xlist_specialuse should  
> be \\ instead of \drafts
> so i tried again with "/" as delimiter for xlist-sent, xlist-trash,  
> xlist-junk, xlist-spam and with
> xlist-Drafts: NewDrafts
>
>
>> cvt_xlist_specialuse -C /etc/imapd_be.conf -v user/zrstes3*
> will set \drafts for folders named NewDrafts
> will set \sent for folders named Mail/sent
> will set \trash for folders named Mail/trash
> will set \junk for folders named Mail/s-spam
> will set \spam for folders named Mail/v-spam
> not setting specialuse \drafts for user.zrstes3.NewDrafts, already  
> exists as user.zrstes3.newdrafts
>
> So what is happening here?
>
> Is cyradm updating the annotation for the admin user cyrus, and not in the
> name of the user zrstes3?
>
> Is the special-uses annotation case-sensitive? And should  
> cvt_xlist_specialuse use \Drafts?
>
>
>
>> Regards
>>  Дилян
>>
>> On August 13, 2018 6:53:06 AM PDT, Michael Menge  
>> <michael.menge at zdv.uni-tuebingen.de> wrote:
>>> Hi,
>>>
>>> We are testing the migration from Cyrus 2.4.20 to 3.0.7
>>> and have discovered a problem with cvt_xlist_specialuse
>>> (migration of the xlist-* options to specialuse annotations).
>>>
>>> Below are the important options from the imapd.conf of the current
>>> backends
>>>
>>> altnamespace: 1
>>> unixhierarchysep: 1
>>> xlist-sent: Mail.sent
>>> xlist-trash: Mail.trash
>>> xlist-drafts: Mail.drafts
>>> xlist-junk: Mail.v-spam
>>> xlist-spam: Mail.v-spam
>>> specialusealways: 1
>>>
>>> --- default ----
>>> virtdomains: off
>>>
>>>
>>> cvt_xlist_specialuse did only output
>>>
>>> will set \drafts for folders named Mail.drafts
>>> will set \sent for folders named Mail.sent
>>> will set \trash for folders named Mail.trash
>>> will set \junk for folders named Mail.s-spam
>>> will set \spam for folders named Mail.v-spam
>>>
>>> but was not able to find any mailboxes to update the annotation.
>>>
>>> First we suspected that with the changed default setting for
>>> unixhierarchysep
>>> the format of xlist- Options might have changed to unixhierarchysep as
>>> well.
>>>
>>> But setting
>>>
>>> xlist-sent: Mail/sent
>>> xlist-trash: Mail/trash
>>> xlist-drafts: Mail/drafts
>>> xlist-junk: Mail/v-spam
>>> xlist-spam: Mail/v-spam
>>>
>>> did only change the output of cvt_xlist_specialuse to
>>>
>>> will set \drafts for folders named Mail/drafts
>>> will set \sent for folders named Mail/sent
>>> will set \trash for folders named Mail/trash
>>> will set \junk for folders named Mail/s-spam
>>> will set \spam for folders named Mail/v-spam
>>>
>>> but it still didn't find any mailboxes to update the annotations.
>>>
>>> After changing the xlist- option to a direct folder of the INBOX e.g.
>>> Mail
>>> cvt_xlist_specialuse was able to find the folder and set the
>>> annotation.
>>>
>>> Is there a way to set the specialuse annotations with cyradm or perl
>>> script?
>>>
>>> Regards
>>>
>>>    Michael Menge
>>>
>>>
>>>
>>>
>>> --------------------------------------------------------------------------------
>>> M.Menge                                Tel.: (49) 7071/29-70316
>>> Universität Tübingen                   Fax.: (49) 7071/29-5912
>>> Zentrum für Datenverarbeitung          mail:
>>> michael.menge at zdv.uni-tuebingen.de
>>> Wächterstraße 76
>>> 72074 Tübingen
>
>
>
> --------------------------------------------------------------------------------
> M.Menge                                Tel.: (49) 7071/29-70316
> Universität Tübingen                   Fax.: (49) 7071/29-5912
> Zentrum für Datenverarbeitung          mail:  
> michael.menge at zdv.uni-tuebingen.de
> Wächterstraße 76
> 72074 Tübingen


----- End message from Michael Menge  
<michael.menge at zdv.uni-tuebingen.de> -----