From me at anatoli.ws Sun Dec 1 13:57:57 2019 From: me at anatoli.ws (Anatoli) Date: Sun, 1 Dec 2019 15:57:57 -0300 Subject: Cyrus webdav with Joplin In-Reply-To: References: Message-ID: <5d3f67e8-2faf-9cd8-a3a5-f4aba87861f2@anatoli.ws> Hi Johan, In RFC 7231 (HTTP 1.1) section 3.1.1.5 (https://tools.ietf.org/html/rfc7231#section-3.1.1.5) it says that CT header SHOULD be present, otherwise the recipient may interpret it the way it wants, so IMO no problem on the Cyrus side here. For application/json for example it MUST be present, application/xml doesn't demand that, but not sending it IMO is not a good behavior for interoperability. For collection that exists, does the user that makes the request have the rights to overwrite the collection? If not, 403 is the correct SC (status code). 405 should be used when the specified method is not allowed at all on the specified path, independently of the current server state, which is not the case here. So, again IMO no problem on the Cyrus side here, but if the user has sufficient rights, instead of 403 I'd use "409 Conflict" which is the recommended SC when a record with specified ID/name already exists. Regards, Anatoli On 28/11/19 04:40, Johan Hattne wrote: > Dear all; > > I?m trying to get Joplin (https://joplinapp.org) to work with Cyrus?s webdav module, and I?ve run into two issues: > > (1) When attempting to MKCOL a collection that already exists, Cyrus is responding with a 403, rather than a 405, which is what Joplin expects. > > (2) Cyrus returns an error if the Content-type isn?t set where additional XML-formatted information is required in a POST to complete a request. > > My skimming of the relevant RFC:s now lead me to believe that Cyrus is right on both counts; however, I don?t know enough about this to say for sure. Can anyone here confirm, or are these genuine Cyrus bugs? > > // Best wishes; Johan > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > From johan at hattne.se Mon Dec 2 05:13:43 2019 From: johan at hattne.se (Johan Hattne) Date: Mon, 2 Dec 2019 02:13:43 -0800 Subject: Cyrus webdav with Joplin In-Reply-To: <5d3f67e8-2faf-9cd8-a3a5-f4aba87861f2@anatoli.ws> References: <5d3f67e8-2faf-9cd8-a3a5-f4aba87861f2@anatoli.ws> Message-ID: Hi Anatoli; Thanks for your reply; I?ll be focusing on the MKCOL for now: I don?t know about permission to overwrite quite yet, but from looking at the source it seems the break (at https://github.com/cyrusimap/cyrus-imapd/blob/master/imap/http_dav.c#L5590) is what causes HTTP_FORBIDDEN to be returned. Now looking at the code in the client (https://github.com/laurent22/joplin/blob/master/ReactNativeClient/lib/file-api-driver-webdav.js#L164) it appears Joplin is expecting 405, or possibly 409, given the explanation in the comment following line 164. Given all that, it would seem to me that Cyrus should possibly change the aforementioned break to a return HTTP_CONFLICT, or HTTP_NOT_ALLOWED if the comment in Joplin is correct. I haven?t tested this yet (nor have read the RFC:s thoroughly), but I?d be happy to submit a pull request if this all checks out. Opinions? // Best wishes; Johan > On Dec 1, 2019, at 10:57, Anatoli wrote: > > Hi Johan, > > In RFC 7231 (HTTP 1.1) section 3.1.1.5 > (https://tools.ietf.org/html/rfc7231#section-3.1.1.5) it says that CT > header SHOULD be present, otherwise the recipient may interpret it the > way it wants, so IMO no problem on the Cyrus side here. For > application/json for example it MUST be present, application/xml doesn't > demand that, but not sending it IMO is not a good behavior for > interoperability. > > For collection that exists, does the user that makes the request have > the rights to overwrite the collection? If not, 403 is the correct SC > (status code). 405 should be used when the specified method is not > allowed at all on the specified path, independently of the current > server state, which is not the case here. > > So, again IMO no problem on the Cyrus side here, but if the user has > sufficient rights, instead of 403 I'd use "409 Conflict" which is the > recommended SC when a record with specified ID/name already exists. > > Regards, > Anatoli > > On 28/11/19 04:40, Johan Hattne wrote: >> Dear all; >> >> I?m trying to get Joplin (https://joplinapp.org) to work with Cyrus?s webdav module, and I?ve run into two issues: >> >> (1) When attempting to MKCOL a collection that already exists, Cyrus is responding with a 403, rather than a 405, which is what Joplin expects. >> >> (2) Cyrus returns an error if the Content-type isn?t set where additional XML-formatted information is required in a POST to complete a request. >> >> My skimming of the relevant RFC:s now lead me to believe that Cyrus is right on both counts; however, I don?t know enough about this to say for sure. Can anyone here confirm, or are these genuine Cyrus bugs? >> >> // Best wishes; Johan >> ---- >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >> > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus From raphael.halimi at gmail.com Tue Dec 3 09:13:54 2019 From: raphael.halimi at gmail.com (=?UTF-8?Q?Rapha=c3=abl_Halimi?=) Date: Tue, 3 Dec 2019 15:13:54 +0100 Subject: IMAP over SSL (only) handshake hangs In-Reply-To: References: <2615668.8Df7ofhZoJ@gongo> <10df14d8-47ef-6c6f-3cdd-ee27d69f2ea9@ednet.ns.ca>

Message-ID: Le 11/11/2019 ? 13:53, Helder Guerreiro via Info-cyrus a ?crit?: > It happened again, the entropy available never got bellow 3600 (logged > it every minute or so). Is Cyrus installed in a VM ? I had the same problem after upgrading Debian from 9 to 10, on some virtual servers, with OpenSSH and OpenVPN. I solved the problem by adding a virtual random number generator on those virtual machines. Here's the XML to add for libvirt/QEMU: /dev/urandom

(beware of word-wrapping, "address" is a single line) Regards, -- Rapha?l Halimi -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: OpenPGP digital signature URL: From raphael.halimi at gmail.com Tue Dec 3 09:13:54 2019 From: raphael.halimi at gmail.com (=?UTF-8?Q?Rapha=c3=abl_Halimi?=) Date: Tue, 3 Dec 2019 15:13:54 +0100 Subject: IMAP over SSL (only) handshake hangs In-Reply-To: References: <2615668.8Df7ofhZoJ@gongo> <10df14d8-47ef-6c6f-3cdd-ee27d69f2ea9@ednet.ns.ca>

(beware of word-wrapping, "address" is a single line) Regards, -- Rapha?l Halimi -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: OpenPGP digital signature URL: From vladislav.kurz at webstep.net Tue Dec 3 12:02:23 2019 From: vladislav.kurz at webstep.net (Vladislav Kurz) Date: Tue, 03 Dec 2019 18:02:23 +0100 Subject: IMAP over SSL (only) handshake hangs In-Reply-To: References: <2615668.8Df7ofhZoJ@gongo>

Message-ID: <2626983.EWypRxZh27@hex> Dne ?ter? 3. prosince 2019 15:13:54 CET, Rapha?l Halimi napsal(a): > Le 11/11/2019 ? 13:53, Helder Guerreiro via Info-cyrus a ?crit : > > It happened again, the entropy available never got bellow 3600 (logged > > it every minute or so). > > Is Cyrus installed in a VM ? > > I had the same problem after upgrading Debian from 9 to 10, on some > virtual servers, with OpenSSH and OpenVPN. > > I solved the problem by adding a virtual random number generator on > those virtual machines. Here's the XML to add for libvirt/QEMU: > > > /dev/urandom > >

function="0x0"/> > > > (beware of word-wrapping, "address" is a single line) > > Regards, I recommend installing haveged to get more entropy. Regards Vladki From helder at paxjulia.org Tue Dec 3 12:28:48 2019 From: helder at paxjulia.org (Helder Guerreiro) Date: Tue, 3 Dec 2019 17:28:48 +0000 Subject: IMAP over SSL (only) handshake hangs In-Reply-To: References: <2615668.8Df7ofhZoJ@gongo> <10df14d8-47ef-6c6f-3cdd-ee27d69f2ea9@ednet.ns.ca>

Message-ID: <777395bb-473b-4325-b4b3-d7efe29d9b78@paxjulia.org> On 03/12/19 14:13, Rapha?l Halimi wrote: > Le 11/11/2019 ? 13:53, Helder Guerreiro via Info-cyrus a ?crit?: >> It happened again, the entropy available never got bellow 3600 (logged >> it every minute or so). > > Is Cyrus installed in a VM ? It's not. > I had the same problem after upgrading Debian from 9 to 10, on some > virtual servers, with OpenSSH and OpenVPN. > > I solved the problem by adding a virtual random number generator on > those virtual machines. Here's the XML to add for libvirt/QEMU: I've installed haveged, same results. However if I use STARTTLS everything runs fine... As a workaround I did just that. From jesper at schmitz.computer Thu Dec 12 15:06:46 2019 From: jesper at schmitz.computer (Jesper Schmitz Mouridsen) Date: Thu, 12 Dec 2019 21:06:46 +0100 Subject: django-cyradm Message-ID: <0a99c502-723a-8e28-91d0-eed6f250ec71@schmitz.computer> Hi cyrus list. (This email contains self promotion of a cyrus/imap related piece of software..) I wrote some years ago (and recently updated) a small django application for cyrus and postfix, with database lookup tables. It has self service support. The system has 3 roles "admin","domain admin" and "account user". If you are interested take a look at https://djcyradm.schmitz.computer Kind regards Jesper Schmitz Mouridsen From nd at syndicat.com Thu Dec 12 15:26:46 2019 From: nd at syndicat.com (Niels Dettenbach) Date: Thu, 12 Dec 2019 21:26:46 +0100 Subject: django-cyradm In-Reply-To: <0a99c502-723a-8e28-91d0-eed6f250ec71@schmitz.computer> References: <0a99c502-723a-8e28-91d0-eed6f250ec71@schmitz.computer> Message-ID: <19012234.VQe7MYvRVb@gongo> Am Donnerstag, 12. Dezember 2019, 21:06:46 CET schrieb Jesper Schmitz Mouridsen via Info-cyrus: > It has self service support. The system has 3 roles "admin","domain > admin" and "account user". > > If you are interested take a look at sounds very interesting - i planned to write a similiar open source product (after the very flexible web-cyradm was very outdated) - but with exim / pam_mysql (should be similiar) and reserved some spare time over christmas and january for it... After planning with django years ago, my current plan was to use flask or similiar more light, because installation of django (compared to the old LAMP based web-cyradm) is more difficult for many users / impossible in some typical mass hosting environments and easier to maintain in regards of security over longer timespan (django typically needs security updates several times a year from what i experienced with it in other projects). And with flask it is (at least by my intention so far...) more easy to use it on i.e. embedded hardware or "small email servers" with a DevOps like software / "firmware" management. I very like the very intelligent (because it allows very flexible email configs as MTA integrations) database design of web_cyradm. While i've not found a way to easily "reuse" that existing database structure (or "rebuilt" it with the django DB subsystem) in flask is has to be done by hand - so "no prob" at this edge for me. But will definitely check it - and (if interested) contribute (where it may makes sense). it may possibly still is a nice base or working solution for what i'm looking for (saving me time to write something complete byself) and if flask or so is still an option, the similiar concept should allow to "reuse" at least the more fiddely python code from a django project. is there any public "main" repo of the project? many thanks for your time and posting, niels. -- --- Niels Dettenbach Syndicat IT & Internet http://www.syndicat.com PGP: https://syndicat.com/pub_key.asc --- -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part. URL: From jesper at schmitz.computer Thu Dec 12 15:45:43 2019 From: jesper at schmitz.computer (Jesper Schmitz Mouridsen) Date: Thu, 12 Dec 2019 21:45:43 +0100 Subject: django-cyradm In-Reply-To: <19012234.VQe7MYvRVb@gongo> References: <0a99c502-723a-8e28-91d0-eed6f250ec71@schmitz.computer> <19012234.VQe7MYvRVb@gongo> Message-ID: <02964a53-39f2-14ff-e3a0-8f38720bfbe5@schmitz.computer> On 12.12.2019 21.26, Niels Dettenbach wrote: > Am Donnerstag, 12. Dezember 2019, 21:06:46 CET schrieb Jesper Schmitz > Mouridsen via Info-cyrus: >> It has self service support. The system has 3 roles "admin","domain >> admin" and "account user". >> >> If you are interested take a look at > sounds very interesting - i planned to write a similiar open source product > (after the very flexible web-cyradm was very outdated) - but with exim / > pam_mysql (should be similiar) and reserved some spare time over christmas > and january for it... > > After planning with django years ago, my current plan was to use flask or > similiar more light, because installation of django (compared to the old LAMP > based web-cyradm) is more difficult for many users / impossible in some typical > mass hosting environments and easier to maintain in regards of security over > longer timespan (django typically needs security updates several times a year > from what i experienced with it in other projects). And with flask it is (at > least by my intention so far...) more easy to use it on i.e. embedded > hardware or "small email servers" with a DevOps like software / "firmware" > management. > > I very like the very intelligent (because it allows very flexible email configs > as MTA integrations) database design of web_cyradm. While i've not found a > way to easily "reuse" that existing database structure (or "rebuilt" it with > the django DB subsystem) in flask is has to be done by hand - so "no prob" at > this edge for me. > > But will definitely check it - and (if interested) contribute (where it may > makes sense). it may possibly still is a nice base or working solution for > what i'm looking for (saving me time to write something complete byself) and > if flask or so is still an option, the similiar concept should allow to > "reuse" at least the more fiddely python code from a django project. > > is there any public "main" repo of the project? https://github.com/jsm222/django-cyradm/ > > many thanks for your time and posting, > > > > niels. From tgp at cs.cmu.edu Thu Dec 12 16:30:08 2019 From: tgp at cs.cmu.edu (Tod Pike) Date: Thu, 12 Dec 2019 16:30:08 -0500 Subject: mailboxes.db disagree between master and replica Message-ID: <6036d0e5-c421-c50e-335e-1513e2115a49@andrew.cmu.edu> Hello: I am running a 3.X cyrus setup with master/slave replication. I have discovered that one of my replicas has gotten out of sync with it's master server, not in the sense that the mailboxes are not replicated, but that the mailboxes live in different partitions on the master and the slave. As an example, if I do a "ctl_mboxlist -d user." on the master, I get back that it lives on partition 2. If I do the same thing on the replica, I get back that it lives on partition 4. What happened at one time is that the user got moved from partition 4 to partition 2 and it worked correctly on the master, but never happened on the slave. BTW, the mupdate server agrees with the master, not the replica. As an experiment, I tried to do a "ctl_mboxlist -m -w" on the replica server, but it returns "couldn't get mupdate server name" since it doesn't even know about the mupdate server in it's role as a replica. What are my options here - I know that I could just copy the mailboxes.db file from the master to the replica, then manually move the mailboxes that are in the "wrong" place to the "right" place, but this seems a little drastic. Any way I can just tell the replica to update the location of the mailbox one at a time? Thanks! Tod Pike -- CMU Computing Services SGRG 105 From michael.menge at zdv.uni-tuebingen.de Thu Dec 12 18:41:07 2019 From: michael.menge at zdv.uni-tuebingen.de (Michael Menge) Date: Fri, 13 Dec 2019 00:41:07 +0100 Subject: mailboxes.db disagree between master and replica In-Reply-To: <6036d0e5-c421-c50e-335e-1513e2115a49@andrew.cmu.edu> Message-ID: <20191213004107.Horde.vylNU0QgNIkFuonbXNkDQgH@webmail.uni-tuebingen.de> Quoting Tod Pike : > Hello: > I am running a 3.X cyrus setup with master/slave replication. I > have discovered that one of my replicas has gotten out of sync with > it's master server, not in the sense that the mailboxes are not > replicated, but that the mailboxes live in different partitions on > the master and the slave. > > As an example, if I do a "ctl_mboxlist -d user." on the master, > I get back that it lives on partition 2. If I do the same thing on > the replica, I get back that it lives on partition 4. What happened > at one time is that the user got moved from partition 4 to partition 2 > and it worked correctly on the master, but never happened on the slave. > BTW, the mupdate server agrees with the master, not the replica. Make sure that the same partitions are configured on master and replica > > As an experiment, I tried to do a "ctl_mboxlist -m -w" on the > replica server, but it returns "couldn't get mupdate server name" since > it doesn't even know about the mupdate server in it's role as a replica. > Cyrus replication is not integrated with the murder setup. A replica MUST NOT have a mupdate server configured. > What are my options here - I know that I could just copy the > mailboxes.db file from the master to the replica, then manually move > the mailboxes that are in the "wrong" place to the "right" place, but > this seems a little drastic. Any way I can just tell the replica to > update the location of the mailbox one at a time? > Have you tried calling sync_client on the master with the "-u" option (See sync_client manpage for details https://linux.die.net/man/8/sync_client) This should sync all mailboxes, of the user and move the mailboxes to the correct partition. If this fails check for errors in the logs. An other option would be to move the mailboxes to the correct partition with cyradm using the renamemailbox command renamemailbox oldname newname [partition] -------------------------------------------------------------------------------- M.Menge Tel.: (49) 7071/29-70316 Universit?t T?bingen Fax.: (49) 7071/29-5912 Zentrum f?r Datenverarbeitung mail: michael.menge at zdv.uni-tuebingen.de W?chterstra?e 76 72074 T?bingen From Jim.Sculley at comcast.net Fri Dec 13 10:08:54 2019 From: Jim.Sculley at comcast.net (Jim Sculley) Date: Fri, 13 Dec 2019 10:08:54 -0500 Subject: Sieve runtime error claiming mailbox doesn't exist Message-ID: <63b049ee-835e-cca5-19fa-7ec24a1c569f@comcast.net> Hello.? I'm a new member of this list but a long time user of cyrus-imapd with postfix on my little home network. I recently replaced CentOS 7 with CentOS 8 (CentOS Linux release 8.0.1905) on my? server which had been running cyrus-imapd with sieve for years without problems.? I thought I had gotten everything set up properly on the new system (cyrus-imapd? 3.0.7-15.el8_0.1 with cyrus-sasl 2.1.27-0.3rc7.el8 and postfix 2:3.3.1-8.el8) but sieve is not moving messages into the correct folders. For example, if I try to send a simple mail on the local network: ======== [root at buttercup quota]# mail jim at home.lan Subject: Test Message Test . EOT ======== The mail log will show: ========================= Dec? 6 12:51:39 buttercup postfix/pickup[27670]: 07FBA216825B: uid=0 from= Dec? 6 12:51:39 buttercup postfix/cleanup[27810]: 07FBA216825B: message-id=<20191206175139.07FBA216825B at mail.home.lan> Dec? 6 12:51:39 buttercup postfix/qmgr[27671]: 07FBA216825B: from=, size=419, nrcpt=1 (queue active) Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: sieve runtime error for jim id <20191206175139.07FBA216825B at mail.home.lan>: Fileinto (user/jim/System Messages): Mailbox does not exist Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: LOSTQUOTA: unable to record change of 697 bytes and 1 messages in quota user.jim: Quota root does not exist Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: Delivered: <20191206175139.07FBA216825B at mail.home.lan> to mailbox: user.jim Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: USAGE jim user: 0.012321 sys: 0.005097 Dec? 6 12:51:39 buttercup postfix/lmtp[27813]: 07FBA216825B: to=, relay=localhost[127.0.0.1]:24, delay=0.27, delays=0.03/0.01/0.03/0.2, dsn=2.1.5, status=sent (250 2.1.5 Ok SESSIONID=) Dec? 6 12:51:39 buttercup postfix/qmgr[27671]: 07FBA216825B: removed ========================= So, sieve is running, and sees the rule in the script, but fails. This behavior is seen with all my rules, none of which are doing anything more than a fileinto based on sender addresses or message subjects. The mailbox does exist, as seen with cyradm: ============================ buttercup.home.lan> lm user/jim/System\ Messages user/jim/System Messages (\HasNoChildren) ============================ This was all working fine before the CentOS 8 install. The relevant (I think) imapd.conf entries: ========================== defaultpartition: default partition-default: /data/spool/imap sievedir: /var/lib/imap/sieve virtdomains: off unixhierarchysep: yes =========================== The spool directory looks like this: =========================== [root at buttercup jim]# ls -l /data/spool/imap/j/user/jim -rw-------. 1 cyrus mail??? 336 Nov 30 17:57? cyrus.annotations -rw-------. 1 cyrus mail 373712 Dec? 6 13:01? cyrus.cache -rw-------. 1 cyrus mail??? 189 Apr 19? 2013? cyrus.header -rw-------. 1 cyrus mail? 23424 Dec? 6 13:01? cyrus.index -rw-------. 1 cyrus mail??? 112 Nov 26 05:31? cyrus.squat drwx------. 2 cyrus mail? 20480 Nov 30 17:57 'System Messages' =========================== I'm not sure if I have some sort of configuration error or if this is an actual bug. I would appreciate any assistance. Thanks, Jim Sculley From boutilpj at ednet.ns.ca Fri Dec 13 16:07:42 2019 From: boutilpj at ednet.ns.ca (Patrick Boutilier) Date: Fri, 13 Dec 2019 17:07:42 -0400 Subject: Sieve runtime error claiming mailbox doesn't exist In-Reply-To: <63b049ee-835e-cca5-19fa-7ec24a1c569f@comcast.net> References: <63b049ee-835e-cca5-19fa-7ec24a1c569f@comcast.net> Message-ID: At first glance it looks like you need to change your sieve rules to use . as the mailbox separator. Something like: user.jim.System Messages On December 13, 2019 11:08:54 AM AST, Jim Sculley wrote: >Hello.? I'm a new member of this list but a long time user of >cyrus-imapd with postfix on my little home network. > > >I recently replaced CentOS 7 with CentOS 8 (CentOS Linux release >8.0.1905) on my? server which had been running >cyrus-imapd with sieve for years without problems.? I thought I had >gotten everything set up properly on the new system >(cyrus-imapd? 3.0.7-15.el8_0.1 with cyrus-sasl 2.1.27-0.3rc7.el8 and >postfix 2:3.3.1-8.el8) but sieve is not moving >messages into the correct folders. > >For example, if I try to send a simple mail on the local network: > >======== >[root at buttercup quota]# mail jim at home.lan >Subject: Test Message >Test >. >EOT >======== > > >The mail log will show: > >========================= >Dec? 6 12:51:39 buttercup postfix/pickup[27670]: 07FBA216825B: uid=0 >from= >Dec? 6 12:51:39 buttercup postfix/cleanup[27810]: 07FBA216825B: >message-id=<20191206175139.07FBA216825B at mail.home.lan> >Dec? 6 12:51:39 buttercup postfix/qmgr[27671]: 07FBA216825B: >from=, size=419, nrcpt=1 (queue active) >Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: sieve runtime error for >jim id <20191206175139.07FBA216825B at mail.home.lan>: >Fileinto (user/jim/System Messages): Mailbox does not exist >Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: LOSTQUOTA: unable to >record change of 697 bytes and 1 messages in quota >user.jim: Quota root does not exist >Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: Delivered: ><20191206175139.07FBA216825B at mail.home.lan> to mailbox: user.jim >Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: USAGE jim user: 0.012321 >sys: 0.005097 >Dec? 6 12:51:39 buttercup postfix/lmtp[27813]: 07FBA216825B: >to=, relay=localhost[127.0.0.1]:24, >delay=0.27, delays=0.03/0.01/0.03/0.2, dsn=2.1.5, status=sent (250 >2.1.5 Ok >SESSIONID=) >Dec? 6 12:51:39 buttercup postfix/qmgr[27671]: 07FBA216825B: removed >========================= > >So, sieve is running, and sees the rule in the script, but fails. This >behavior is seen with all my rules, none of which >are doing anything more than a fileinto based on sender addresses or >message subjects. > >The mailbox does exist, as seen with cyradm: > >============================ >buttercup.home.lan> lm user/jim/System\ Messages >user/jim/System Messages (\HasNoChildren) >============================ > > >This was all working fine before the CentOS 8 install. The relevant (I >think) imapd.conf entries: > > >========================== >defaultpartition: default >partition-default: /data/spool/imap >sievedir: /var/lib/imap/sieve >virtdomains: off >unixhierarchysep: yes >=========================== > >The spool directory looks like this: > >=========================== >[root at buttercup jim]# ls -l /data/spool/imap/j/user/jim >-rw-------. 1 cyrus mail??? 336 Nov 30 17:57? cyrus.annotations >-rw-------. 1 cyrus mail 373712 Dec? 6 13:01? cyrus.cache >-rw-------. 1 cyrus mail??? 189 Apr 19? 2013? cyrus.header >-rw-------. 1 cyrus mail? 23424 Dec? 6 13:01? cyrus.index >-rw-------. 1 cyrus mail??? 112 Nov 26 05:31? cyrus.squat >drwx------. 2 cyrus mail? 20480 Nov 30 17:57 'System Messages' >=========================== > >I'm not sure if I have some sort of configuration error or if this is >an actual bug. I would appreciate any assistance. > > >Thanks, > >Jim Sculley > > > > >---- >Cyrus Home Page: http://www.cyrusimap.org/ >List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >To Unsubscribe: >https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.menge at zdv.uni-tuebingen.de Fri Dec 13 17:48:31 2019 From: michael.menge at zdv.uni-tuebingen.de (Michael Menge) Date: Fri, 13 Dec 2019 23:48:31 +0100 Subject: Sieve runtime error claiming mailbox doesn't exist In-Reply-To: References: <63b049ee-835e-cca5-19fa-7ec24a1c569f@comcast.net> Message-ID: <20191213234831.Horde.GMZqCT8YHi1W5kMQwa2EQwL@webmail.uni-tuebingen.de> Hi Patrick Quoting Patrick Boutilier : > At first glance it looks like you need to change your sieve rules to > use . as the mailbox separator. Something like: > > user.jim.System Messages > I don't think the hierarchy separator is the problem. Some log outputs use the internal format e.g. "delivered to mailbox: user.jim" but the posted conf shows he is using "/" as separator. > On December 13, 2019 11:08:54 AM AST, Jim Sculley > wrote: >> Hello.? I'm a new member of this list but a long time user of >> cyrus-imapd with postfix on my little home network. >> >> >> I recently replaced CentOS 7 with CentOS 8 (CentOS Linux release >> 8.0.1905) on my? server which had been running >> cyrus-imapd with sieve for years without problems.? I thought I had >> gotten everything set up properly on the new system >> (cyrus-imapd? 3.0.7-15.el8_0.1 with cyrus-sasl 2.1.27-0.3rc7.el8 and >> postfix 2:3.3.1-8.el8) but sieve is not moving >> messages into the correct folders. >> which cyrus version did you use with CentOS7? >> For example, if I try to send a simple mail on the local network: >> >> ======== >> [root at buttercup quota]# mail jim at home.lan >> Subject: Test Message >> Test >> . >> EOT >> ======== >> >> >> The mail log will show: >> >> ========================= >> Dec? 6 12:51:39 buttercup postfix/pickup[27670]: 07FBA216825B: uid=0 >> from= >> Dec? 6 12:51:39 buttercup postfix/cleanup[27810]: 07FBA216825B: >> message-id=<20191206175139.07FBA216825B at mail.home.lan> >> Dec? 6 12:51:39 buttercup postfix/qmgr[27671]: 07FBA216825B: >> from=, size=419, nrcpt=1 (queue active) >> Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: sieve runtime error for >> jim id <20191206175139.07FBA216825B at mail.home.lan>: >> Fileinto (user/jim/System Messages): Mailbox does not exist Can you post the sieve script. I don't know if the log shows the full path, but make sure that the fileinto command does not contain the "user/jim" part as the fileinto is relative to the INBOX of the user. >> Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: LOSTQUOTA: unable to >> record change of 697 bytes and 1 messages in quota >> user.jim: Quota root does not exist something else is not working correct. Is the configdirectory configured? Does it exist? Do directory and quotafiles exists? >> Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: Delivered: >> <20191206175139.07FBA216825B at mail.home.lan> to mailbox: user.jim >> Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: USAGE jim user: 0.012321 >> sys: 0.005097 >> Dec? 6 12:51:39 buttercup postfix/lmtp[27813]: 07FBA216825B: >> to=, relay=localhost[127.0.0.1]:24, >> delay=0.27, delays=0.03/0.01/0.03/0.2, dsn=2.1.5, status=sent (250 >> 2.1.5 Ok >> SESSIONID=) >> Dec? 6 12:51:39 buttercup postfix/qmgr[27671]: 07FBA216825B: removed >> ========================= >> >> So, sieve is running, and sees the rule in the script, but fails. This >> behavior is seen with all my rules, none of which >> are doing anything more than a fileinto based on sender addresses or >> message subjects. >> >> The mailbox does exist, as seen with cyradm: >> >> ============================ >> buttercup.home.lan> lm user/jim/System\ Messages >> user/jim/System Messages (\HasNoChildren) >> ============================ >> >> >> This was all working fine before the CentOS 8 install. The relevant (I >> think) imapd.conf entries: >> >> >> ========================== >> defaultpartition: default >> partition-default: /data/spool/imap >> sievedir: /var/lib/imap/sieve >> virtdomains: off >> unixhierarchysep: yes >> =========================== >> >> The spool directory looks like this: >> >> =========================== >> [root at buttercup jim]# ls -l /data/spool/imap/j/user/jim >> -rw-------. 1 cyrus mail??? 336 Nov 30 17:57? cyrus.annotations >> -rw-------. 1 cyrus mail 373712 Dec? 6 13:01? cyrus.cache >> -rw-------. 1 cyrus mail??? 189 Apr 19? 2013? cyrus.header >> -rw-------. 1 cyrus mail? 23424 Dec? 6 13:01? cyrus.index >> -rw-------. 1 cyrus mail??? 112 Nov 26 05:31? cyrus.squat >> drwx------. 2 cyrus mail? 20480 Nov 30 17:57 'System Messages' >> =========================== >> >> I'm not sure if I have some sort of configuration error or if this is >> an actual bug. I would appreciate any assistance. >> -------------------------------------------------------------------------------- M.Menge Tel.: (49) 7071/29-70316 Universit?t T?bingen Fax.: (49) 7071/29-5912 Zentrum f?r Datenverarbeitung mail: michael.menge at zdv.uni-tuebingen.de W?chterstra?e 76 72074 T?bingen From Jim.Sculley at comcast.net Sat Dec 14 19:25:04 2019 From: Jim.Sculley at comcast.net (Jim Sculley) Date: Sat, 14 Dec 2019 19:25:04 -0500 Subject: Sieve runtime error claiming mailbox doesn't exist In-Reply-To: <20191213234831.Horde.GMZqCT8YHi1W5kMQwa2EQwL@webmail.uni-tuebingen.de> References: <63b049ee-835e-cca5-19fa-7ec24a1c569f@comcast.net> <20191213234831.Horde.GMZqCT8YHi1W5kMQwa2EQwL@webmail.uni-tuebingen.de> Message-ID: <389ed3db-7cfc-08b7-7403-093b495d52b0@comcast.net> On 12/13/19 5:48 PM, Michael Menge wrote > > which cyrus version did you use with CentOS7? > I can't be certain since the jump to CentOS 8 killed off my CentOS 7 install, but the latest packages listed for CentoOS 7 is cyrus-imapd-2.4.17-15.el7.x86_64.rpm > Can you post the sieve script. I don't know if the log shows the full path, > but make sure that the fileinto command does not contain the "user/jim" part > as the fileinto is relative to the INBOX of the user. Yep. That was it.? I tried just about every combination except that. ======================= Dec 14 19:12:49 buttercup cyrus/lmtp[10349]: Delivered: <20191215001249.5320B2167685 at mail.home.lan> to mailbox: user.jim.System Messages ======================= >>> Dec? 6 12:51:39 buttercup cyrus/lmtp[27814]: LOSTQUOTA: unable to >>> record change of 697 bytes and 1 messages in quota >>> user.jim: Quota root does not exist > > something else is not working correct. Is the configdirectory configured? > Does it exist? Do directory and quotafiles exists? I hadn't started tackling that error message yet.? I will due my due diligence with Google, Stack Exchange, etc... and come back to the lists if I can't get it sorted out. Thanks for helping me with the immediate problem. From pgnet.dev at gmail.com Sun Dec 15 23:12:58 2019 From: pgnet.dev at gmail.com (PGNet Dev) Date: Sun, 15 Dec 2019 20:12:58 -0800 Subject: Just-Enuf Cyrus IMAP for standalone Card-/CalDAV? Message-ID: I run my own smtp, imap & card/cal-dav services. Atm, none are using Cyrus. I'm well aware of Cyrus Imap's Cal/Card-dav support; particularly as the basis for FastMail's excellent services. My current card/cal-dav is Radicale. To my read, the project's crumbling. I need to implement/deploy an alternative. But, for **now**, ONLY for the *Dav pieces; I need to leave mail services as they are, at least 'til mid-2020. For my domain, 'example.com' -- with smtp/imap @ host 'mx.example.com', I'd like to deploy a standalone Dav server elsewhere, @ host 'dav.example.com'. I'd like it to be Cyrus based. IIUC, Cyrus' *Dav are built on top of its IMAP install. Can Cyrus be deployed in a standalone *Dav server mode like this? Is there a doc/how-to/etc that address a Just-Enough-Cyrus-IMAP install for this kind of setup? Thx! From me at anatoli.ws Sun Dec 15 23:34:40 2019 From: me at anatoli.ws (Anatoli) Date: Mon, 16 Dec 2019 01:34:40 -0300 Subject: Just-Enuf Cyrus IMAP for standalone Card-/CalDAV? In-Reply-To: References: Message-ID: You can have a complete Cyrus install but only use the DAV features. You just don't open the IMAP ports in firewall and that's it. On 16/12/19 01:12, PGNet Dev wrote: > I run my own smtp, imap & card/cal-dav services. > > Atm, none are using Cyrus. > > I'm well aware of Cyrus Imap's Cal/Card-dav support; particularly as the basis for FastMail's excellent services. > > My current card/cal-dav is Radicale. To my read, the project's crumbling. > > I need to implement/deploy an alternative. > > But, for **now**, ONLY for the *Dav pieces; I need to leave mail services as they are, at least 'til mid-2020. > > For my domain, 'example.com' -- with smtp/imap @ host 'mx.example.com', I'd like to deploy a standalone Dav server elsewhere, @ host 'dav.example.com'. > > I'd like it to be Cyrus based. > > IIUC, Cyrus' *Dav are built on top of its IMAP install. > > Can Cyrus be deployed in a standalone *Dav server mode like this? > > Is there a doc/how-to/etc that address a Just-Enough-Cyrus-IMAP install for this kind of setup? > > Thx! > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > From ellie at fastmail.com Mon Dec 16 00:04:36 2019 From: ellie at fastmail.com (ellie timoney) Date: Mon, 16 Dec 2019 16:04:36 +1100 Subject: Just-Enuf Cyrus IMAP for standalone Card-/CalDAV? In-Reply-To: References:

Message-ID: <7e21ee5e-fdec-4269-94b4-9aa00ccff6b1@www.fastmail.com> I think you might run into problems with calendar notifications like that, because Cyrus will think it owns the accounts' email, and try to send notifications to its own mailboxes instead of over smtp to the real mail server? On Mon, Dec 16, 2019, at 3:34 PM, Anatoli wrote: > You can have a complete Cyrus install but only use the DAV features. You > just don't open the IMAP ports in firewall and that's it. > > On 16/12/19 01:12, PGNet Dev wrote: > > I run my own smtp, imap & card/cal-dav services. > > > > Atm, none are using Cyrus. > > > > I'm well aware of Cyrus Imap's Cal/Card-dav support; particularly as the basis for FastMail's excellent services. > > > > My current card/cal-dav is Radicale. To my read, the project's crumbling. > > > > I need to implement/deploy an alternative. > > > > But, for **now**, ONLY for the *Dav pieces; I need to leave mail services as they are, at least 'til mid-2020. > > > > For my domain, 'example.com' -- with smtp/imap @ host 'mx.example.com', I'd like to deploy a standalone Dav server elsewhere, @ host 'dav.example.com'. > > > > I'd like it to be Cyrus based. > > > > IIUC, Cyrus' *Dav are built on top of its IMAP install. > > > > Can Cyrus be deployed in a standalone *Dav server mode like this? > > > > Is there a doc/how-to/etc that address a Just-Enough-Cyrus-IMAP install for this kind of setup? > > > > Thx! > > ---- > > Cyrus Home Page: http://www.cyrusimap.org/ > > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > > To Unsubscribe: > > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > From ellie at fastmail.com Mon Dec 16 00:15:54 2019 From: ellie at fastmail.com (ellie timoney) Date: Mon, 16 Dec 2019 16:15:54 +1100 Subject: Cyrus IMAP 2.5.15 released Message-ID: <0d71f988-fcac-4ac9-90a4-b1797d1af4a7@www.fastmail.com> The Cyrus team is proud to announce the immediate availability of a new version of Cyrus IMAP: 2.5.15 This release contains a fix for CVE-2019-19783, a privilege escalation vulnerability that permits creation of arbitrary mailboxes using the 'fileinto' directive in user sieve scripts. If you allow your users to upload custom sieve scripts, and if you have the 'anysievefolder' option enabled, you will need this upgrade. I'm trialling hosting the release files using Github's releases feature. Please use the Github download links if possible, and advise if you have any problems! (It may even download faster due to Github's content delivery network.) Download URLs: https://github.com/cyrusimap/cyrus-imapd/releases/download/cyrus-imapd-2.5.15/cyrus-imapd-2.5.15.tar.gz https://github.com/cyrusimap/cyrus-imapd/releases/download/cyrus-imapd-2.5.15/cyrus-imapd-2.5.15.tar.gz.sig https://www.cyrusimap.org/releases/cyrus-imapd-2.5.15.tar.gz https://www.cyrusimap.org/releases/cyrus-imapd-2.5.15.tar.gz.sig Please consult the release notes before upgrading to 2.5.15: https://www.cyrusimap.org/imap/download/release-notes/2.5/x/2.5.15.html And join us on Github at https://github.com/cyrusimap/cyrus-imapd to report issues, join in the deliberations of new features for the next Cyrus IMAP release, and to contribute to the documentation. On behalf of the Cyrus team, Kind regards, ellie timoney From ellie at fastmail.com Mon Dec 16 00:18:12 2019 From: ellie at fastmail.com (ellie timoney) Date: Mon, 16 Dec 2019 16:18:12 +1100 Subject: Cyrus IMAP 3.0.13 released Message-ID: <029cc34e-fb55-461f-8f45-6180cc259a47@www.fastmail.com> The Cyrus team is proud to announce the immediate availability of a new version of Cyrus IMAP: 3.0.13 This release contains a fix for CVE-2019-19783, a privilege escalation vulnerability that permits creation of arbitrary mailboxes using the 'fileinto' directive in user sieve scripts. If you allow your users to upload custom sieve scripts, and if you have the 'mailbox' sieve extension or the 'anysievefolder' option enabled, you will need this upgrade. I'm trialling hosting the release files using Github's releases feature. Please use the Github download links if possible, and advise if you have any problems! (It may even download faster due to Github's content delivery network.) Download URLs: https://github.com/cyrusimap/cyrus-imapd/releases/download/cyrus-imapd-3.0.13/cyrus-imapd-3.0.13.tar.gz https://github.com/cyrusimap/cyrus-imapd/releases/download/cyrus-imapd-3.0.13/cyrus-imapd-3.0.13.tar.gz.sig https://www.cyrusimap.org/releases/cyrus-imapd-3.0.13.tar.gz https://www.cyrusimap.org/releases/cyrus-imapd-3.0.13.tar.gz.sig Please consult the release notes and upgrade documentation before upgrading to 3.0.13: https://www.cyrusimap.org/imap/download/release-notes/3.0/x/3.0.13.html https://www.cyrusimap.org/imap/download/upgrade.html And join us on Github at https://github.com/cyrusimap/cyrus-imapd to report issues, join in the deliberations of new features for the next Cyrus IMAP release, and to contribute to the documentation. On behalf of the Cyrus team, Kind regards, ellie timoney From lists at nethead.se Mon Dec 16 07:29:12 2019 From: lists at nethead.se (Lists Nethead) Date: Mon, 16 Dec 2019 13:29:12 +0100 Subject: lmdb: MDB_PAGE_FULL: Internal error - page has no more space Message-ID: <20191216132912.Horde.xTHLLDutqvA4cK4_hFGI-io@webmail.nethead.se> This error started after update to 3.0.12 yesterday. No idea where to start - hints anyone? Thanks, Per From byrnejb at harte-lyne.ca Mon Dec 16 12:22:17 2019 From: byrnejb at harte-lyne.ca (James B. Byrne) Date: Mon, 16 Dec 2019 12:22:17 -0500 Subject: Transfer existing quotas to new cyrus imap service Message-ID: <205247f0ff32664bc79e59e13af06765.squirrel@webmail.harte-lyne.ca> We are transferring an existing cyrus-imapd 3.0.11 mailstore to another host also running cyrus-imapd 3.0.11. I cannot find any documentation on how one transfers user quotas. Can someone provide me with the link to the documentation or explain how it is done? Thanks, -- *** e-Mail is NOT a SECURE channel *** Do NOT transmit sensitive data via e-Mail Do NOT open attachments nor follow links sent by e-Mail James B. Byrne mailto:ByrneJB at Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3 From lists at nethead.se Mon Dec 16 18:00:48 2019 From: lists at nethead.se (Lists Nethead) Date: Tue, 17 Dec 2019 00:00:48 +0100 Subject: lmdb: MDB_PAGE_FULL: Internal error - page has no more space In-Reply-To: <20191216132912.Horde.xTHLLDutqvA4cK4_hFGI-io@webmail.nethead.se> Message-ID: <20191217000048.Horde.BX7vBRwOKuBFUU47x1HYkRt@webmail.nethead.se> Since there are no answers I can assume I'm alone with the problem. Apparently lmdc has hit a limit someplace but I have no idea how to fix. Maybe be someone can answer the following instead: If I want to switch from lmdb to twoskip for the conversations.db, can I simply, set conversations: 0 stop squatter rm -r everything under search/* change to conversations_db: twoskip and then set conversations to 1 and restart squatter ? Quoting Lists Nethead : > This error started after update to 3.0.12 yesterday. > > No idea where to start - hints anyone? > > Thanks, > > Per From gbulfon at sonicle.com Tue Dec 17 01:26:35 2019 From: gbulfon at sonicle.com (Gabriele Bulfon) Date: Tue, 17 Dec 2019 07:26:35 +0100 (CET) Subject: Squatter segfault on big mailbox In-Reply-To: References: <9279660e-7c86-a00d-45a5-e3516cafbf64@peralex.com> Message-ID: <1399826115.779.1576563995688@www> Hello, we're having the same problem with Cyrus 2.5.11 on XStreamOS/illumos Any idea how to fix this? Same problem both on 32 bit large files build, and 64 bit. Looks like the Debian issue has no solution. Can't believe this has never been addressed. ? Gabriele ? ? Sonicle S.r.l.? :? http://www.sonicle.com Music:? http://www.gabrielebulfon.com Quantum Mechanics :? http://www.cdbaby.com/cd/gabrielebulfon ---------------------------------------------------------------------------------- Da: Vladislav Kurz A: Mark info-cyrus Data: 11 ottobre 2017 9.37.21 CEST Oggetto: Re: Squatter segfault on big mailbox On 10/11/17 07:37, Mark wrote: On 2017-10-10 16:05, Vladislav Kurz wrote: Hello everyone, we have recently migrated our mail server from cyrus 2.2 to 2.5 (both debian packages). We have a problem that squatter segfaults on some quite big mailboxes. I had to add -i (--incremental) to at least index new mails. But users cannot search in old mails (before migration). I used strace to find out if it is some particular mail, but it parsed everything fine, and failed when dealing with some of the many temporary files squatXXXX. The problematic mailboxes have just inbox10 GB (and15 GB with all subfolders) and20000 emails in inbox. Has anyone else hit similar problem and solved it somehow? I'm not sure about fixes, but there are some reports of similar problems.? Have a look at the thread with the subject "Cyrus 2.5.10 IMAP search" from the beginning of June this year.? Apparently the format of the squatter database changed between 2.4 and 2.5, and some people (at least) have seen a 10x increase in the size of the squatter database for the same mailbox.? For systems that only allow 2GB mmap, this can be a problem. I'm not sure if this increase in size is a bug or a feature. Mark Hello Mark, Thanks for the hint about old discussion. Unfortunately, there was no solution to this issue mentioned. I found out that this was already reported to debian, I wonder whetrher the maintainer forwarded the bug report upstream. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=871921 This is on 64-bit Linux, so mmap should not be limited to 2 GB. And I also have noticed the extreme increase in squat file sizes. -- Best Regards Vladislav Kurz ---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -------------- next part -------------- An HTML attachment was scrubbed... URL: From come.chilliet at fusiondirectory.org Tue Dec 17 08:14:04 2019 From: come.chilliet at fusiondirectory.org (=?ISO-8859-1?Q?C=F4me?= Chilliet) Date: Tue, 17 Dec 2019 14:14:04 +0100 Subject: ACLs user ids Message-ID: <10375016.Z6NHPf3ssp@mcmic-probook> Hello, I?m trying to fix ACL support for cyrus in our software written in PHP. It?s hard to understand which user id is expected when calling imap_setacl: https://www.php.net/manual/en/function.imap-setacl.php Does using the user email address works? Is using imap_setacl the same as using the setacl command in cyradm? Why doesn?t setacl complains if I give wrong user id? I tried "setacl share/test at example.org nonexistinguser lr" and it does not complain, and lists it back when using listacl, even if the user does not exists. -- C?me Chilliet FusionDirectory - https://www.fusiondirectory.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part. URL: From michael.menge at zdv.uni-tuebingen.de Tue Dec 17 09:17:09 2019 From: michael.menge at zdv.uni-tuebingen.de (Michael Menge) Date: Tue, 17 Dec 2019 15:17:09 +0100 Subject: Transfer existing quotas to new cyrus imap service In-Reply-To: <205247f0ff32664bc79e59e13af06765.squirrel@webmail.harte-lyne.ca> Message-ID: <20191217151709.Horde.s1QDT16j2qgx_Vn2EzTryXa@webmail.uni-tuebingen.de> Hi James, Quoting "James B. Byrne via Info-cyrus" : > We are transferring an existing cyrus-imapd 3.0.11 mailstore to > another host also running cyrus-imapd 3.0.11. I cannot find any > documentation on how one transfers user quotas. Can someone provide > me with the link to the documentation or explain how it is done? > > Thanks, > Cyrus stores important information in different paths. See imapd.conf manpage: configdirectory: /mailboxes.db: list of all mailboxes and acls /annotations.db: mailbox Annotations /deliver.db: Information about recent delivered mails (needed for duplicate delivery suppression and sieve vacation) /user/**: subscribed folders and seen information /quota/**: Quota-Information sievedir: Sieve Scripts Depending on the configured partitions one ore more partition-: Mailboxes/Mails of the users on the partition and optional metapartition-: Meta-Files of the Mailboxes on the partition archivepartition-: Old Mails of Mailboxes on the partition You can use cyrus-replication to transfer all user/information to the second server and keep them in sync. https://www.cyrusimap.org/imap/reference/admin/sop/replication.html Michael -------------------------------------------------------------------------------- M.Menge Tel.: (49) 7071/29-70316 Universit?t T?bingen Fax.: (49) 7071/29-5912 Zentrum f?r Datenverarbeitung mail: michael.menge at zdv.uni-tuebingen.de W?chterstra?e 76 72074 T?bingen From g.danti at assyoma.it Fri Dec 20 03:51:25 2019 From: g.danti at assyoma.it (Gionatan Danti) Date: Fri, 20 Dec 2019 09:51:25 +0100 Subject: Migration issue with seen/subscription/sieve databases Message-ID: <291efd45-51cb-e9a8-cb6e-ade02594a397@assyoma.it> Hi all, I am tasked to migrate an old cyrus 2.3.x CentOS6 installation to a new CentOS8 server with cyrus 3.0.7. I'm following the instructions at https://cyrusimap.org/imap/download/upgrade.html (compresive of berkeley db format change to skiplist) and all is mostly working, except for some important per-user databases: seen, subscription and sieve. The issue basically seems one of a different file names, revolving around using "." (dot) instead of "^" (circumflex). For example, the old cyrus installation has the following per-user database structure: [root at ceres imap]# cd /var/lib/imap; find -iname "*^*" | grep danti ./sieve/domain/a/assyoma.it/g/g^danti ./domain/a/assyoma.it/user/g/g^danti.seen ./domain/a/assyoma.it/user/g/g^danti.sub ./domain/a/assyoma.it/quota/g/user.g^danti However the new installation simply ignores the above files, creating *new* empty files with "." (dot) separator, except for the quota (which is correctly understand and no new file was created). This means I lose all information about subscribed folder, seen mails and sive filtering. A very simple workaround seems to soft-link the dot-enabled files with the circumflex ones, ie: "cd ./domain/a/assyoma.it/user/g/; ln -s g^danti.seen g.danti.seen" I wonder if this is the right thing to do, or if I am missing something. Please also consider that on the old server I already had "unixhierarchysep: 1", matching the new 3.x default. Anyway, flipping that option has no effect on how cyrus names the above files. Any suggestion to solve this problem? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8 From g.danti at assyoma.it Mon Dec 23 06:04:32 2019 From: g.danti at assyoma.it (Gionatan Danti) Date: Mon, 23 Dec 2019 12:04:32 +0100 Subject: Migration issue with seen/subscription/sieve databases In-Reply-To: <291efd45-51cb-e9a8-cb6e-ade02594a397@assyoma.it> References: <291efd45-51cb-e9a8-cb6e-ade02594a397@assyoma.it> Message-ID: <1dc0f703-44c2-4077-74fa-02e1e197d6ae@assyoma.it> On 20/12/19 09:51, Gionatan Danti wrote: > Hi all, > I am tasked to migrate an old cyrus 2.3.x CentOS6 installation to a new > CentOS8 server with cyrus 3.0.7. > > I'm following the instructions at > https://cyrusimap.org/imap/download/upgrade.html (compresive of berkeley > db format change to skiplist) and all is mostly working, except for some > important per-user databases: seen, subscription and sieve. > > The issue basically seems one of a different file names, revolving > around using "." (dot) instead of "^" (circumflex). For example, the old > cyrus installation has the following per-user database structure: > > [root at ceres imap]# cd /var/lib/imap; find -iname "*^*" | grep danti > ./sieve/domain/a/assyoma.it/g/g^danti > ./domain/a/assyoma.it/user/g/g^danti.seen > ./domain/a/assyoma.it/user/g/g^danti.sub > ./domain/a/assyoma.it/quota/g/user.g^danti > > However the new installation simply ignores the above files, creating > *new* empty files with "." (dot) separator, except for the quota (which > is correctly understand and no new file was created). This means I lose > all information about subscribed folder, seen mails and sive filtering. > > A very simple workaround seems to soft-link the dot-enabled files with > the circumflex ones, ie: "cd ./domain/a/assyoma.it/user/g/; ln -s > g^danti.seen g.danti.seen" > > I wonder if this is the right thing to do, or if I am missing something. > Please also consider that on the old server I already had > "unixhierarchysep: 1", matching the new 3.x default. Anyway, flipping > that option has no effect on how cyrus names the above files. > > Any suggestion to solve this problem? > Thanks. Hi all, any idea/suggestion/etc? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8 From boutilpj at ednet.ns.ca Mon Dec 23 12:28:30 2019 From: boutilpj at ednet.ns.ca (Patrick Boutilier) Date: Mon, 23 Dec 2019 13:28:30 -0400 Subject: Migration issue with seen/subscription/sieve databases In-Reply-To: <1dc0f703-44c2-4077-74fa-02e1e197d6ae@assyoma.it> References: <291efd45-51cb-e9a8-cb6e-ade02594a397@assyoma.it> <1dc0f703-44c2-4077-74fa-02e1e197d6ae@assyoma.it> Message-ID: <81e2b832-9b8d-b470-d77e-6e275c7d8762@ednet.ns.ca> On 12/23/19 7:04 AM, Gionatan Danti wrote: > On 20/12/19 09:51, Gionatan Danti wrote: >> Hi all, >> I am tasked to migrate an old cyrus 2.3.x CentOS6 installation to a >> new CentOS8 server with cyrus 3.0.7. >> >> I'm following the instructions at >> https://cyrusimap.org/imap/download/upgrade.html (compresive of >> berkeley db format change to skiplist) and all is mostly working, >> except for some important per-user databases: seen, subscription and >> sieve. >> >> The issue basically seems one of a different file names, revolving >> around using "." (dot) instead of "^" (circumflex). For example, the >> old cyrus installation has the following per-user database structure: >> >> [root at ceres imap]# cd /var/lib/imap; find -iname "*^*" | grep danti >> ./sieve/domain/a/assyoma.it/g/g^danti >> ./domain/a/assyoma.it/user/g/g^danti.seen >> ./domain/a/assyoma.it/user/g/g^danti.sub >> ./domain/a/assyoma.it/quota/g/user.g^danti >> >> However the new installation simply ignores the above files, creating >> *new* empty files with "." (dot) separator, except for the quota >> (which is correctly understand and no new file was created). This >> means I lose all information about subscribed folder, seen mails and >> sive filtering. >> >> A very simple workaround seems to soft-link the dot-enabled files with >> the circumflex ones, ie: "cd ./domain/a/assyoma.it/user/g/; ln -s >> g^danti.seen g.danti.seen" >> >> I wonder if this is the right thing to do, or if I am missing >> something. Please also consider that on the old server I already had >> "unixhierarchysep: 1", matching the new 3.x default. Anyway, flipping >> that option has no effect on how cyrus names the above files. >> >> Any suggestion to solve this problem? >> Thanks. > > Hi all, > any idea/suggestion/etc? > I can't find any .seen files in my test setup on CentOS 8 and not using sieve. But for .sub I see what you are seeing in file name but contents of the sub file has the ^ and it does work as expected. Not using virtualdomains on my end. Only thing I can think of is that a different config file is actually being used? > Thanks. > -------------- next part -------------- A non-text attachment was scrubbed... Name: boutilpj.vcf Type: text/x-vcard Size: 286 bytes Desc: not available URL: From g.danti at assyoma.it Tue Dec 24 03:52:49 2019 From: g.danti at assyoma.it (Gionatan Danti) Date: Tue, 24 Dec 2019 09:52:49 +0100 Subject: Migration issue with seen/subscription/sieve databases In-Reply-To: <81e2b832-9b8d-b470-d77e-6e275c7d8762@ednet.ns.ca> References: <291efd45-51cb-e9a8-cb6e-ade02594a397@assyoma.it> <1dc0f703-44c2-4077-74fa-02e1e197d6ae@assyoma.it> <81e2b832-9b8d-b470-d77e-6e275c7d8762@ednet.ns.ca> Message-ID: <3902c934-61cb-9057-0d3f-f461828dff4a@assyoma.it> On 23/12/19 18:28, Patrick Boutilier wrote: > Only thing I can think of is that a different config file is actually > being used? Hi Patrick, I double-checked my config and it seems the same (within the limits imposed by the Cyrus version change). If seems to me that when default for unixhierarchysep was changed to 1/yes the hashing function for .sub and .seen files was changed as well. Does anyone know more about that? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8 From lambert at lambertfam.org Thu Dec 26 22:07:17 2019 From: lambert at lambertfam.org (Scott Lambert) Date: Thu, 26 Dec 2019 20:07:17 -0700 Subject: Migration issue with seen/subscription/sieve databases In-Reply-To: <3902c934-61cb-9057-0d3f-f461828dff4a@assyoma.it> References: <291efd45-51cb-e9a8-cb6e-ade02594a397@assyoma.it> <1dc0f703-44c2-4077-74fa-02e1e197d6ae@assyoma.it> <81e2b832-9b8d-b470-d77e-6e275c7d8762@ednet.ns.ca> <3902c934-61cb-9057-0d3f-f461828dff4a@assyoma.it> Message-ID: On 12/24/19 1:52 AM, Gionatan Danti wrote: > On 23/12/19 18:28, Patrick Boutilier wrote: >> Only thing I can think of is that a different config file is actually >> being used? > > Hi Patrick, I double-checked my config and it seems the same (within > the limits imposed by the Cyrus version change). > > If seems to me that when default for unixhierarchysep was changed to > 1/yes the hashing function for .sub and .seen files was changed as well. > > Does anyone know more about that? > Thanks. Didn't the seen status database get moved from it's own file to the message index database for 3.x?? I'm not running 3.x yet but think I remember seeing something about that. From g.danti at assyoma.it Fri Dec 27 07:40:39 2019 From: g.danti at assyoma.it (Gionatan Danti) Date: Fri, 27 Dec 2019 13:40:39 +0100 Subject: Migration issue with seen/subscription/sieve databases In-Reply-To: References: <291efd45-51cb-e9a8-cb6e-ade02594a397@assyoma.it> <1dc0f703-44c2-4077-74fa-02e1e197d6ae@assyoma.it> <81e2b832-9b8d-b470-d77e-6e275c7d8762@ednet.ns.ca> <3902c934-61cb-9057-0d3f-f461828dff4a@assyoma.it> Message-ID: <7f50db208e33fb0264a0d938aec0f8ee@assyoma.it> Il 27-12-2019 04:07 Scott Lambert ha scritto: > Didn't the seen status database get moved from it's own file to the > message index database for 3.x?? I'm not running 3.x yet but think I > remember seeing something about that. Hi, from what I read here [1], and from my experience when testing the migration, the seen database has its own file (userid.seen). Regards. [1] https://cyrusimap.org/imap/concepts/deployment/databases.html#databases -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8 From ellie at fastmail.com Sun Dec 29 19:51:14 2019 From: ellie at fastmail.com (ellie timoney) Date: Mon, 30 Dec 2019 11:51:14 +1100 Subject: Migration issue with seen/subscription/sieve databases In-Reply-To: <7f50db208e33fb0264a0d938aec0f8ee@assyoma.it> References: <291efd45-51cb-e9a8-cb6e-ade02594a397@assyoma.it> <1dc0f703-44c2-4077-74fa-02e1e197d6ae@assyoma.it> <81e2b832-9b8d-b470-d77e-6e275c7d8762@ednet.ns.ca> <3902c934-61cb-9057-0d3f-f461828dff4a@assyoma.it> <7f50db208e33fb0264a0d938aec0f8ee@assyoma.it> Message-ID: <7a69cf4f-646b-47e0-a788-d693e61285b5@www.fastmail.com> On Fri, Dec 27, 2019, at 11:40 PM, Gionatan Danti wrote: > Il 27-12-2019 04:07 Scott Lambert ha scritto: > > Didn't the seen status database get moved from it's own file to the > > message index database for 3.x?? I'm not running 3.x yet but think I > > remember seeing something about that. > > Hi, from what I read here [1], and from my experience when testing the > migration, the seen database has its own file (userid.seen). You're both right. Since 2.4ish (?? MAILBOX_MINOR_VERSION=12, anyway), a user's seen status on their own mailboxes is stored in the mailbox index. But the separate userid.seen database is still used for storing their seen state on mailboxes that they don't own (but have ACL access to see). On a system that was set up after that change, I think the userid.seen file will only be created if the user views messages in a mailbox that they don't own. If no-one has access to view anyone else's mailbox, this will never happen, which might explain why Patrick doesn't have any .seen files. > A very simple workaround seems to soft-link the dot-enabled files with > the circumflex ones, ie: "cd ./domain/a/assyoma.it/user/g/; ln -s > g^danti.seen g.danti.seen" It's probably safer to just rename them -- if nothing else, it'll be less confusing next time you or someone else looks at them. If you wanted to move cautiously, you could hard link the corrected name, and then unlink the old name later once you're sure everything's working correctly. > I am tasked to migrate an old cyrus 2.3.x CentOS6 installation to a new > CentOS8 server with cyrus 3.0.7. With your old installation being as old as it is, it may contain data susceptible to https://github.com/cyrusimap/cyrus-imapd/issues/2839 (I have just fixed a typo in the upgrade documentation that linked to the wrong issue number, which will update on the web after the next hour.) This issue wasn't fixed until 3.0.11, so I would check carefully whether CentOS has backported the patch to their 3.0.7 package. There have also been security fixes in releases since 3.0.7, so you probably also want to check whether CentOS's 3.0.7 has backported these too. If the CentOS package hasn't backported these fixes, consider running the current version of Cyrus rather than the one CentOS provides. Cheers, ellie From g.danti at assyoma.it Mon Dec 30 04:59:25 2019 From: g.danti at assyoma.it (Gionatan Danti) Date: Mon, 30 Dec 2019 10:59:25 +0100 Subject: Migration issue with seen/subscription/sieve databases In-Reply-To: <7a69cf4f-646b-47e0-a788-d693e61285b5@www.fastmail.com> References: <291efd45-51cb-e9a8-cb6e-ade02594a397@assyoma.it> <1dc0f703-44c2-4077-74fa-02e1e197d6ae@assyoma.it> <81e2b832-9b8d-b470-d77e-6e275c7d8762@ednet.ns.ca> <3902c934-61cb-9057-0d3f-f461828dff4a@assyoma.it> <7f50db208e33fb0264a0d938aec0f8ee@assyoma.it> <7a69cf4f-646b-47e0-a788-d693e61285b5@www.fastmail.com> Message-ID: <3a9df4da960f672e02d5603563560c5c@assyoma.it> Il 30-12-2019 01:51 ellie timoney ha scritto: > You're both right. Since 2.4ish (?? MAILBOX_MINOR_VERSION=12, > anyway), a user's seen status on their own mailboxes is stored in the > mailbox index. But the separate userid.seen database is still used > for storing their seen state on mailboxes that they don't own (but > have ACL access to see). > > On a system that was set up after that change, I think the userid.seen > file will only be created if the user views messages in a mailbox that > they don't own. If no-one has access to view anyone else's mailbox, > this will never happen, which might explain why Patrick doesn't have > any .seen files. Very valuable information, thanks! > It's probably safer to just rename them -- if nothing else, it'll be > less confusing next time you or someone else looks at them. If you > wanted to move cautiously, you could hard link the corrected name, and > then unlink the old name later once you're sure everything's working > correctly. True: in fact, I already prepared a "find" script which rename the affected files rather than soft-linking them (exactly to avoid confusions). > With your old installation being as old as it is, it may contain data > susceptible to https://github.com/cyrusimap/cyrus-imapd/issues/2839 (I > have just fixed a typo in the upgrade documentation that linked to the > wrong issue number, which will update on the web after the next hour.) > This issue wasn't fixed until 3.0.11, so I would check carefully > whether CentOS has backported the patch to their 3.0.7 package. There > have also been security fixes in releases since 3.0.7, so you probably > also want to check whether CentOS's 3.0.7 has backported these too. > If the CentOS package hasn't backported these fixes, consider running > the current version of Cyrus rather than the one CentOS provides. Are you referring to the problem described here [1]? If so, from the linked page I read: "Versions of 3.0 prior to 3.0.11 contained a bug (Issue #2839) that could lead to loss of seen state/flags during reconstruct for some messages that already existed prior to Cyrus 2.3" My Cyrus installation was never older than 2.3.x, so I thought the bug should not affect me. Am I wrong? Thanks a lot. [1] https://cyrusimap.org/imap/download/upgrade.html#reconstruct-databases-and-cache > > Cheers, > > ellie > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8