From ml at netfence.it Fri Sep 4 04:30:35 2020 From: ml at netfence.it (Andrea Venturoli) Date: Fri, 4 Sep 2020 10:30:35 +0200 Subject: OutLook support Message-ID: Hello. A long time ago I had a customer who used OutLook with a Cyrus-IMAP server (I think it was 2.2 at the time). Up to OL 2010 everything was (almost) fine, but upgrading to OL 2013 was a disaster, e.g. local-only folders, ability to move messages to folders which didn't exist on the server (thus losing it!), renaming folders seemed to work, but on the server nothing happened, etc... What's the status of interoperability today? Will OL 2013 work reliably with CyrusIMAP 3.0? 3.2? What about newer versions of OL? bye & Thanks av. From debbiep at polyfoam.com.au Fri Sep 4 21:11:40 2020 From: debbiep at polyfoam.com.au (Deborah Pickett) Date: Sat, 5 Sep 2020 11:11:40 +1000 Subject: OutLook support In-Reply-To: References: Message-ID: <0a71dd93-e1c6-86ba-e24f-45583a409cba@polyfoam.com.au> On 2020-09-04 18.30, Andrea Venturoli wrote: > What's the status of interoperability today? > Will OL 2013 work reliably with CyrusIMAP 3.0? 3.2? > What about newer versions of OL? Hi Andrea, I can offer anecdata of interoperability between Cyrus 3.0.x and Outlook 2016 from ten months of experience. Like Outlook in general, it works fine, until it doesn't. With ~40 users with mailboxes up to 15 GB in size, I've had to help users start with fresh Outlook profiles about half a dozen times when their Outlook data file got corrupted. Some specific observations: Like any IMAP account in Outlook, you lose the ability to tag messages with colour.? All you can do is flag or unflag a message. Outlook does not handle the \Deleted flag well, and it has a habit of showing messages which are \Deleted as if they are still present (especially in Drafts). I recommend configuring all clients sharing an account with Outlook to purge folders early and often. Outlook respects special-use folders about as well as other IMAP clients, which is to say, not super-well.? If you try to change which folder is the Trash folder using cyradm, Outlook may not notice the change.? That said, it's not as bad as Windows 10 mail, which flat out refuses to respect special-use folder tags at all. 64-bit Outlook is needed if both (a) your data file is > 2 GB, and (b) you want search indexing to work.? With 32-bit Outlook, search will silently miss messages. Outlook is inefficient at IMAP synchronization.? If you have large mailboxes, it can spend a lot of its time synchronizing subscribed folders.? There are ways of winnowing the list of folders that Outlook tries to sync during Send And Receive, and this helps a bit. Outlook by default wants to send read receipts.? These invisible messages can silently clog up the outbox, preventing any messages from being sent.? It is a very good idea to disable sending read receipts entirely; I have set a Group Policy object to do this. I've had to use a free program called MFCMAPI to manually delete queued read receipts that accidentally got through. To sync Outlook with Cyrus contacts, calendars and tasks, the free Outlook CalDav Synchronizer add-in works well (but sync rules are tedious to set up, and I haven't found a way to deploy preconfigured sync rules to users).? Outlook tries to disable the add-in because it slows down startup, but you can prevent this with a registry setting. Rarely, Outlook will decide that a folder is local-only, and any messages moved into that folder will stop syncing with Cyrus.? The only fix I have found is to create a new Outlook profile (and then hunt for lost messages to drag back under synchronized folders). Renaming a folder, or moving it somewhere else, in Outlook usually works.? Rarely, it'll make the folder local-only.? A slightly more robust way of renaming a folder is to make a new folder with the new name, and move the contents of the old folder over to the new folder.? I have seen this fail too. These corruptions seem to occur more often in "Other Users" and "Shared Folders", but this might just be because said folders are huge (> 4 GB) in my company. Of course, it's unlikely that any of these irritations will ever get fixed by Microsoft.? In the long term, I am considering migrating my users from Outlook to Thunderbird or webmail. Hope these data points are useful. Deborah From ml at netfence.it Sat Sep 5 06:51:13 2020 From: ml at netfence.it (Andrea Venturoli) Date: Sat, 5 Sep 2020 12:51:13 +0200 Subject: OutLook support In-Reply-To: <0a71dd93-e1c6-86ba-e24f-45583a409cba@polyfoam.com.au> References: <0a71dd93-e1c6-86ba-e24f-45583a409cba@polyfoam.com.au> Message-ID: <813ccd74-cff1-c164-f2d4-3d8839addb9d@netfence.it> On 2020-09-05 03:11, Deborah Pickett wrote: > On 2020-09-04 18.30, Andrea Venturoli wrote: >> What's the status of interoperability today? >> Will OL 2013 work reliably with CyrusIMAP 3.0? 3.2? >> What about newer versions of OL? > > Hi Andrea, Hello. > I can offer anecdata of interoperability between Cyrus 3.0.x and Outlook > 2016 from ten months of experience. Thank you very much for sharing. > Like Outlook in general, it works fine, until it doesn't. With ~40 users > with mailboxes up to 15 GB in size, I've had to help users start with > fresh Outlook profiles about half a dozen times when their Outlook data > file got corrupted. Nothing new here: when I said pre-2013 version worked, what I meant is a ~50 users installation required 2-3 reconfigurations per month. I can live with that. Of course using ThunderBird would mean max. 2-3 per *year* or even less, but it's up to the customer to choose. > Some specific observations: > > Like any IMAP account in Outlook, you lose the ability to tag messages > with colour.? All you can do is flag or unflag a message. Fine, I guess. > Outlook does not handle the \Deleted flag well, and it has a habit of > showing messages which are \Deleted as if they are still present > (especially in Drafts). I recommend configuring all clients sharing an > account with Outlook to purge folders early and often. True, I remember that. Showing deleted messages with an overstroke and forcing the user to "delete twice" was puzzling most of my users. > Outlook respects special-use folders about as well as other IMAP > clients, which is to say, not super-well.? If you try to change which > folder is the Trash folder using cyradm, Outlook may not notice the > change. Fine as long as I can set this up for a new account. Then I can always delete/recreate. > 64-bit Outlook is needed if both (a) your data file is > 2 GB, and (b) > you want search indexing to work.? With 32-bit Outlook, search will > silently miss messages. :-O Thanks for pointing this out! > Outlook is inefficient at IMAP synchronization.? If you have large > mailboxes, it can spend a lot of its time synchronizing subscribed > folders.? There are ways of winnowing the list of folders that Outlook > tries to sync during Send And Receive, and this helps a bit. Now I remember this too... > Outlook by default wants to send read receipts. What I found really annoying is "non-read" receipts. In case a user loses view of a shared folders, OL will send a "non-read" receipt for any unread message, potentially queueing thousands of messages in a few seconds! This could be turned off in older versions; on post-2010 the option is either not there or hidden. >:-< > To sync Outlook with Cyrus contacts, calendars and tasks, the free > Outlook CalDav Synchronizer add-in works well (but sync rules are > tedious to set up, and I haven't found a way to deploy preconfigured > sync rules to users).? Outlook tries to disable the add-in because it > slows down startup, but you can prevent this with a registry setting. Thanks again. I don't think I'll need this soon, but it's useful to know. > Rarely, Outlook will decide that a folder is local-only, and any > messages moved into that folder will stop syncing with Cyrus.? The only > fix I have found is to create a new Outlook profile (and then hunt for > lost messages to drag back under synchronized folders). Yes, I remember this too :-< Does it at least shows the folder is local? Then I could train the users to stop using it and call me. > These corruptions seem to occur more often in "Other Users" and "Shared > Folders", but this might just be because said folders are huge (> 4 GB) > in my company. I hope I don't need to use this, at least for now. > Of course, it's unlikely that any of these irritations will ever get > fixed by Microsoft. That's sure. > In the long term, I am considering migrating my > users from Outlook to Thunderbird or webmail. All the users of my servers are using TB (along with mobile devices and occasionally RoundCube). Now I'm evaulating installing a new instance at a customer who is already using OutLook (so to overcome the limitations of their ISP, which only offers POP3). I'll suggest they move to ThunderBird but: a) habits are hard to lose; b) some use a management software who might require OL :-( bye & Thanks av. From debbiep at polyfoam.com.au Tue Sep 8 02:01:43 2020 From: debbiep at polyfoam.com.au (Deborah Pickett) Date: Tue, 8 Sep 2020 16:01:43 +1000 Subject: OutLook support In-Reply-To: <813ccd74-cff1-c164-f2d4-3d8839addb9d@netfence.it> References: <0a71dd93-e1c6-86ba-e24f-45583a409cba@polyfoam.com.au> <813ccd74-cff1-c164-f2d4-3d8839addb9d@netfence.it> Message-ID: <01d401d685a5$874ac290$95e047b0$@polyfoam.com.au> Hi Andrea, > What I found really annoying is "non-read" receipts. Ouch, I did not know about this! I will file it away in case it ever bites me. > > Rarely, Outlook will decide that a folder is local-only > > [...] > Does it at least shows the folder is local? > Then I could train the users to stop using it and call me. Haha, no, no one notices until (if you're lucky) someone else who sees the same folder in a shared account notices they can't see It, or (if you're not) the computer housing these unexpected local folders dies, and then you discover that the messages have been absent for months. From ruprechtsberger at volkshilfe-ooe.at Fri Sep 11 08:06:26 2020 From: ruprechtsberger at volkshilfe-ooe.at (Rainer Ruprechtsberger) Date: Fri, 11 Sep 2020 14:06:26 +0200 Subject: ptloader doesn't work anymore (SEGFAULT) Message-ID: Hi, I used ptlaoder quite successfully up to version 2.4 of cyrus. Since the upgrade to 2.5 I get SEGFAULTs. After the upgrade to 3.2.2 I tested it again, but still: process type:SERVICE name:ptloader path:/usr/lib/cyrus/bin/ptloader age:0.665s pid:19082 signaled to death by signal 11 (Segmentation fault) I tried to enter 'ptloader -d' in cyrus.conf but this did not work for me (no ptloader started at all). My environment is Debian Buster and I use the packages from backports. My ldap server is an openldap slapd accessed via local unix socket. Any hint on where and how to debug this would be appreciated. /rupi -- Rainer Ruprechtsberger Volkshilfe Ober?sterreich IT 4020 Linz, Glimpfingerstrasse 48 Tel.: +43 732 3405 123 Mobil.: +43 676 8734 1123 ZVR Zahl: 064371505 Volkshilfe. Wir sind f?r die Menschen da. From ismael.tanguy at univ-brest.fr Mon Sep 14 05:43:03 2020 From: ismael.tanguy at univ-brest.fr (=?UTF-8?Q?Isma=c3=abl_Tanguy?=) Date: Mon, 14 Sep 2020 11:43:03 +0200 Subject: How to find mailbox name from id Message-ID: <0f92f073-8544-19ba-3598-691719cb01cd@univ-brest.fr> Hello, cyrus-imapd-3.0.7-16.el8.src.rpm installed on Centos 8.2 An user fails to receive an email from? a mailing list of 1000 subscribers. From SMTP server, I found the uid of the mail and look for it in the IMAP store logs. While the delivery is OK for the most part, I found 5 mistakes : # grep 280aa530-f9a0-c1fd-b100-51201aa6b2fc /var/log/maillog | grep -v Delivered dupelim: eliminated duplicate message to i21mdycbfokfu8yq6jul3hvt id <280aa530-f9a0-c1fd-b100-51201aa6b2fc> dupelim: eliminated duplicate message to gpcnws97fd5kke9ywveu8ms5 dupelim: eliminated duplicate message to izl2yz009e4dr9q0fnq4fls0 dupelim: eliminated duplicate message to e4l4zysb2f14o3ins52g2nb1 dupelim: eliminated duplicate message to ziaim4ec3kh0sxwf1angvq21 dupelim: eliminated duplicate message to 74zgw35o41swdp4aixmo7b8f dupelim: eliminated duplicate message to iualfalhbrp0lq3mk37uxnx9 With the mailbox name of this user, it's easy to found the Unique ID of the mailbox : $ mbexamine user.e22005103 ?Mailbox ACL: e22005103??????? lrswipkxtecdan ? Unique ID: 74zgw35o41swdp4aixmo7b8f Then, I would like to know the mailbox names corresponding to the uids reported in the logs. How can I do it? Is there any commands? For the problem of eliminating duplicate message, singleinstancestore is activated on cyrus-imapd. Do you think the problem is coming from this parameter? Thank you, Isma?l Tanguy Universit? de Bretagne Occidentale Brest - France -- From falon at ruparpiemonte.it Tue Sep 15 03:21:40 2020 From: falon at ruparpiemonte.it (Marco) Date: Tue, 15 Sep 2020 09:21:40 +0200 Subject: How to find mailbox name from id In-Reply-To: <0f92f073-8544-19ba-3598-691719cb01cd@univ-brest.fr> References: <0f92f073-8544-19ba-3598-691719cb01cd@univ-brest.fr> Message-ID: <79ff94f9-5372-621a-c2ab-02256ed21405@ruparpiemonte.it> On 14/09/2020 11:43, Isma?l Tanguy has written: > Hello, > > cyrus-imapd-3.0.7-16.el8.src.rpm installed on Centos 8.2 > > An user fails to receive an email from? a mailing list of 1000 subscribers. > From SMTP server, I found the uid of the mail and look for it in the > IMAP store logs. > > While the delivery is OK for the most part, I found 5 mistakes : > > # grep 280aa530-f9a0-c1fd-b100-51201aa6b2fc /var/log/maillog | grep -v > Delivered > > dupelim: eliminated duplicate message to i21mdycbfokfu8yq6jul3hvt id > <280aa530-f9a0-c1fd-b100-51201aa6b2fc> [...] Hello, good question! btw... on Cyrus 2.4 these logs told me the mailbox name too. On Cyrus 3.x this info disappears. Ouch :( If you delete the mailbox the syslog shows the mailbox name and the uniqueid together, only dupelim lacks of this info. Cheers Marco From sofkam at rpi.edu Tue Sep 15 10:49:00 2020 From: sofkam at rpi.edu (Michael Sofka) Date: Tue, 15 Sep 2020 10:49:00 -0400 Subject: Sync failing. Possible db corruption Message-ID: <20cc5ddb-266e-b389-f116-c78eec23d681@rpi.edu> Before I proceed, I am seeking advice. This is on Cyrus IMAP Murder v2.4.18-Debian-2.4.18-3 On one of our back-end servers sync started failing.? I restarted sync_server on the replication, and on the backend server, which did not work.? There are the following errors in the logs Sep 15 10:04:52 imap-be6 cyrus/sync_client[29665]: Processing sync log file /var/lib/cyrus/sync/l og-29665 failed: Bad protocol Sep 15 10:04:52 imap-be6 cyrus/sync_client[29665]: DBERROR db5: BDB1581 File handles still open a t environment close Sep 15 10:04:52 imap-be6 cyrus/sync_client[29665]: DBERROR db5: BDB1582 Open file handle: /var/li b/cyrus/db/__db.001 Sep 15 10:04:52 imap-be6 cyrus/sync_client[29665]: DBERROR db5: BDB1582 Open file handle: /var/li b/cyrus/db/__db.002 Sep 15 10:04:52 imap-be6 cyrus/sync_client[29665]: DBERROR db5: BDB1582 Open file handle: /var/li b/cyrus/db/__db.003 Sep 15 10:04:52 imap-be6 cyrus/sync_client[29665]: DBERROR db5: BDB0060 PANIC: fatal region error ?detected; run recovery Sep 15 10:04:52 imap-be6 cyrus/sync_client[29665]: DBERROR: critical database situation ... Sep 15 10:21:18 imap-be6 cyrus/sync_client[19816]: Inbox missing on master for MAILBOX user.travi t Sep 15 10:21:18 imap-be6 cyrus/sync_client[19816]: Inbox missing on master for MAILBOX user.sutto e Sep 15 10:21:18 imap-be6 cyrus/sync_client[19816]: Inbox missing on master for MAILBOX user.maran v On the replication server I have: Sep 15 10:34:40 imap-be5 cyrus/syncserver[31550]: cannot unlink /var/lib/cyrus/user/m/MAILBOX use r.linr7.mboxkey: No such file or directory Sep 15 10:34:40 imap-be5 cyrus/syncserver[31550]: cannot unlink /var/lib/cyrus/user/m/MAILBOX use r.kjells.mboxkey: No such file or directory Sep 15 10:34:40 imap-be5 cyrus/syncserver[31550]: cannot unlink /var/lib/cyrus/user/m/MAILBOX use r.ngang.mboxkey: No such file or directory When I try to process the sync/log-xxxx files. Any pointers or help appreciated.? IMAP is functioning as far as I can tell, and only the replication process is failing. Thank you, MIke -- -- Michael D. Sofka sofkam at rpi.edu ITI Software Architect, Email, TeX, Epistemology Rensselaer Polytechnic Institute, Troy, NY. http://www.rpi.edu/~sofkam/ From sofkam at rpi.edu Tue Sep 15 11:40:39 2020 From: sofkam at rpi.edu (Michael Sofka) Date: Tue, 15 Sep 2020 11:40:39 -0400 Subject: Sync failing. Possible db corruption In-Reply-To: <20cc5ddb-266e-b389-f116-c78eec23d681@rpi.edu> References: <20cc5ddb-266e-b389-f116-c78eec23d681@rpi.edu> Message-ID: <4278ce2a-7f4e-04e8-e190-fd5f99680e5b@rpi.edu> Okay, these errors may be admin error. In running the old logs I specified -u, instead of -m. I am still concerned about the DBERROR, but those did not appear after I restarted cyrus.? And rolling replication is still not working. Mike On 9/15/20 10:49 AM, Michael Sofka wrote: > > Sep 15 10:21:18 imap-be6 cyrus/sync_client[19816]: Inbox missing on > master for MAILBOX user.travi > t > Sep 15 10:21:18 imap-be6 cyrus/sync_client[19816]: Inbox missing on > master for MAILBOX user.sutto > e > Sep 15 10:21:18 imap-be6 cyrus/sync_client[19816]: Inbox missing on > master for MAILBOX user.maran > v > > > On the replication server I have: > > Sep 15 10:34:40 imap-be5 cyrus/syncserver[31550]: cannot unlink > /var/lib/cyrus/user/m/MAILBOX use > r.linr7.mboxkey: No such file or directory > Sep 15 10:34:40 imap-be5 cyrus/syncserver[31550]: cannot unlink > /var/lib/cyrus/user/m/MAILBOX use > r.kjells.mboxkey: No such file or directory > Sep 15 10:34:40 imap-be5 cyrus/syncserver[31550]: cannot unlink > /var/lib/cyrus/user/m/MAILBOX use > r.ngang.mboxkey: No such file or directory > > When I try to process the sync/log-xxxx files. > > > Any pointers or help appreciated.? IMAP is functioning as far as I can > tell, and only the replication process is failing. > > Thank you, > > MIke > > -- -- Michael D. Sofka sofkam at rpi.edu ITI Software Architect, Email, TeX, Epistemology Rensselaer Polytechnic Institute, Troy, NY. http://www.rpi.edu/~sofkam/ From sofkam at rpi.edu Tue Sep 15 16:20:34 2020 From: sofkam at rpi.edu (Michael Sofka) Date: Tue, 15 Sep 2020 16:20:34 -0400 Subject: Sync failing. Possible db corruption In-Reply-To: <4278ce2a-7f4e-04e8-e190-fd5f99680e5b@rpi.edu> References: <20cc5ddb-266e-b389-f116-c78eec23d681@rpi.edu> <4278ce2a-7f4e-04e8-e190-fd5f99680e5b@rpi.edu> Message-ID: <2280434f-b659-88dc-c64d-611619bf163c@rpi.edu> I have confirmed that running sync_client on individual mailboxes works, as does running sync on the accumulated log files.? But rolling replication is not working. ? After sync_client is restarted there is a burst of sync activity on the accumulated log, then nothing.? The new log file continues to accumulate, sync_client continues to run, but is not processing anything so far as I can determine. Sep 15 15:59:17 imap-be6 cyrus/sync_client[22370]: SYNCNOTICE: replica uid:7451 modseq:19935 last_updated:1600023598 internaldate:1600023598 flags:( \Seen) Sep 15 15:59:17 imap-be6 cyrus/sync_client[22370]: inefficient replication (1487 > 1484) user.jins4.Sent Sep 15 15:59:18 imap-be6 cyrus/sync_client[22370]: inefficient replication (182427 > 182409) user.newelj Sep 15 15:59:18 imap-be6 cyrus/sync_client[22370]: SYNCERROR: only exists on replica user.newelj 121180 (a27c1dffed0326fb930b072fd85f04a8c2cbdf24) Sep 15 15:59:23 imap-be6 cyrus/sync_client[22370]: inefficient replication (5598 > 5571) user.berryk3 Sep 15 15:59:23 imap-be6 cyrus/sync_client[22370]: SYNCNOTICE: record mismatch with replica: user.berryk3 more recent on master Sep 15 15:59:23 imap-be6 cyrus/sync_client[22370]: SYNCNOTICE: master uid:2614 modseq:5636 last_updated:1600191010 internaldate:1599245383 flags:(\Seen) From the previous day's log I see: Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: MAILBOX received NO response: System I/O error Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: do_folders(): update failed: user.chauhs2 'The remote Server(s) denied the operation' Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: seen_db: user swankd opened /var/lib/cyrus/user/s/swankd.seen Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: seen_db: user edena2 opened /var/lib/cyrus/user/e/edena2.seen Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: seen_db: user searsm2 opened /var/lib/cyrus/user/s/searsm2.seen Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: USER received NO response: System I/O error Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: Error in do_sync(): bailing out! The remote Server(s) denied the operation Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: Processing sync log file /var/lib/cyrus/sync/log-29665 failed: The remote Server(s) denied the operation Sep 15 06:25:11 imap-be6 cyrus/sync_client[29665]: Reprocessing sync log file /var/lib/cyrus/sync/log-29665 I have not seen a repeat of the DBERROR, and have since restarted cyrus services.? So I'm not sure a ctl_cyrusdb -r is needed, or even it it would fix things. -- -- Michael D. Sofkasofkam at rpi.edu ITI Software Architect, Email, TeX, Epistemology Rensselaer Polytechnic Institute, Troy, NY.http://www.rpi.edu/~sofkam/ From ismael.tanguy at univ-brest.fr Wed Sep 16 06:01:06 2020 From: ismael.tanguy at univ-brest.fr (=?UTF-8?Q?Isma=c3=abl_Tanguy?=) Date: Wed, 16 Sep 2020 12:01:06 +0200 Subject: How to find mailbox name from id In-Reply-To: <79ff94f9-5372-621a-c2ab-02256ed21405@ruparpiemonte.it> References: <0f92f073-8544-19ba-3598-691719cb01cd@univ-brest.fr> <79ff94f9-5372-621a-c2ab-02256ed21405@ruparpiemonte.it> Message-ID: <31805dad-2fa1-9ad4-21b8-58372395e1ee@univ-brest.fr> > On 14/09/2020 11:43, Isma?l Tanguy has written: >> Hello, >> >> cyrus-imapd-3.0.7-16.el8.src.rpm installed on Centos 8.2 >> >> An user fails to receive an email from? a mailing list of 1000 >> subscribers. >> ?From SMTP server, I found the uid of the mail and look for it in the >> IMAP store logs. >> >> While the delivery is OK for the most part, I found 5 mistakes : >> >> # grep 280aa530-f9a0-c1fd-b100-51201aa6b2fc /var/log/maillog | grep >> -v Delivered >> >> dupelim: eliminated duplicate message to i21mdycbfokfu8yq6jul3hvt id >> <280aa530-f9a0-c1fd-b100-51201aa6b2fc> > [...] > > Hello, > ?good question! I answer myself. With this command : $ cyr_dbtool /var/lib/imap/mailboxes.db twoskip show | grep i21mdycbfokfu8yq6jul3hvt But I found two mailboxes : user.e22009540? %(A %(e22009540 lrswipkxtecdan) I i21mdycbfokfu8yq6jul3hvt P splitmeta V 1599471223 F 1 M 1599471222) user.e22009563? %(A %(e22009563 lrswipkxtecdan) I i21mdycbfokfu8yq6jul3hvt P splitmeta V 1599471224 F 1 M 1599471223) Is it the expected behavior? I thought Unique ID was unique for one mailbox.. > > btw... on Cyrus 2.4 these logs told me the mailbox name too. On Cyrus > 3.x this info disappears. Ouch :( > > If you delete the mailbox the syslog shows the mailbox name and the > uniqueid together, only dupelim lacks of this info. > > Cheers > Marco > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus From sofkam at rpi.edu Wed Sep 16 15:31:03 2020 From: sofkam at rpi.edu (Michael Sofka) Date: Wed, 16 Sep 2020 15:31:03 -0400 Subject: Sync failing. Possible db corruption In-Reply-To: <2280434f-b659-88dc-c64d-611619bf163c@rpi.edu> References: <20cc5ddb-266e-b389-f116-c78eec23d681@rpi.edu> <4278ce2a-7f4e-04e8-e190-fd5f99680e5b@rpi.edu> <2280434f-b659-88dc-c64d-611619bf163c@rpi.edu> Message-ID: <88b277b3-b919-730e-4f23-d3f547353ab0@rpi.edu> Problem resolved, or resolving.? This version of Cyrus may be old, but it's been rock solid and compared to previous versions I've had no problems with sync_client dying. It turns out the behavior I was seeing is normal recover behavior.? There are still errors I have not seen before, but they appear to be from turning on verbose logging, and the synchronization being out of step from the previous errors.? As people check mailboxes, it is catching up. Mike On 9/15/20 4:20 PM, Michael Sofka wrote: > I have confirmed that running sync_client on individual mailboxes > works, as does running sync on the accumulated log files.? But rolling > replication is not working. ? After sync_client is restarted there is > a burst of sync activity on the accumulated log, then nothing. The new > log file continues to accumulate, sync_client continues to run, but is > not processing anything so far as I can determine. > > -- -- Michael D. Sofka sofkam at rpi.edu ITI Software Architect, Email, TeX, Epistemology Rensselaer Polytechnic Institute, Troy, NY. http://www.rpi.edu/~sofkam/ From thomas.fricker at bluemind.net Mon Sep 21 11:00:45 2020 From: thomas.fricker at bluemind.net (Thomas Fricker) Date: Mon, 21 Sep 2020 17:00:45 +0200 Subject: Mailshare conversations Message-ID: <4029b2bdcb35a835f832a6e7c4112bf3@blue-mind.net> Hello, I noticed that Cyrus does not keep track of conversations in mailshare folders. There is a code comment addressing this issue in conversations.c: /* only users have conversations. Later we may introduce the idea of * "conversationroot" in the same way we have quotaroot, but for now * it's hard-coded as the user */ Is it still imaginable adding this feature in a future Cyrus version? Can you elaborate on a reasonable approach of implementing this on the current codebase? My goal is to have 1 conversation.db per mailshare subtree (not per single folder). Any ideas? Thanks, -- Thomas Fricker Thomas Fricker DEVELOPPEUR BlueMind +33 (0)5 81 91 55 60 Hotel des T?l?coms, 40 rue du village d'entreprises 31670 Lab?ge, France www.bluemind.net / https://blog.bluemind.net/fr/ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: x-disclaimer1569892563-0.png Type: image/png Size: 858 bytes Desc: x-disclaimer1569892563-0.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: x-disclaimer1569892563-1.png Type: image/png Size: 432 bytes Desc: x-disclaimer1569892563-1.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: x-disclaimer1569892563-2.png Type: image/png Size: 400 bytes Desc: x-disclaimer1569892563-2.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: x-disclaimer1569892563-3.png Type: image/png Size: 407 bytes Desc: x-disclaimer1569892563-3.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: x-disclaimer1569892563-4.png Type: image/png Size: 2333 bytes Desc: x-disclaimer1569892563-4.png URL: From delepine at u-picardie.fr Wed Sep 23 11:44:38 2020 From: delepine at u-picardie.fr (Jean Charles =?utf-8?B?RGVsw6lwaW5l?=) Date: Wed, 23 Sep 2020 17:44:38 +0200 Subject: Replication : IMAP_PROTOCOL_ERROR Protocol error Message-ID: <20200923154438.GK6698@u-picardie.fr> Hello, I'm on the way to migrate one quite big murder config with Cyrus IMAP 3.0.8-Debian-3.0.8-6+deb10u4 to Cyrus IMAP 3.2.3-Debian-3.2.3-1~bpo10+1. My plan is to replicate 3.0.8's backends on 3.2.3 ones. This plan has work before for 2.5 to 3.0. migration. Il can replicate empty mailboxes. So the conf (attached) seems ok. But if the mailbox isn't empty here's the result (completes traces attached) : cyrus/sync_client[3082351]: MAILBOX received NO response: IMAP_PROTOCOL_ERROR Protocol error cyrus/sync_client[3082351]: do_folders(): update failed: user.t 'Bad protocol' cyrus/sync_client[3082351]: IOERROR: do_user_main: Bad protocol for t to [no channel] (test-3.2.3) Error from sync_do_user(t): bailing out! cyrus/sync_client[3082351]: Error in sync_do_user(t): bailing out! The destination server says : SYNCERROR: failed to parse uploaded record If I upgrade this test server to 3.2.3-Debian-3.2.3-1~bpo10+1 with the same configuration, the same mailbox can be sync whith the problem being corrected : APPLY MAILBOX %(UNIQUEID 6kjf8ro4032wfjefcdewaqyl MBOXNAME user.t SYNC_CRC 3435668400 SYNC_CRC_ANNOT 1359939586 LAST_UID 1 HIGHESTMODSEQ 3 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 3 UIDVALIDITY 1600855436 PARTITION default ACL "" OPTIONS P FOLDERMODSEQ 3 SINCE_MODSEQ 3 SINCE_CRC 0 SINCE_CRC_ANNOT 12345678 RECORD ()) <1600872848 1600872848>APPLY MAILBOX %(UNIQUEID 6kjf8ro4032wfjefcdewaqyl > MBOXNAME user.t SYNC_CRC 3435668400 SYNC_CRC_ANNOT 1370712396 > LAST_UID 1 HIGHESTMODSEQ 3 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE > 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 3 UIDVALIDITY > 1600855436 PARTITION default ACL "" OPTIONS P FOLDERMODSEQ 3 RECORD > ()) <1600872848 1600872848>GET FULLMAILBOX user.t <1600872848<* MAILBOX %(UNIQUEID 6kjf8ro4032wfjefcdewaqyl MBOXNAME user.t SYNC_CRC 0 SYNC_CRC_ANNOT 12345678 LAST_UID 1 HIGHESTMODSEQ 3 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 3 UIDVALIDITY 1600855436 PARTITION default ACL "" OPTIONS P FOLDERMODSEQ 3 RECORD ()) OK success cyrus/sync_client[3131948]: user.t: same message appears twice 1 2 MAILBOX user.t > 1600872849>APPLY MESSAGE (%{default > f0eeef2cc42f23884089760cf5de1961b358a498 2627} ) <1600872849 1600872849>APPLY MAILBOX %(UNIQUEID 6kjf8ro4032wfjefcdewaqyl > MBOXNAME user.t SYNC_CRC 1009437458 SYNC_CRC_ANNOT 455745080 > LAST_UID 2 HIGHESTMODSEQ 5 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE > 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 5 UIDVALIDITY > 1600855436 PARTITION default ACL "" OPTIONS P FOLDERMODSEQ 3 RECORD > (%(UID 1 MODSEQ 5 LAST_UPDATED 1600872848 FLAGS (\Expunged) > INTERNALDATE 1600855652 SIZE 2627 GUID > f0eeef2cc42f23884089760cf5de1961b358a498 ANNOTATIONS (%(ENTRY > /vendor/cmu/cyrus-imapd/thrid USERID "" MODSEQ 0 VALUE > 611d36cdcf46289a))) %(UID 2 MODSEQ 4 LAST_UPDATED 1600872848 FLAGS > () INTERNALDATE 1600855652 SIZE 2627 GUID > f0eeef2cc42f23884089760cf5de1961b358a498 ANNOTATIONS (%(ENTRY > /vendor/cmu/cyrus-imapd/thrid USERID "" MODSEQ 0 VALUE > 611d36cdcf46289a))))) <1600872849 1600872849>APPLY MAILBOX %(UNIQUEID nckzm4o710aom3c22o9ugy6a > MBOXNAME user.t.Drafts SYNC_CRC 0 SYNC_CRC_ANNOT 0 LAST_UID 0 > HIGHESTMODSEQ 2 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 > POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 2 UIDVALIDITY > 1600855557 PARTITION default ACL "" OPTIONS P FOLDERMODSEQ 2 RECORD > ()) <1600872849 1600872849>EXIT <1600872849 remote= >1600855733>COMPRESS DEFLATE <16008557331600855733>GET USER t <16008557331600855733>APPLY MESSAGE (%{default f0eeef2cc42f23884089760cf5de1961b358a498 2627} Return-Path: [... some mail ] ) <16008557331600855733>APPLY MAILBOX %(UNIQUEID 6kjf8ro4032wfjefcdewaqyl MBOXNAME user.t SYNC_CRC 3435668400 SYNC_CRC_ANNOT 1359939586 LAST_UID 1 HIGHESTMODSEQ 3 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 3 UIDVALIDITY 1600855436 PARTITION default ACL "" OPTIONS P RECORD (%(UID 1 MODSEQ 3 LAST_UPDATED 1600855676 FLAGS () INTERNALDATE 1600855652 SIZE 2627 GUID f0eeef2cc42f23884089760cf5de1961b358a498 ANNOTATIONS (%(ENTRY /vendor/cmu/cyrus-imapd/thrid USERID NIL VALUE 611d36cdcf46289a))))) <16008557331600855733>EXIT <1600855733 remote= >1600872848>COMPRESS DEFLATE <16008728481600872848>GET USER t <1600872848<* MAILBOX %(UNIQUEID 6kjf8ro4032wfjefcdewaqyl MBOXNAME user.t SYNC_CRC 0 SYNC_CRC_ANNOT 12345678 LAST_UID 0 HIGHESTMODSEQ 3 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 3 UIDVALIDITY 1600855436 PARTITION default ACL "" OPTIONS P CREATEDMODSEQ 3 FOLDERMODSEQ 3) OK success cyrus/sync_client[3131948]: skiplist: recovered /var/lib/cyrus/user/t/t.conversations (8 records, 1184 bytes) in 0 seconds cyrus/sync_client[3131948]: skiplist: checkpointed /var/lib/cyrus/user/t/t.conversations (8 records, 796 bytes) in 0.002 sec >1600872848>APPLY RESERVE %(PARTITION default MBOXNAME (user.t) GUID (f0eeef2cc42f23884089760cf5de1961b358a498)) <1600872848<* MISSING (f0eeef2cc42f23884089760cf5de1961b358a498) OK success MAILBOX user.t >1600872848>APPLY MAILBOX %(UNIQUEID 6kjf8ro4032wfjefcdewaqyl MBOXNAME user.t SYNC_CRC 3435668400 SYNC_CRC_ANNOT 1359939586 LAST_UID 1 HIGHESTMODSEQ 3 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 3 UIDVALIDITY 1600855436 PARTITION default ACL "" OPTIONS P FOLDERMODSEQ 3 SINCE_MODSEQ 3 SINCE_CRC 0 SINCE_CRC_ANNOT 12345678 RECORD ()) <16008728481600872848>APPLY MAILBOX %(UNIQUEID 6kjf8ro4032wfjefcdewaqyl MBOXNAME user.t SYNC_CRC 3435668400 SYNC_CRC_ANNOT 1370712396 LAST_UID 1 HIGHESTMODSEQ 3 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 3 UIDVALIDITY 1600855436 PARTITION default ACL "" OPTIONS P FOLDERMODSEQ 3 RECORD ()) <16008728481600872848>GET FULLMAILBOX user.t <1600872848<* MAILBOX %(UNIQUEID 6kjf8ro4032wfjefcdewaqyl MBOXNAME user.t SYNC_CRC 0 SYNC_CRC_ANNOT 12345678 LAST_UID 1 HIGHESTMODSEQ 3 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 3 UIDVALIDITY 1600855436 PARTITION default ACL "" OPTIONS P FOLDERMODSEQ 3 RECORD ()) OK success cyrus/sync_client[3131948]: user.t: same message appears twice 1 2 MAILBOX user.t >1600872849>APPLY MESSAGE (%{default f0eeef2cc42f23884089760cf5de1961b358a498 2627} Return-Path: ) <16008728491600872849>APPLY MAILBOX %(UNIQUEID 6kjf8ro4032wfjefcdewaqyl MBOXNAME user.t SYNC_CRC 1009437458 SYNC_CRC_ANNOT 455745080 LAST_UID 2 HIGHESTMODSEQ 5 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 5 UIDVALIDITY 1600855436 PARTITION default ACL "" OPTIONS P FOLDERMODSEQ 3 RECORD (%(UID 1 MODSEQ 5 LAST_UPDATED 1600872848 FLAGS (\Expunged) INTERNALDATE 1600855652 SIZE 2627 GUID f0eeef2cc42f23884089760cf5de1961b358a498 ANNOTATIONS (%(ENTRY /vendor/cmu/cyrus-imapd/thrid USERID "" MODSEQ 0 VALUE 611d36cdcf46289a))) %(UID 2 MODSEQ 4 LAST_UPDATED 1600872848 FLAGS () INTERNALDATE 1600855652 SIZE 2627 GUID f0eeef2cc42f23884089760cf5de1961b358a498 ANNOTATIONS (%(ENTRY /vendor/cmu/cyrus-imapd/thrid USERID "" MODSEQ 0 VALUE 611d36cdcf46289a))))) <16008728491600872849>APPLY MAILBOX %(UNIQUEID nckzm4o710aom3c22o9ugy6a MBOXNAME user.t.Drafts SYNC_CRC 0 SYNC_CRC_ANNOT 0 LAST_UID 0 HIGHESTMODSEQ 2 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 2 UIDVALIDITY 1600855557 PARTITION default ACL "" OPTIONS P FOLDERMODSEQ 2 RECORD ()) <16008728491600872849>EXIT <1600872849 References: <20200923154438.GK6698@u-picardie.fr> Message-ID: <20200924133616.GC41724@u-picardie.fr> Jean Charles Del?pine ?crivait (wrote) : > Hello, > > I'm on the way to migrate one quite big murder config with Cyrus IMAP > 3.0.8-Debian-3.0.8-6+deb10u4 > to Cyrus IMAP 3.2.3-Debian-3.2.3-1~bpo10+1. > > My plan is to replicate 3.0.8's backends on 3.2.3 ones. This plan has work > before for 2.5 > to 3.0. migration. > > Il can replicate empty mailboxes. So the conf (attached) seems ok. > But if the mailbox isn't empty here's the result (completes traces attached) : The conf might not be _that_ ok. If the option conversation is set to 1 and I create new mailboxes, ond send mails to this mailboxes, no sync of these mailboxes is possible. If I remove the option 'conversation: 1', any new populated mailboxes can be sync. So, the problem seems to be in this option. But even after removing "conversation: 1" and zeroed conversation indexes (ctl_conversationsdb -z), the old "bad" mailboxes can't be synced. Even removing their .conversations and .counters files doesn't help. Can I, and how can I, get rid of those conversation indexes in order to have my mailboxes being "as if there never been conversation" ? Sincerly, Jean charles Del?pine From ellie at fastmail.com Thu Sep 24 19:12:30 2020 From: ellie at fastmail.com (ellie timoney) Date: Fri, 25 Sep 2020 09:12:30 +1000 Subject: Replication : IMAP_PROTOCOL_ERROR Protocol error In-Reply-To: <20200923154438.GK6698@u-picardie.fr> References: <20200923154438.GK6698@u-picardie.fr> Message-ID: <3b55f7c3-dffc-4ea6-bd6b-aaf34bda6c81@www.fastmail.com> On Thu, 24 Sep 2020, at 1:44 AM, Jean Charles Del?pine wrote: > Is this a known problem corrected after 3.0.9 ? Off the top of my head I no longer remember, but the current release in the 3.0 series is 3.0.14. I'd suggest, if you haven't already, that you look in the release notes from 3.0.8-3.0.14 and see if anything looks relevant. They're here: https://www.cyrusimap.org/imap/download/release-notes/3.0/index.html We don't generally recommend in-place upgrades between series (so, upgrading in-place from 3.0.8 to 3.2.3 is not something we'd recommend). Within-series, an in-place upgrade ought to be safe -- but please check the release notes carefully for extra steps/considerations you may need to make, depending on your deployment. I think you'll probably want to upgrade your 3.0 systems in place as far forward as you can (while staying 3.0), and then use the replication strategy to upgrade to 3.2 after that. -------------- next part -------------- An HTML attachment was scrubbed... URL: From delepine at u-picardie.fr Sat Sep 26 07:15:23 2020 From: delepine at u-picardie.fr (Jean Charles =?utf-8?B?RGVsw6lwaW5l?=) Date: Sat, 26 Sep 2020 13:15:23 +0200 Subject: Replication : IMAP_PROTOCOL_ERROR Protocol error In-Reply-To: <3b55f7c3-dffc-4ea6-bd6b-aaf34bda6c81@www.fastmail.com> References: <20200923154438.GK6698@u-picardie.fr> <3b55f7c3-dffc-4ea6-bd6b-aaf34bda6c81@www.fastmail.com> Message-ID: <20200926111523.GA147144@u-picardie.fr> ellie timoney ?crivait (wrote) : > On Thu, 24 Sep 2020, at 1:44 AM, Jean Charles Del?pine wrote: > > Is this a known problem corrected after 3.0.9 ? > > Off the top of my head I no longer remember, but the current release in the 3.0 series is 3.0.14. I'd suggest, if you haven't already, that you look in the release notes from 3.0.8-3.0.14 and see if anything looks relevant. They're here: https://www.cyrusimap.org/imap/download/release-notes/3.0/index.html I didn't find anything relevant. > We don't generally recommend in-place upgrades between series (so, upgrading in-place from 3.0.8 to 3.2.3 is not something we'd recommend). Within-series, an in-place upgrade ought to be safe -- but please check the release notes carefully for extra steps/considerations you may need to make, depending on your deployment. I think you'll probably want to upgrade your 3.0 systems in place as far forward as you can (while staying 3.0), and then use the replication strategy to upgrade to 3.2 after that. I just do that. My test server is now using 3.0.14 (self build debian package) : >1601118406>APPLY MAILBOX %(UNIQUEID m8lfz12835tr5y3dfucebk95 MBOXNAME user.testes2 SYNC_CRC 2393559122 SYNC_CRC_ANNOT 4164967045 LAST_UID 2 HIGHESTMODSEQ 4 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 1601117744 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 XCONVMODSEQ 4 UIDVALIDITY 1601117689 PARTITION default ACL "testes2 lrswipkxtecdan " OPTIONS P RECORD (%(UID 1 MODSEQ 4 LAST_UPDATED 1601118406 FLAGS (\Expunged) INTERNALDATE 1601117744 SIZE 2890 GUID 6f160d7026f4adbaebb2d0941c6398272a8692da ANNOTATIONS (%(ENTRY /vendor/cmu/cyrus-imapd/thrid USERID NIL VALUE fee5d4912d9da6a8))) %(UID 2 MODSEQ 3 LAST_UPDATED 1601118406 FLAGS () INTERNALDATE 1601117744 SIZE 2890 GUID 6f160d7026f4adbaebb2d0941c6398272a8692da ANNOTATIONS (%(ENTRY /vendor/cmu/cyrus-imapd/thrid USERID NIL VALUE fee5d4912d9da6a8))))) <16011184061601118406>EXIT <1601118406 Hello list, I had a working replication between to cyrus 3.0 servers running. The sync_client is started once a day. In the last days the sync_client bails out with do_folders: failed to order folders correctly every time on a specific mailbox of a specific user. I can't see that there is a problem with the mailbox ( cyr_expire and squatter are running correctly ). How can I get the sync_client work again? thanks in advance and kind regards, Konrad -- Konrad Mauz - Rechenzentrum HTWG Konstanz Tel: +497531206472 - Fax: +497531206153 E-Mail: kmauz at htwg-konstanz.de Web: http://www.htwg-konstanz.de/rz.html