From paul at vandervlis.nl Mon Oct 1 04:04:57 2018 From: paul at vandervlis.nl (Paul van der Vlis) Date: Mon, 1 Oct 2018 10:04:57 +0200 Subject: Slow start of imap service In-Reply-To: <1538361123.2738140.1525959928.7D8240C3@webmail.messagingengine.com> References: <20180928153446.Horde.oxWiSjgdecguOsIQkrEiVi1@webmail.uni-tuebingen.de> <1538361123.2738140.1525959928.7D8240C3@webmail.messagingengine.com> Message-ID: Op 01-10-18 om 04:32 schreef ellie timoney: > > > On Sat, Sep 29, 2018, at 8:59 AM, Paul van der Vlis wrote: >> Op 28-09-18 om 15:34 schreef Michael Menge: >>> >>> Quoting Paul van der Vlis : >>> >>>> Hello, >>>> >>>> I am using Cyrus-imapd from Debian stable (2.5.10-3), and starting up >>>> takes very long. I see processes starting, but no imapd. >>>> >>>> In most cases I restart Cyrus more then ones before it works. Not sure I >>>> have to wait longer, or restarting after some time helps. >>>> >>>> This problem occurs on only one machine, on two other less busy machine >>>> with the same Cyrus I don't have problems. >>>> >>>> Maybe somebody here knows more about what could be wrong? Or how to >>>> debug this? >>>> >>> >>> >>> What is cyrus logging to your logfiles when you restart? >> >> In my crontab I have this line: >> 00 4 * * * root /usr/sbin/service cyrus-imapd restart >> >> First I see many of this lines in /var/log/mail.log: >> Sep 25 04:00:01 sigmund cyrus/imap[21598]: graceful shutdown >> >> Then I see this between those lines this: >> ----- >> Sep 25 04:00:02 sigmund cyrus/idled[5844]: graceful shutdown initiated >> by unexpected process 5838 (/usr/sbin/cyrmaster -l 32 -C /etc/imapd.conf >> -M /etc/cyrus.conf) >> Sep 25 04:00:02 sigmund cyrus/imaps[16434]: IDLE: error sending message >> DONE to idled for mailbox user.nospam.Junk: Connection refused. >> ----- >> >> This line: >> Sep 25 04:00:02 sigmund cyrus/master[5838]: process type:SERVICE >> name:notify path:/usr/lib/cyrus/bin/notifyd age:85080.426s pid:6024 >> exited, status 75 >> >> Many of these lines: >> Sep 25 04:00:02 sigmund cyrus/master[5838]: process type:SERVICE >> name:imap path:/usr/lib/cyrus/bin/imapd age:85073.234s pid:6027 exited, >> status 75 >> >> Then this: >> -------- >> Sep 25 04:00:05 sigmund cyrus/ctl_cyrusdb[21829]: skiplist: clean >> shutdown file missing, updating recovery stamp >> Sep 25 04:00:05 sigmund cyrus/ctl_cyrusdb[21829]: recovering cyrus databases >> Sep 25 04:00:05 sigmund cyrus/ctl_cyrusdb[21829]: done recovering cyrus >> databases >> Sep 25 04:00:05 sigmund cyrus/cyr_expire[21834]: skiplist: recovered >> /var/lib/cyrus/deliver.db (9290 records, 1759220 bytes) in 0 seconds >> Sep 25 04:00:05 sigmund cyrus/cyr_expire[21834]: skiplist: checkpointed >> /var/lib/cyrus/deliver.db (9290 records, 1412288 bytes) in 0.227 sec >> Sep 25 04:00:19 sigmund cyrus/cyr_expire[21834]: Expired 0 and expunged >> 0 out of 1312483 messages from 2984 mailboxes >> Sep 25 04:00:19 sigmund cyrus/cyr_expire[21834]: duplicate_prune: >> pruning back 3.00 days >> Sep 25 04:00:30 sigmund cyrus/cyr_expire[21834]: skiplist: longlock >> /var/lib/cyrus/deliver.db for 1.8 seconds >> Sep 25 04:00:33 sigmund cyrus/cyr_expire[21834]: skiplist: longlock >> /var/lib/cyrus/deliver.db for 2.2 seconds >> Sep 25 04:00:39 sigmund cyrus/cyr_expire[21834]: skiplist: longlock >> /var/lib/cyrus/deliver.db for 1.3 seconds >> Sep 25 04:05:36 sigmund cyrus/cyr_expire[21834]: duplicate_prune: purged >> 2217 out of 9290 entries >> Sep 25 04:05:36 sigmund cyrus/tls_prune[21860]: skiplist: recovered >> /var/lib/cyrus/tls_sessions.db (10219 records, 2235748 bytes) in 0 seconds >> Sep 25 04:05:36 sigmund cyrus/tls_prune[21860]: skiplist: checkpointed >> /var/lib/cyrus/tls_sessions.db (10219 records, 2147768 bytes) in 0.308 sec >> Sep 25 04:09:47 sigmund cyrus/tls_prune[21860]: skiplist: longlock >> /var/lib/cyrus/tls_sessions.db for 1.4 seconds >> Sep 25 04:10:23 sigmund cyrus/tls_prune[21860]: skiplist: longlock >> /var/lib/cyrus/tls_sessions.db for 2.2 seconds >> Sep 25 04:10:45 sigmund cyrus/tls_prune[21860]: skiplist: longlock >> /var/lib/cyrus/tls_sessions.db for 2.2 seconds >> Sep 25 04:12:21 sigmund cyrus/tls_prune[21860]: skiplist: longlock >> /var/lib/cyrus/tls_sessions.db for 1.3 seconds >> Sep 25 04:12:47 sigmund cyrus/tls_prune[21860]: skiplist: longlock >> /var/lib/cyrus/tls_sessions.db for 1.0 seconds >> Sep 25 04:12:49 sigmund cyrus/tls_prune[21860]: skiplist: longlock >> /var/lib/cyrus/tls_sessions.db for 1.8 seconds >> Sep 25 04:17:33 sigmund cyrus/tls_prune[21860]: skiplist: longlock >> /var/lib/cyrus/tls_sessions.db for 1.0 seconds >> Sep 25 04:23:11 sigmund cyrus/tls_prune[21860]: skiplist: longlock >> /var/lib/cyrus/tls_sessions.db for 1.0 seconds >> Sep 25 04:25:31 sigmund cyrus/tls_prune[21860]: tls_prune: purged 4463 >> out of 10219 entries >> Sep 25 04:25:31 sigmund cyrus/master[21826]: unable to >> setsocketopt(IP_TOS) service lmtpunix/unix: Operation not supported >> Sep 25 04:25:31 sigmund cyrus/master[21826]: unable to >> setsocketopt(IP_TOS) service notify/unix: Operation not supported >> Sep 25 04:25:31 sigmund cyrus/ctl_cyrusdb[22345]: checkpointing cyrus >> databases >> Sep 25 04:25:31 sigmund cyrus/ctl_cyrusdb[22345]: done checkpointing >> cyrus databases >> Sep 25 04:25:32 sigmund cyrus/imaps[22349]: inittls: Loading hard-coded >> DH parameters >> Sep 25 04:25:33 sigmund cyrus/imaps[22349]: starttls: TLSv1.2 with >> cipher ECDHE-RSA-AES128-SHA (128/128 bits new) no authentication >> Sep 25 04:26:20 sigmund cyrus/imap[22362]: inittls: Loading hard-coded >> DH parameters >> Sep 25 04:26:20 sigmund cyrus/imap[22363]: inittls: Loading hard-coded >> DH parameters >> --------- >> >> So you can see imap is active after 25 minutes... >> >>> What is in the START section of your /etc/cyrus.conf? >> ---- >> START { >> # do not delete this entry! >> recover cmd="/usr/sbin/cyrus ctl_cyrusdb -r" >> >> # this is only necessary if idlemethod is set to "idled" in >> # imapd.conf >> idled cmd="idled" >> >> # this is useful on backend nodes of a Murder cluster >> # it causes the backend to syncronize its mailbox list with >> # the mupdate master upon startup >> #mupdatepush cmd="/usr/sbin/cyrus ctl_mboxlist -m" >> >> # this is recommended if using duplicate delivery suppression >> delprune cmd="/usr/sbin/cyrus expire -E 3" >> # this is recommended if caching TLS sessions >> tlsprune cmd="/usr/sbin/cyrus tls_prune" >> } >> --------- >> >> Thanks for you help! >> >> With regards, >> Paul van der Vlis > > Looks like most of the time (nearly 20 minutes) is being taken up by tls_prune cleaning up the tls_sessions.db: > >> Sep 25 04:05:36 sigmund cyrus/tls_prune[21860]: skiplist: recovered >> /var/lib/cyrus/tls_sessions.db (10219 records, 2235748 bytes) in 0 seconds >> Sep 25 04:05:36 sigmund cyrus/tls_prune[21860]: skiplist: checkpointed >> /var/lib/cyrus/tls_sessions.db (10219 records, 2147768 bytes) in 0.308 sec >> [...] >> Sep 25 04:25:31 sigmund cyrus/tls_prune[21860]: tls_prune: purged 4463 >> out of 10219 entries Thanks for your conclusion! > You could use the tls_sessions_db_path imapd.conf(5) option to put this database onto faster storage? > >> tls_sessions_db_path: >> The absolute path to the TLS sessions db file. If not specified, will be >> configdirectory/tls_sessions.db Interesting... I use normal harddisks with software raid and LVM. And a qcow2-image for the virtualisation. > If you have the RAM for it, you should be able to put tls_sessions.db on a tmpfs filesystem. This database is only a cache, so nothing valuable will be lost if the machine is rebooted; and as a cache, it benefits from being on the fastest storage you have available. :) I will think about it, not sure how big it is. With regards, Paul van der Vlis -- Paul van der Vlis Linux systeembeheer Groningen https://www.vandervlis.nl/ From paul at vandervlis.nl Mon Oct 1 04:32:14 2018 From: paul at vandervlis.nl (Paul van der Vlis) Date: Mon, 1 Oct 2018 10:32:14 +0200 Subject: Slow start of imap service In-Reply-To: <1538363604.2747884.1525991400.0ABFAD3C@webmail.messagingengine.com> References: <20180928153446.Horde.oxWiSjgdecguOsIQkrEiVi1@webmail.uni-tuebingen.de> <1538361123.2738140.1525959928.7D8240C3@webmail.messagingengine.com> <1538363604.2747884.1525991400.0ABFAD3C@webmail.messagingengine.com> Message-ID: Op 01-10-18 om 05:13 schreef ellie timoney: > On Mon, Oct 1, 2018, at 12:32 PM, ellie timoney wrote: >> You could use the tls_sessions_db_path imapd.conf(5) option to put this >> database onto faster storage? >> >>> tls_sessions_db_path: >>> The absolute path to the TLS sessions db file. If not specified, will be >>> configdirectory/tls_sessions.db >> >> If you have the RAM for it, you should be able to put tls_sessions.db on >> a tmpfs filesystem. This database is only a cache, so nothing valuable >> will be lost if the machine is rebooted; and as a cache, it benefits >> from being on the fastest storage you have available. :) > > Buuut, note that there's a bug in current releases of 2.5 where tls_prune will fail if the tls_sessions.db doesn't exist, preventing the server starting up. This will occur after ever reboot if you put this database on ephemeral storage! You can work around this by having your service init script touch the file before running master. When I understand you well, I could also remove the database and create an empty file before starting. As a work-arround. > The real fix for this is already in git, so it will be included in 2.5.12, which will hopefully be out this week! My problem is that I use the version in Debian, what is not good maintained at the moment. Cyrus-imap is removed from Debian testing last year. This means that when nobody cares, Cyrus will not be in the next Debian version. And also not in many other Linux distro's like Ubuntu. The freeze is in Januar/Februar. https://tracker.debian.org/news/859151/cyrus-imapd-removed-from-testing/ https://tracker.debian.org/pkg/cyrus-imapd https://release.debian.org/#release-dates You will say: use the upstream version. But sorry, I have to worry about many programs. My choice at the moment is to use software what's in Debian. I am using Cyrus imap about 17 years now, but it's possible I even have to switch to something else for this reason. Much thanks for your support! With regards, Paul > Cheers, > > ellie > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > -- Paul van der Vlis Linux systeembeheer Groningen https://www.vandervlis.nl/ From clarra at cc.upv.es Mon Oct 1 08:26:08 2018 From: clarra at cc.upv.es (=?UTF-8?Q?Carlos_Larra=c3=b1aga?=) Date: Mon, 1 Oct 2018 14:26:08 +0200 Subject: From 2.3.16 to 3.0.8 - lost expunged messages Message-ID: <7932368d-e596-dd33-2d8f-4d7ae4d18f86@cc.upv.es> Hello, We are now testing a per user based migration from cyrus-imapd 2.3.16 to 3.0.8 (both with delayed expunge) and noticed that we lose user's expunged messages when executing "reconstruct -r -G -V max user.foo". Is there any way to not lose the expunged messages? Regards and thanks in advance. Carlos Larra?aga Analista?de?Sistemas ASIC?-?UPV- SPAIN -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3627 bytes Desc: Firma criptogr??fica S/MIME URL: From paul at vandervlis.nl Mon Oct 1 18:56:06 2018 From: paul at vandervlis.nl (Paul van der Vlis) Date: Tue, 2 Oct 2018 00:56:06 +0200 Subject: Slow start of imap service In-Reply-To: References: <20180928153446.Horde.oxWiSjgdecguOsIQkrEiVi1@webmail.uni-tuebingen.de> <1538361123.2738140.1525959928.7D8240C3@webmail.messagingengine.com> <1538363604.2747884.1525991400.0ABFAD3C@webmail.messagingengine.com> Message-ID: Op 01-10-18 om 10:32 schreef Paul van der Vlis: > Op 01-10-18 om 05:13 schreef ellie timoney: >> On Mon, Oct 1, 2018, at 12:32 PM, ellie timoney wrote: >>> You could use the tls_sessions_db_path imapd.conf(5) option to put this >>> database onto faster storage? >>> >>>> tls_sessions_db_path: >>>> The absolute path to the TLS sessions db file. If not specified, will be >>>> configdirectory/tls_sessions.db >>> >>> If you have the RAM for it, you should be able to put tls_sessions.db on >>> a tmpfs filesystem. This database is only a cache, so nothing valuable >>> will be lost if the machine is rebooted; and as a cache, it benefits >>> from being on the fastest storage you have available. :) >> >> Buuut, note that there's a bug in current releases of 2.5 where tls_prune will fail if the tls_sessions.db doesn't exist, preventing the server starting up. This will occur after ever reboot if you put this database on ephemeral storage! You can work around this by having your service init script touch the file before running master. > > When I understand you well, I could also remove the database and create > an empty file before starting. As a work-arround. I do this now, and restarting takes now 2-3 minutes. So much better. But I will also investigatie for faster storage or tmpfs. With regards, Paul van der Vlis -- Paul van der Vlis Linux systeembeheer Groningen https://www.vandervlis.nl/ From lauffer at ph-freiburg.de Thu Oct 4 04:37:27 2018 From: lauffer at ph-freiburg.de (Stephan Lauffer) Date: Thu, 04 Oct 2018 10:37:27 +0200 Subject: building 3.0.8 fails with --disable-pcre on opensuse/sles Message-ID: <20181004103727.Horde.6JxT6miv56Oj56DKX-oAKX0@pmail.phfr.de> Hello! On a opensuse/sles cyrus-imapd-3.0.8 fails building if I use the configure switch --disable-pcre during compiling lib/glob.c I got errors about...: ... unknown type name 'regex_t' ... 'REG_EXTENDED' undeclared (first use in this function) ... unknown type name 'regmatch_t' ...and so on. If we do not use --disable-pcre cyrus-imapd is building but not useable, it is crashing very often. In the past we patched the pcre of suse to get this fixed but I would not like to patch the distro pcre package. With cyrus-imapd- 3.0.7 we could live with just disabling pcre in cyrus and everything was fine. -- Liebe Gruesse, with best regards Stephan Lauffer Pedagogical University Freiburg - Germany http://www.ph-freiburg.de/zik/ Fon/ Fax: +49 761 682 -559/ -486 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5962 bytes Desc: S/MIME-Signatur URL: From lauffer at ph-freiburg.de Thu Oct 4 05:00:33 2018 From: lauffer at ph-freiburg.de (Stephan Lauffer) Date: Thu, 04 Oct 2018 11:00:33 +0200 Subject: building 3.0.8 fails with --disable-pcre on opensuse/sles In-Reply-To: <20181004103727.Horde.6JxT6miv56Oj56DKX-oAKX0@pmail.phfr.de> Message-ID: <20181004110033.Horde.0Z-avP0saRTgb0fgEf3FlL2@pmail.phfr.de> ...in the past we patched pcre with this simple patch... https://sources.debian.org/patches/pcre3/2:8.39-11/pcreposix.patch/ ...and the issue on cyrus was that one: https://github.com/cyrusimap/cyrus-imapd/issues/1731 Zitat von Stephan Lauffer : > Hello! > > On a opensuse/sles cyrus-imapd-3.0.8 fails building if I use the > configure switch --disable-pcre during compiling lib/glob.c > I got errors about...: > > ... unknown type name 'regex_t' > ... 'REG_EXTENDED' undeclared (first use in this function) > ... unknown type name 'regmatch_t' > > ...and so on. > > If we do not use --disable-pcre cyrus-imapd is building but not > useable, it is crashing very often. In the past we patched the pcre > of suse to get this fixed but I would not like to patch the distro > pcre package. With cyrus-imapd- 3.0.7 we could live with just > disabling pcre in cyrus and everything was fine. > > > > > -- > Liebe Gruesse, with best regards > Stephan Lauffer > > Pedagogical University Freiburg - Germany > http://www.ph-freiburg.de/zik/ > Fon/ Fax: +49 761 682 -559/ -486 -- Liebe Gruesse, with best regards Stephan Lauffer Pedagogical University Freiburg - Germany http://www.ph-freiburg.de/zik/ Fon/ Fax: +49 761 682 -559/ -486 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5962 bytes Desc: S/MIME-Signatur URL: From ellie at fastmail.com Sun Oct 7 23:14:09 2018 From: ellie at fastmail.com (ellie timoney) Date: Mon, 08 Oct 2018 14:14:09 +1100 Subject: Cyrus IMAP 2.5.12 released Message-ID: <1538968449.2067917.1534051768.7D758987@webmail.messagingengine.com> The Cyrus team is proud to announce the immediate availability of a new version of Cyrus IMAP: 2.5.12 I'm trialling hosting the release files using Github's releases feature. Please use the Github download links if possible, and advise if you have any problems! (It may even download faster due to Github's content delivery network.) Download URLs: https://github.com/cyrusimap/cyrus-imapd/releases/download/cyrus-imapd-2.5.12/cyrus-imapd-2.5.12.tar.gz https://github.com/cyrusimap/cyrus-imapd/releases/download/cyrus-imapd-2.5.12/cyrus-imapd-2.5.12.tar.gz.sig https://www.cyrusimap.org/releases/cyrus-imapd-2.5.12.tar.gz https://www.cyrusimap.org/releases/cyrus-imapd-2.5.12.tar.gz.sig ftp://ftp.cyrusimap.org/cyrus-imapd/cyrus-imapd-2.5.12.tar.gz ftp://ftp.cyrusimap.org/cyrus-imapd/cyrus-imapd-2.5.12.tar.gz.sig Please consult the release notes before upgrading to 2.5.12: https://www.cyrusimap.org/imap/download/release-notes/2.5/x/2.5.12.html And join us on Github at https://github.com/cyrusimap/cyrus-imapd to report issues, join in the deliberations of new features for the next Cyrus IMAP release, and to contribute to the documentation. On behalf of the Cyrus team, Kind regards, ellie timoney From Albert.Shih at obspm.fr Mon Oct 8 04:36:10 2018 From: Albert.Shih at obspm.fr (Albert Shih) Date: Mon, 8 Oct 2018 10:36:10 +0200 Subject: Problem with sync Message-ID: <20181008083610.GA19743@io.chezmoi.fr> Hi everyone, I got two level of synchro: master --- sync --> imap-mirror-1 --- sync --> imap-mirror-2 The first level work fine, the second level (imap-mirror-1 --> imap-mirror-2) crash sometime ago. Now I try to restart the sync, and currently I'm not sure it's working, in the imap-mirror-1 I got [root at imap-mirror-1 /var/imap/sync/log]# ls -l total 133519 -rw------- 1 cyrus cyrus 16080624 Oct 8 10:27 log -rw------- 1 cyrus cyrus 256631941 Oct 3 16:39 log-run [root at imap-mirror-1 /var/imap/sync/log]# top When I restart the imap daemon on both mirror, I see a sync_client on the imap-mirror-1 and imapd on the imap-mirror-2 at 100% during sometime ( 4 days), now I still got the /var/imap/sync/log/log-run file and the /var/imap/sync/log/log who still growing up, but very few activity on those mirror. What should I do after long period of missing sync ? How can I re-sync everything ? Regards -- Albert SHIH DIO b?timent 15 Observatoire de Paris xmpp: jas at obspm.fr Heure local/Local time: Mon Oct 8 10:30:46 CEST 2018 From piessens at icsense.com Mon Oct 8 04:49:30 2018 From: piessens at icsense.com (Tim Piessens) Date: Mon, 8 Oct 2018 10:49:30 +0200 Subject: mac mail and murder configuration In-Reply-To: References: Message-ID: Hi all, we have a set-up with 3 backend servers and one frontend in a murder cluster. When I move a message to a folder on another backend than the one my inbox is residing on, I get the following error : > 1 messages could not be moved to the mailbox ?Shared Folders.RunningProjects.BLAH.Archive? > > An error occurred while moving messages to mailbox ??. Although the file has been moved In the Mail.app log files I see everything should be ok : > WROTE(encrypted) Oct 04 17:12:19.852 [kCFStreamSocketSecurityLevelNone] -- host:imap.icsense.biz -- port:143 -- socket:0x6000008a17a0 -- thread:0x60000266c280 > 6840.148 UID COPY 382909 "Shared Folders.RunningProjects.BLAH.Archive" > > READ(encrypted) Oct 04 17:12:19.870 [kCFStreamSocketSecurityLevelNone] -- host:imap.icsense.biz -- port:143 -- socket:0x6000008a17a0 -- thread:0x60000266c280 > 6840.148 OK [COPYUID 1403621845 1301] Completed > > WROTE(encrypted) Oct 04 17:12:20.004 [kCFStreamSocketSecurityLevelNone] -- host:imap.icsense.biz -- port:143 -- socket:0x6000008a17a0 -- thread:0x604005072580 > 6841.148 UID STORE 382909 +FLAGS.SILENT (\Deleted) > > READ(encrypted) Oct 04 17:12:20.010 [kCFStreamSocketSecurityLevelNone] -- host:imap.icsense.biz -- port:143 -- socket:0x6000008a17a0 -- thread:0x604005072580 > * 105 FETCH (FLAGS (\Deleted \Seen NotJunk $NotJunk) UID 382909 MODSEQ (447695)) > 6841.148 OK Completed When doing this on a folder which resides on the same backend as my inbox. There is no error. The log files on my Mac look the same though ( at first sight ) > WROTE(encrypted) Oct 04 17:17:54.411 [kCFStreamSocketSecurityLevelNone] -- host:imap.icsense.biz -- port:143 -- socket:0x6000008a17a0 -- thread:0x604003c6a380 > 6980.148 UID COPY 382594 IT > > READ(encrypted) Oct 04 17:17:54.439 [kCFStreamSocketSecurityLevelNone] -- host:imap.icsense.biz -- port:143 -- socket:0x6000008a17a0 -- thread:0x604003c6a380 > 6980.148 OK [COPYUID 1391422263 382594 65664] Completed > > WROTE(encrypted) Oct 04 17:17:54.457 [kCFStreamSocketSecurityLevelNone] -- host:imap.icsense.biz -- port:143 -- socket:0x6000008a17a0 -- thread:0x604003c6a380 > 6981.148 UID STORE 382594 +FLAGS.SILENT (\Deleted) > > READ(encrypted) Oct 04 17:17:54.470 [kCFStreamSocketSecurityLevelNone] -- host:imap.icsense.biz -- port:143 -- socket:0x6000008a17a0 -- thread:0x604003c6a380 > * 59 FETCH (FLAGS (\Answered \Deleted \Seen NotJunk $NotJunk) UID 382594 MODSEQ (447708)) > 6981.148 OK Completed What can be wrong ? I have the impression that this has to do with referrals. Does anyone experience the same issues ? Which settings should resolve this ? Thanks, Tim Dr. Ir. Tim Piessens CTO and Co-Founder Gaston Geenslaan 14, 3001 Leuven, Belgium Tel. +32 16 589 705 | Fax. +32 16 589 720 www.icsense.com |piessens at icsense.com "The information contained in this e-mail may be confidential." -------------- next part -------------- An HTML attachment was scrubbed... URL: From kvapss at gmail.com Tue Oct 9 06:57:48 2018 From: kvapss at gmail.com (kvaps) Date: Tue, 9 Oct 2018 12:57:48 +0200 Subject: Sieve scripts execution for connected folders Message-ID: Hello, I have shared folder which connected and subscribed in my mailbox, it is looks like normal subfolder., but my sieve scripts not handle mail inside it. I want to set seen flag for new messages inside this folder automatically by sieve script. Is it possible for configure sieve-script working for connected folders too? - kvaps From nic at onlight.com Tue Oct 9 07:07:59 2018 From: nic at onlight.com (Nic Bernstein) Date: Tue, 9 Oct 2018 12:07:59 +0100 Subject: Sieve scripts execution for connected folders In-Reply-To: References: Message-ID: <9c892899-bf24-d6c2-bf7f-e0e29a6c1c0c@onlight.com> Kvaps, It is unclear from your message just where this "shared folder" is rooted and where your sieve scripts are.? Do you mean a folder which is outside of the "user" name space?? If so, you cannot manage message delivery to this folder via user sieve scripts, but must use global sieve scripts instead. Please see the documentation here: https://www.cyrusimap.org/imap/reference/admin/sieve.html#sieve-scripts-in-shared-folders I hope this is helpful information, ??? -nic On 10/09/2018 11:57 AM, kvaps wrote: > Hello, > > I have shared folder which connected and subscribed in my mailbox, it > is looks like normal subfolder., but my sieve scripts not handle mail > inside it. > > I want to set seen flag for new messages inside this folder > automatically by sieve script. > Is it possible for configure sieve-script working for connected folders too? > > - kvaps > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Nic Bernstein nic at onlight.com Onlight Inc. www.onlight.com 6525 W Bluemound Rd., Ste 24 v. 414.272.4477 Milwaukee, Wisconsin 53213-4073 f. 414.290.0335 From kvapss at gmail.com Tue Oct 9 07:16:15 2018 From: kvapss at gmail.com (kvaps) Date: Tue, 9 Oct 2018 13:16:15 +0200 Subject: Sieve scripts execution for connected folders In-Reply-To: <9c892899-bf24-d6c2-bf7f-e0e29a6c1c0c@onlight.com> References: <9c892899-bf24-d6c2-bf7f-e0e29a6c1c0c@onlight.com> Message-ID: Hi, thanks for quick answer, Yes, I know about shared folder sieves, but I have aniother case: Eg I have some users: user/user1 at example.org user/user2 at example.org user/user3 at example.org and shared folder shared/test at example.org This folder connected to all users and have no shared seen state. I need to set seen flag automatically for user1, and keep second as unseen. - kvaps On Tue, Oct 9, 2018 at 1:08 PM Nic Bernstein wrote: > > Kvaps, > It is unclear from your message just where this "shared folder" is > rooted and where your sieve scripts are. Do you mean a folder which is > outside of the "user" name space? If so, you cannot manage message > delivery to this folder via user sieve scripts, but must use global > sieve scripts instead. > > Please see the documentation here: > https://www.cyrusimap.org/imap/reference/admin/sieve.html#sieve-scripts-in-shared-folders > > I hope this is helpful information, > -nic > > On 10/09/2018 11:57 AM, kvaps wrote: > > Hello, > > > > I have shared folder which connected and subscribed in my mailbox, it > > is looks like normal subfolder., but my sieve scripts not handle mail > > inside it. > > > > I want to set seen flag for new messages inside this folder > > automatically by sieve script. > > Is it possible for configure sieve-script working for connected folders too? > > > > - kvaps > > ---- > > Cyrus Home Page: http://www.cyrusimap.org/ > > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > > To Unsubscribe: > > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > -- > Nic Bernstein nic at onlight.com > Onlight Inc. www.onlight.com > 6525 W Bluemound Rd., Ste 24 v. 414.272.4477 > Milwaukee, Wisconsin 53213-4073 f. 414.290.0335 > From lists at nethead.se Tue Oct 9 10:27:00 2018 From: lists at nethead.se (Lists Nethead) Date: Tue, 09 Oct 2018 16:27:00 +0200 Subject: inaccessible mailbox for one user Message-ID: <20181009162700.Horde.5MCWYF8mPI_cSEpukH2_uK0@webmail.nethead.se> Hi, A user clicks on a mail in Horde IMP and next the inbox appears completely empty. This message files all still there of course, but the problem is that I cannot run any cyrus command on that mailbox, reconstruct, chk_cyrus or anything. Is there a lock possibly sitting somewhere that prohibits running any commands? Any other hints as to where I should start? Thanks! //per From nic at onlight.com Tue Oct 9 11:10:38 2018 From: nic at onlight.com (Nic Bernstein) Date: Tue, 9 Oct 2018 16:10:38 +0100 Subject: Sieve scripts execution for connected folders In-Reply-To: References: <9c892899-bf24-d6c2-bf7f-e0e29a6c1c0c@onlight.com> Message-ID: <03015d92-ac4a-a0c5-7ecf-dc40877f3f88@onlight.com> Your use of the phrase "connected to all users" and, earlier, "connected and subscribed", is confusing.? Am I correct to assume you mean the folder is "subscribed" by all users?? There is no such thing asa mailbox/folder being "connected" as a state.? It may be subscribed or not, it may be connected to, by a client, but "connected" is not a static condition. Cyrus can support shared seen state, but it's also not clear, from your messages, if you want shared seen state, or don't want it? Just what are you trying to achieve? Also, which version of Cyrus are you using?? Current is 3.0.8, historic is 2.5.12, development is 3.1.X. The rule with user sieve scripts is that they may only act on that user's mailboxes, not on shared mailboxes which one is subscribed to.? If you want to have new messages in a shared mailbox show as new for each user, separately, then you need to disable shared seen.? This may be done on the mailbox level, using the cyradm(8) command, with the mboxcfg subcommand: localhost>*info office* {office}: private: check: NIL checkperiod: NIL comment: NIL sort: NIL specialuse: NIL thread: NIL expire: NIL news2mail: NIL sieve: NIL squat: NIL shared: check: NIL checkperiod: NIL comment: NIL sort: NIL specialuse: NIL thread: NIL annotsize: 0 duplicatedeliver: false expire: NIL lastpop: NIL lastupdate: 31-Jul-2018 09:59:20 -0500 news2mail: NIL partition: default pop3newuidl: true pop3showafter: NIL *sharedseen: false* sieve: NIL size: 4101 squat: NIL synccrcs: 1118531081 0 uniqueid: 4c8f66f0-9f6a-4fa1-b8bf-547afa1ec5e8 localhost> mboxcfg office sharedseen true localhost>*info office* {office}: private: check: NIL checkperiod: NIL comment: NIL sort: NIL specialuse: NIL thread: NIL expire: NIL news2mail: NIL sieve: NIL squat: NIL shared: check: NIL checkperiod: NIL comment: NIL sort: NIL specialuse: NIL thread: NIL annotsize: 0 duplicatedeliver: false expire: NIL lastpop: NIL lastupdate: 9-Oct-2018 10:09:19 -0500 news2mail: NIL partition: default pop3newuidl: true pop3showafter: NIL *sharedseen: true* sieve: NIL size: 4101 squat: NIL synccrcs: 1118531081 0 uniqueid: 4c8f66f0-9f6a-4fa1-b8bf-547afa1ec5e8 Is that what you're trying to achieve, changing the setting of Shared \Seen? Cheers, ??? -nic On 10/09/2018 12:16 PM, kvaps wrote: > Hi, thanks for quick answer, > > Yes, I know about shared folder sieves, but I have aniother case: > > Eg I have some users: > > user/user1 at example.org > user/user2 at example.org > user/user3 at example.org > > and shared folder > > shared/test at example.org > > This folder connected to all users and have no shared seen state. > I need to set seen flag automatically for user1, and keep second as unseen. > > - kvaps > On Tue, Oct 9, 2018 at 1:08 PM Nic Bernstein wrote: >> Kvaps, >> It is unclear from your message just where this "shared folder" is >> rooted and where your sieve scripts are. Do you mean a folder which is >> outside of the "user" name space? If so, you cannot manage message >> delivery to this folder via user sieve scripts, but must use global >> sieve scripts instead. >> >> Please see the documentation here: >> https://www.cyrusimap.org/imap/reference/admin/sieve.html#sieve-scripts-in-shared-folders >> >> I hope this is helpful information, >> -nic >> >> On 10/09/2018 11:57 AM, kvaps wrote: >>> Hello, >>> >>> I have shared folder which connected and subscribed in my mailbox, it >>> is looks like normal subfolder., but my sieve scripts not handle mail >>> inside it. >>> >>> I want to set seen flag for new messages inside this folder >>> automatically by sieve script. >>> Is it possible for configure sieve-script working for connected folders too? >>> >>> - kvaps >>> ---- >>> Cyrus Home Page: http://www.cyrusimap.org/ >>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>> To Unsubscribe: >>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >> -- >> Nic Bernstein nic at onlight.com >> Onlight Inc. www.onlight.com >> 6525 W Bluemound Rd., Ste 24 v. 414.272.4477 >> Milwaukee, Wisconsin 53213-4073 f. 414.290.0335 >> -- Nic Bernstein nic at onlight.com Onlight Inc. www.onlight.com 6525 W Bluemound Rd., Ste 24 v. 414.272.4477 Milwaukee, Wisconsin 53213-4073 f. 414.290.0335 -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at nethead.se Tue Oct 9 11:38:13 2018 From: lists at nethead.se (Lists Nethead) Date: Tue, 09 Oct 2018 17:38:13 +0200 Subject: inaccessible mailbox for one user Message-ID: <20181009173813.Horde.-W2Y255S7qcWiXosCOmMwhG@webmail.nethead.se> > Hi, > > A user clicks on a mail in Horde IMP and next the inbox appears > completely empty. This message files all still there of course, but > the problem is that I cannot run any cyrus command on that mailbox, > reconstruct, chk_cyrus or anything. > > Is there a lock possibly sitting somewhere that prohibits running > any commands? Any other hints as to where I should start? > More: Cyrus is 3.0.8 I suspected that the problem of not being able to run any utilities was not only a Cyrus problem, su I restarted the virtual server (FreeBSD jail), then the locking was gone. Fine, so I ran the usual reconstruct. Now one can see the messages that arrived *after* the crash, but none of the ones arriving before (>6200 in inbox and plenty of subfolders). It is just one account, the others work ok. Where then is the problem? If reconstruct cannot fix it, what can? From kvapss at gmail.com Tue Oct 9 15:13:35 2018 From: kvapss at gmail.com (kvaps) Date: Tue, 9 Oct 2018 21:13:35 +0200 Subject: Sieve scripts execution for connected folders In-Reply-To: <03015d92-ac4a-a0c5-7ecf-dc40877f3f88@onlight.com> References: <9c892899-bf24-d6c2-bf7f-e0e29a6c1c0c@onlight.com> <03015d92-ac4a-a0c5-7ecf-dc40877f3f88@onlight.com> Message-ID: Sorry for confuse. I was little confused myself :) Yes, my user and few other users subscribed to this shared folder. Each user have own seen status for this folder - this is ok. But folder is big and have new mails quite often. Each user should react different mails in this mailbox. For now I receive notifications for all new mails, this is really annoying. I need notifications only for few of them, eg "mails with specific word in subject", the rest should be marked as seen. Usually this behavior can be configured via sieve script, but in case with shared folders it is not working. Global sieve script for shared folder will not work too, because it changes global seen state, but state for each user. Probably no way, as you said before. - kvaps On Tue, Oct 9, 2018 at 5:10 PM Nic Bernstein wrote: > > Your use of the phrase "connected to all users" and, earlier, "connected and subscribed", is confusing. Am I correct to assume you mean the folder is "subscribed" by all users? There is no such thing asa mailbox/folder being "connected" as a state. It may be subscribed or not, it may be connected to, by a client, but "connected" is not a static condition. > > Cyrus can support shared seen state, but it's also not clear, from your messages, if you want shared seen state, or don't want it? Just what are you trying to achieve? > > Also, which version of Cyrus are you using? Current is 3.0.8, historic is 2.5.12, development is 3.1.X. > > The rule with user sieve scripts is that they may only act on that user's mailboxes, not on shared mailboxes which one is subscribed to. If you want to have new messages in a shared mailbox show as new for each user, separately, then you need to disable shared seen. This may be done on the mailbox level, using the cyradm(8) command, with the mboxcfg subcommand: > > localhost> info office > {office}: > private: > check: NIL > checkperiod: NIL > comment: NIL > sort: NIL > specialuse: NIL > thread: NIL > expire: NIL > news2mail: NIL > sieve: NIL > squat: NIL > shared: > check: NIL > checkperiod: NIL > comment: NIL > sort: NIL > specialuse: NIL > thread: NIL > annotsize: 0 > duplicatedeliver: false > expire: NIL > lastpop: NIL > lastupdate: 31-Jul-2018 09:59:20 -0500 > news2mail: NIL > partition: default > pop3newuidl: true > pop3showafter: NIL > sharedseen: false > sieve: NIL > size: 4101 > squat: NIL > synccrcs: 1118531081 0 > uniqueid: 4c8f66f0-9f6a-4fa1-b8bf-547afa1ec5e8 > localhost> mboxcfg office sharedseen true > localhost> info office > {office}: > private: > check: NIL > checkperiod: NIL > comment: NIL > sort: NIL > specialuse: NIL > thread: NIL > expire: NIL > news2mail: NIL > sieve: NIL > squat: NIL > shared: > check: NIL > checkperiod: NIL > comment: NIL > sort: NIL > specialuse: NIL > thread: NIL > annotsize: 0 > duplicatedeliver: false > expire: NIL > lastpop: NIL > lastupdate: 9-Oct-2018 10:09:19 -0500 > news2mail: NIL > partition: default > pop3newuidl: true > pop3showafter: NIL > sharedseen: true > sieve: NIL > size: 4101 > squat: NIL > synccrcs: 1118531081 0 > uniqueid: 4c8f66f0-9f6a-4fa1-b8bf-547afa1ec5e8 > > Is that what you're trying to achieve, changing the setting of Shared \Seen? > > Cheers, > -nic > > On 10/09/2018 12:16 PM, kvaps wrote: > > Hi, thanks for quick answer, > > Yes, I know about shared folder sieves, but I have aniother case: > > Eg I have some users: > > user/user1 at example.org > user/user2 at example.org > user/user3 at example.org > > and shared folder > > shared/test at example.org > > This folder connected to all users and have no shared seen state. > I need to set seen flag automatically for user1, and keep second as unseen. > > - kvaps > On Tue, Oct 9, 2018 at 1:08 PM Nic Bernstein wrote: > > Kvaps, > It is unclear from your message just where this "shared folder" is > rooted and where your sieve scripts are. Do you mean a folder which is > outside of the "user" name space? If so, you cannot manage message > delivery to this folder via user sieve scripts, but must use global > sieve scripts instead. > > Please see the documentation here: > https://www.cyrusimap.org/imap/reference/admin/sieve.html#sieve-scripts-in-shared-folders > > I hope this is helpful information, > -nic > > On 10/09/2018 11:57 AM, kvaps wrote: > > Hello, > > I have shared folder which connected and subscribed in my mailbox, it > is looks like normal subfolder., but my sieve scripts not handle mail > inside it. > > I want to set seen flag for new messages inside this folder > automatically by sieve script. > Is it possible for configure sieve-script working for connected folders too? > > - kvaps > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > -- > Nic Bernstein nic at onlight.com > Onlight Inc. www.onlight.com > 6525 W Bluemound Rd., Ste 24 v. 414.272.4477 > Milwaukee, Wisconsin 53213-4073 f. 414.290.0335 > > > -- > Nic Bernstein nic at onlight.com > Onlight Inc. www.onlight.com > 6525 W Bluemound Rd., Ste 24 v. 414.272.4477 > Milwaukee, Wisconsin 53213-4073 f. 414.290.0335 From kvapss at gmail.com Tue Oct 9 15:17:45 2018 From: kvapss at gmail.com (kvaps) Date: Tue, 9 Oct 2018 21:17:45 +0200 Subject: Sieve scripts execution for connected folders In-Reply-To: References: <9c892899-bf24-d6c2-bf7f-e0e29a6c1c0c@onlight.com> <03015d92-ac4a-a0c5-7ecf-dc40877f3f88@onlight.com> Message-ID: Yes, my version is 2.5.11-10.11.el7.kolab_16, and seems there is this bug, so shared folder sieves not working there https://github.com/cyrusimap/cyrus-imapd/issues/1453 - kvaps On Tue, Oct 9, 2018 at 9:13 PM kvaps wrote: > > Sorry for confuse. I was little confused myself :) > > Yes, my user and few other users subscribed to this shared folder. > Each user have own seen status for this folder - this is ok. > But folder is big and have new mails quite often. > > Each user should react different mails in this mailbox. > For now I receive notifications for all new mails, this is really annoying. > I need notifications only for few of them, eg "mails with specific > word in subject", the rest should be marked as seen. > Usually this behavior can be configured via sieve script, but in case > with shared folders it is not working. > Global sieve script for shared folder will not work too, because it > changes global seen state, but state for each user. > > Probably no way, as you said before. > > - kvaps > On Tue, Oct 9, 2018 at 5:10 PM Nic Bernstein wrote: > > > > Your use of the phrase "connected to all users" and, earlier, "connected and subscribed", is confusing. Am I correct to assume you mean the folder is "subscribed" by all users? There is no such thing asa mailbox/folder being "connected" as a state. It may be subscribed or not, it may be connected to, by a client, but "connected" is not a static condition. > > > > Cyrus can support shared seen state, but it's also not clear, from your messages, if you want shared seen state, or don't want it? Just what are you trying to achieve? > > > > Also, which version of Cyrus are you using? Current is 3.0.8, historic is 2.5.12, development is 3.1.X. > > > > The rule with user sieve scripts is that they may only act on that user's mailboxes, not on shared mailboxes which one is subscribed to. If you want to have new messages in a shared mailbox show as new for each user, separately, then you need to disable shared seen. This may be done on the mailbox level, using the cyradm(8) command, with the mboxcfg subcommand: > > > > localhost> info office > > {office}: > > private: > > check: NIL > > checkperiod: NIL > > comment: NIL > > sort: NIL > > specialuse: NIL > > thread: NIL > > expire: NIL > > news2mail: NIL > > sieve: NIL > > squat: NIL > > shared: > > check: NIL > > checkperiod: NIL > > comment: NIL > > sort: NIL > > specialuse: NIL > > thread: NIL > > annotsize: 0 > > duplicatedeliver: false > > expire: NIL > > lastpop: NIL > > lastupdate: 31-Jul-2018 09:59:20 -0500 > > news2mail: NIL > > partition: default > > pop3newuidl: true > > pop3showafter: NIL > > sharedseen: false > > sieve: NIL > > size: 4101 > > squat: NIL > > synccrcs: 1118531081 0 > > uniqueid: 4c8f66f0-9f6a-4fa1-b8bf-547afa1ec5e8 > > localhost> mboxcfg office sharedseen true > > localhost> info office > > {office}: > > private: > > check: NIL > > checkperiod: NIL > > comment: NIL > > sort: NIL > > specialuse: NIL > > thread: NIL > > expire: NIL > > news2mail: NIL > > sieve: NIL > > squat: NIL > > shared: > > check: NIL > > checkperiod: NIL > > comment: NIL > > sort: NIL > > specialuse: NIL > > thread: NIL > > annotsize: 0 > > duplicatedeliver: false > > expire: NIL > > lastpop: NIL > > lastupdate: 9-Oct-2018 10:09:19 -0500 > > news2mail: NIL > > partition: default > > pop3newuidl: true > > pop3showafter: NIL > > sharedseen: true > > sieve: NIL > > size: 4101 > > squat: NIL > > synccrcs: 1118531081 0 > > uniqueid: 4c8f66f0-9f6a-4fa1-b8bf-547afa1ec5e8 > > > > Is that what you're trying to achieve, changing the setting of Shared \Seen? > > > > Cheers, > > -nic > > > > On 10/09/2018 12:16 PM, kvaps wrote: > > > > Hi, thanks for quick answer, > > > > Yes, I know about shared folder sieves, but I have aniother case: > > > > Eg I have some users: > > > > user/user1 at example.org > > user/user2 at example.org > > user/user3 at example.org > > > > and shared folder > > > > shared/test at example.org > > > > This folder connected to all users and have no shared seen state. > > I need to set seen flag automatically for user1, and keep second as unseen. > > > > - kvaps > > On Tue, Oct 9, 2018 at 1:08 PM Nic Bernstein wrote: > > > > Kvaps, > > It is unclear from your message just where this "shared folder" is > > rooted and where your sieve scripts are. Do you mean a folder which is > > outside of the "user" name space? If so, you cannot manage message > > delivery to this folder via user sieve scripts, but must use global > > sieve scripts instead. > > > > Please see the documentation here: > > https://www.cyrusimap.org/imap/reference/admin/sieve.html#sieve-scripts-in-shared-folders > > > > I hope this is helpful information, > > -nic > > > > On 10/09/2018 11:57 AM, kvaps wrote: > > > > Hello, > > > > I have shared folder which connected and subscribed in my mailbox, it > > is looks like normal subfolder., but my sieve scripts not handle mail > > inside it. > > > > I want to set seen flag for new messages inside this folder > > automatically by sieve script. > > Is it possible for configure sieve-script working for connected folders too? > > > > - kvaps > > ---- > > Cyrus Home Page: http://www.cyrusimap.org/ > > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > > To Unsubscribe: > > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > > > -- > > Nic Bernstein nic at onlight.com > > Onlight Inc. www.onlight.com > > 6525 W Bluemound Rd., Ste 24 v. 414.272.4477 > > Milwaukee, Wisconsin 53213-4073 f. 414.290.0335 > > > > > > -- > > Nic Bernstein nic at onlight.com > > Onlight Inc. www.onlight.com > > 6525 W Bluemound Rd., Ste 24 v. 414.272.4477 > > Milwaukee, Wisconsin 53213-4073 f. 414.290.0335 From lists at nethead.se Tue Oct 9 15:26:28 2018 From: lists at nethead.se (Lists Nethead) Date: Tue, 09 Oct 2018 21:26:28 +0200 Subject: inaccessible mailbox for one user In-Reply-To: <20181009173813.Horde.-W2Y255S7qcWiXosCOmMwhG@webmail.nethead.se> Message-ID: <20181009212628.Horde.UL3C1ffStIjNdAd0WOdr-VL@webmail.nethead.se> Quoting Lists Nethead : >> Hi, >> >> A user clicks on a mail in Horde IMP and next the inbox appears >> completely empty. This message files all still there of course, but >> the problem is that I cannot run any cyrus command on that mailbox, >> reconstruct, chk_cyrus or anything. >> >> Is there a lock possibly sitting somewhere that prohibits running >> any commands? Any other hints as to where I should start? >> Restored users seen and sub files from backup, ran reconstruct again and then I had to run unexpunge. Now user files are back but I see the following error for this user in the logs IOERROR: conversations_audit on store Should I try sbin/ctl_conversationsdb -b ? From Albert.Shih at obspm.fr Tue Oct 9 16:11:01 2018 From: Albert.Shih at obspm.fr (Albert Shih) Date: Tue, 9 Oct 2018 22:11:01 +0200 Subject: inaccessible mailbox for one user In-Reply-To: <20181009212628.Horde.UL3C1ffStIjNdAd0WOdr-VL@webmail.nethead.se> References: <20181009173813.Horde.-W2Y255S7qcWiXosCOmMwhG@webmail.nethead.se> <20181009212628.Horde.UL3C1ffStIjNdAd0WOdr-VL@webmail.nethead.se> Message-ID: <20181009201101.GA1673@io.chezmoi.fr> Le 09/10/2018 ? 21:26:28+0200, Lists Nethead a ?crit > > Quoting Lists Nethead : > > > > Hi, > > > > > > A user clicks on a mail in Horde IMP and next the inbox appears > > > completely empty. This message files all still there of course, but > > > the problem is that I cannot run any cyrus command on that mailbox, > > > reconstruct, chk_cyrus or anything. > > > > > > Is there a lock possibly sitting somewhere that prohibits running > > > any commands? Any other hints as to where I should start? > > > > > Restored users seen and sub files from backup, ran reconstruct again and > then I had to run unexpunge. Now user files are back but I see the following > error for this user in the logs > > IOERROR: conversations_audit on store > > Should I try sbin/ctl_conversationsdb -b ? I'm not a cyrus guru but, are you sure your FS are clean ? I'm running FreeBSD and long time ago I got a FS error without crash. It was a standard server (apache) and I got very strange error, like unable to remove file. I eventually restart the server and run a fsck -f manually to fix de problem. I never find out where and how I got that problem The FS was UFS2. I'm absolutly not sure it's the problem, just try to help. Regards. -- Albert SHIH DIO b?timent 15 Observatoire de Paris xmpp: jas at obspm.fr Heure local/Local time: Tue Oct 9 22:07:55 CEST 2018 From lists at nethead.se Wed Oct 10 01:41:52 2018 From: lists at nethead.se (Lists Nethead) Date: Wed, 10 Oct 2018 07:41:52 +0200 Subject: inaccessible mailbox for one user In-Reply-To: <20181009201101.GA1673@io.chezmoi.fr> References: <20181009173813.Horde.-W2Y255S7qcWiXosCOmMwhG@webmail.nethead.se> <20181009212628.Horde.UL3C1ffStIjNdAd0WOdr-VL@webmail.nethead.se> <20181009201101.GA1673@io.chezmoi.fr> Message-ID: <20181010074152.Horde.BcWE1kJPfdnEb-WgPMkyfMI@webmail.nethead.se> Quoting Albert Shih : > Le 09/10/2018 ? 21:26:28+0200, Lists Nethead a ?crit >> >> Quoting Lists Nethead : >> >> > > Hi, >> > > >> > > A user clicks on a mail in Horde IMP and next the inbox appears >> > > completely empty. This message files all still there of course, but >> > > the problem is that I cannot run any cyrus command on that mailbox, >> > > reconstruct, chk_cyrus or anything. >> > > >> > > Is there a lock possibly sitting somewhere that prohibits running >> > > any commands? Any other hints as to where I should start? >> > > >> >> Restored users seen and sub files from backup, ran reconstruct again and >> then I had to run unexpunge. Now user files are back but I see the following >> error for this user in the logs >> >> IOERROR: conversations_audit on store >> >> Should I try sbin/ctl_conversationsdb -b ? > > I'm not a cyrus guru but, are you sure your FS are clean ? I'm running > FreeBSD and long time ago I got a FS error without crash. It was a standard > server (apache) and I got very strange error, like unable to remove file. > I eventually restart the server and run a fsck -f manually to fix de > problem. I never find out where and how I got that problem > > The FS was UFS2. > > I'm absolutly not sure it's the problem, just try to help. Thank you for your response. Yes, I think the initial fault was not a Cyrus issue and the locking problem vanished after the jail was restarted. Mail works ok now except the error above. We run zfs so fsck is not of much help, but none of the pools report a problem. From pim at zandbergen.org Thu Oct 11 13:13:42 2018 From: pim at zandbergen.org (Pim Zandbergen) Date: Thu, 11 Oct 2018 19:13:42 +0200 Subject: different sasl_mech_lists for imap and http? Message-ID: <5067c46b-afc5-d2b1-6505-1020b8a799f9@zandbergen.org> Is there a simple way to configure a different sasl_mech_list for imap and http? I was allowing login, plain, cram-md5 and digest-md5 using sasldb, when I noticed digest-md5 is not working at all for http. But it works fine for imap. I started noticing this when it turned out iOS CalDAV/CardDAV clients could not authenticate. iOS, once it sees digest-md5 offered, apparently will not fall back to other mechs. Other *dav clients appear to be less picky, especially over SSL. But other *dav clients will start to fail too, if digest-md5 is the only mech available. Since I can't figure out what's wrong with my SASL setup, I'd like to disable digest-md5 for http, but leave it enabled for imap where it works fine. I'm using cyrus imap 3.0.8 on Fedora. From murch at fastmail.com Thu Oct 11 13:28:39 2018 From: murch at fastmail.com (Ken Murchison) Date: Thu, 11 Oct 2018 13:28:39 -0400 Subject: different sasl_mech_lists for imap and http? In-Reply-To: <5067c46b-afc5-d2b1-6505-1020b8a799f9@zandbergen.org> References: <5067c46b-afc5-d2b1-6505-1020b8a799f9@zandbergen.org> Message-ID: <650009cc-cb5b-ec24-26fc-17065f9f0eef@fastmail.com> Yes.? Each option in imapd.conf can be prefixed with a service name from cyrus.conf. So if your services are named 'imap' and 'http', use: imap_sasl_mech_list http_sasl_mech_list Definitely don't rely on Digest for http.? Most implementations are either broken or incompatible.? I should have never attempted to make DIGEST-MD5 work for http. On 10/11/18 1:13 PM, Pim Zandbergen wrote: > Is there a simple way to configure a different sasl_mech_list for imap > and http? > > I was allowing login, plain, cram-md5 and digest-md5 using sasldb, > when I noticed digest-md5 is not working at all for http. > But it works fine for imap. > > I started noticing this when it turned out iOS CalDAV/CardDAV clients > could not authenticate. > iOS, once it sees digest-md5 offered, apparently will not fall back to > other mechs. > Other *dav clients appear to be less picky, especially over SSL. > But other *dav clients will start to fail too, if digest-md5 is the > only mech available. > > Since I can't figure out what's wrong with my SASL setup, I'd like to > disable digest-md5 for http, > but leave it enabled for imap where it works fine. > > I'm using cyrus imap 3.0.8 on Fedora. > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Ken Murchison Cyrus Development Team FastMail US LLC -------------- next part -------------- A non-text attachment was scrubbed... Name: murch.vcf Type: text/x-vcard Size: 4 bytes Desc: not available URL: From pim at zandbergen.org Thu Oct 11 13:50:46 2018 From: pim at zandbergen.org (Pim Zandbergen) Date: Thu, 11 Oct 2018 19:50:46 +0200 Subject: different sasl_mech_lists for imap and http? In-Reply-To: <650009cc-cb5b-ec24-26fc-17065f9f0eef@fastmail.com> References: <5067c46b-afc5-d2b1-6505-1020b8a799f9@zandbergen.org> <650009cc-cb5b-ec24-26fc-17065f9f0eef@fastmail.com> Message-ID: <47a67b1b-2417-ff3f-82e5-157a8bf11c6f@zandbergen.org> Right, I tried that, but used the wrong prefix ("dav" instead of? "http"). So I would probably need to define imaps_sasl_mech_list and https_sasl_mech_list too. Can I save one line by mixing prefixed? and non-prefixed options as a default? Like sasl_mech_list: login plain cram-md5 digest-md5 http_sasl_mech_list: login plain cram-md5 https_sasl_mech_list: login plain cram-md5 Thanks, also for not to have to worry about digest-md5 not working. On 11-10-2018 19:28, Ken Murchison wrote: > Yes. Each option in imapd.conf can be prefixed with a service name > from cyrus.conf. > > So if your services are named 'imap' and 'http', use: > > imap_sasl_mech_list > > http_sasl_mech_list > > > Definitely don't rely on Digest for http.? Most implementations are > either broken or incompatible.? I should have never attempted to make > DIGEST-MD5 work for http. > > > On 10/11/18 1:13 PM, Pim Zandbergen wrote: >> Is there a simple way to configure a different sasl_mech_list for >> imap and http? >> >> I was allowing login, plain, cram-md5 and digest-md5 using sasldb, >> when I noticed digest-md5 is not working at all for http. >> But it works fine for imap. >> >> I started noticing this when it turned out iOS CalDAV/CardDAV clients >> could not authenticate. >> iOS, once it sees digest-md5 offered, apparently will not fall back >> to other mechs. >> Other *dav clients appear to be less picky, especially over SSL. >> But other *dav clients will start to fail too, if digest-md5 is the >> only mech available. >> >> Since I can't figure out what's wrong with my SASL setup, I'd like to >> disable digest-md5 for http, >> but leave it enabled for imap where it works fine. >> >> I'm using cyrus imap 3.0.8 on Fedora. >> ---- >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > > > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus From Albert.Shih at obspm.fr Fri Oct 12 12:24:17 2018 From: Albert.Shih at obspm.fr (Albert Shih) Date: Fri, 12 Oct 2018 18:24:17 +0200 Subject: inaccessible mailbox for one user In-Reply-To: <20181010074152.Horde.BcWE1kJPfdnEb-WgPMkyfMI@webmail.nethead.se> References: <20181009173813.Horde.-W2Y255S7qcWiXosCOmMwhG@webmail.nethead.se> <20181009212628.Horde.UL3C1ffStIjNdAd0WOdr-VL@webmail.nethead.se> <20181009201101.GA1673@io.chezmoi.fr> <20181010074152.Horde.BcWE1kJPfdnEb-WgPMkyfMI@webmail.nethead.se> Message-ID: <20181012162417.GK996@chezmoi.fr> Le 10/10/2018 ? 07:41:52+0200, Lists Nethead a ?crit > > Quoting Albert Shih : > > > Le 09/10/2018 ? 21:26:28+0200, Lists Nethead a ?crit > >> > >> Quoting Lists Nethead : > >> > >> > > Hi, > >> > > > >> > > A user clicks on a mail in Horde IMP and next the inbox appears > >> > > completely empty. This message files all still there of course, but > >> > > the problem is that I cannot run any cyrus command on that mailbox, > >> > > reconstruct, chk_cyrus or anything. > >> > > > >> > > Is there a lock possibly sitting somewhere that prohibits running > >> > > any commands? Any other hints as to where I should start? > >> > > > >> > >> Restored users seen and sub files from backup, ran reconstruct again and > >> then I had to run unexpunge. Now user files are back but I see the following > >> error for this user in the logs > >> > >> IOERROR: conversations_audit on store > >> > >> Should I try sbin/ctl_conversationsdb -b ? > > > > I'm not a cyrus guru but, are you sure your FS are clean ? I'm running > > FreeBSD and long time ago I got a FS error without crash. It was a standard > > server (apache) and I got very strange error, like unable to remove file. > > I eventually restart the server and run a fsck -f manually to fix de > > problem. I never find out where and how I got that problem > > > > The FS was UFS2. > > > > I'm absolutly not sure it's the problem, just try to help. > > Thank you for your response. You'r welcome. And seem...my answer was not very usefull ;-) > Yes, I think the initial fault was not a Cyrus issue and the locking > problem vanished after the jail was restarted. Mail works ok now > except the error above. Did you find a solution ? > We run zfs so fsck is not of much help, but none of the pools report a > problem. We too, run FreeBSD + jail + ZFS (two, one for the mailbox on mechanical disk and another on ssd). Regards. -- Albert SHIH Observatoire de Paris xmpp: jas at obspm.fr Heure local/Local time: Fri Oct 12 18:22:32 CEST 2018 From lists at nethead.se Sat Oct 13 07:35:09 2018 From: lists at nethead.se (Lists Nethead) Date: Sat, 13 Oct 2018 13:35:09 +0200 Subject: inaccessible mailbox for one user In-Reply-To: <20181012162417.GK996@chezmoi.fr> References: <20181009173813.Horde.-W2Y255S7qcWiXosCOmMwhG@webmail.nethead.se> <20181009212628.Horde.UL3C1ffStIjNdAd0WOdr-VL@webmail.nethead.se> <20181009201101.GA1673@io.chezmoi.fr> <20181010074152.Horde.BcWE1kJPfdnEb-WgPMkyfMI@webmail.nethead.se> <20181012162417.GK996@chezmoi.fr> Message-ID: <20181013133509.Horde.Y7sam-kF_QGDon5RWxPg9CI@webmail.nethead.se> Quoting Albert Shih : > Le 10/10/2018 ? 07:41:52+0200, Lists Nethead a ?crit >> >> Quoting Albert Shih : >> >> > Le 09/10/2018 ? 21:26:28+0200, Lists Nethead a ?crit >> >> >> >> Quoting Lists Nethead : >> >> >> >> > > Hi, >> >> > > >> >> > > A user clicks on a mail in Horde IMP and next the inbox appears >> >> > > completely empty. This message files all still there of course, but >> >> > > the problem is that I cannot run any cyrus command on that mailbox, >> >> > > reconstruct, chk_cyrus or anything. >> >> > > >> >> > > Is there a lock possibly sitting somewhere that prohibits running >> >> > > any commands? Any other hints as to where I should start? >> >> > > >> >> >> >> Restored users seen and sub files from backup, ran reconstruct again and >> >> then I had to run unexpunge. Now user files are back but I see >> the following >> >> error for this user in the logs >> >> >> >> IOERROR: conversations_audit on store >> >> >> >> Should I try sbin/ctl_conversationsdb -b ? >> > >> > I'm not a cyrus guru but, are you sure your FS are clean ? I'm running >> > FreeBSD and long time ago I got a FS error without crash. It was >> a standard >> > server (apache) and I got very strange error, like unable to remove file. >> > I eventually restart the server and run a fsck -f manually to fix de >> > problem. I never find out where and how I got that problem >> > >> > The FS was UFS2. >> > >> > I'm absolutly not sure it's the problem, just try to help. >> >> Thank you for your response. > > You'r welcome. And seem...my answer was not very usefull ;-) > >> Yes, I think the initial fault was not a Cyrus issue and the locking >> problem vanished after the jail was restarted. Mail works ok now >> except the error above. > > Did you find a solution ? > >> We run zfs so fsck is not of much help, but none of the pools report a >> problem. > > We too, run FreeBSD + jail + ZFS (two, one for the mailbox on mechanical > disk and another on ssd). > No solution found so far, still get "IOERROR: conversations_audit on store" on this account, tried all switches for ctl_conversationsdb. Currently out of ideas. From brong at fastmailteam.com Sun Oct 14 22:23:42 2018 From: brong at fastmailteam.com (Bron Gondwana) Date: Sun, 14 Oct 2018 22:23:42 -0400 Subject: inaccessible mailbox for one user In-Reply-To: <20181013133509.Horde.Y7sam-kF_QGDon5RWxPg9CI@webmail.nethead.se> References: <20181009173813.Horde.-W2Y255S7qcWiXosCOmMwhG@webmail.nethead.se> <20181009212628.Horde.UL3C1ffStIjNdAd0WOdr-VL@webmail.nethead.se> <20181009201101.GA1673@io.chezmoi.fr> <20181010074152.Horde.BcWE1kJPfdnEb-WgPMkyfMI@webmail.nethead.se> <20181012162417.GK996@chezmoi.fr> <20181013133509.Horde.Y7sam-kF_QGDon5RWxPg9CI@webmail.nethead.se> Message-ID: <32ce55ac-bc17-467b-9e45-d88112ae3ceb@sloti22d1t06> "ctl_conversationsdb -R -r" should rebuild all conversations databases for all users. Bron. On Sat, Oct 13, 2018, at 22:35, Lists Nethead wrote: > > Quoting Albert Shih : > > > Le 10/10/2018 ? 07:41:52+0200, Lists Nethead a ?crit > >> > >> Quoting Albert Shih : > >> > >> > Le 09/10/2018 ? 21:26:28+0200, Lists Nethead a ?crit > >> >> > >> >> Quoting Lists Nethead : > >> >> > >> >> > > Hi, > >> >> > > > >> >> > > A user clicks on a mail in Horde IMP and next the inbox appears > >> >> > > completely empty. This message files all still there of course, but > >> >> > > the problem is that I cannot run any cyrus command on that mailbox, > >> >> > > reconstruct, chk_cyrus or anything. > >> >> > > > >> >> > > Is there a lock possibly sitting somewhere that prohibits running > >> >> > > any commands? Any other hints as to where I should start? > >> >> > > > >> >> > >> >> Restored users seen and sub files from backup, ran reconstruct again and > >> >> then I had to run unexpunge. Now user files are back but I see > >> the following > >> >> error for this user in the logs > >> >> > >> >> IOERROR: conversations_audit on store > >> >> > >> >> Should I try sbin/ctl_conversationsdb -b ? > >> > > >> > I'm not a cyrus guru but, are you sure your FS are clean ? I'm running > >> > FreeBSD and long time ago I got a FS error without crash. It was > >> a standard > >> > server (apache) and I got very strange error, like unable to remove file. > >> > I eventually restart the server and run a fsck -f manually to fix de > >> > problem. I never find out where and how I got that problem > >> > > >> > The FS was UFS2. > >> > > >> > I'm absolutly not sure it's the problem, just try to help. > >> > >> Thank you for your response. > > > > You'r welcome. And seem...my answer was not very usefull ;-) > > > >> Yes, I think the initial fault was not a Cyrus issue and the locking > >> problem vanished after the jail was restarted. Mail works ok now > >> except the error above. > > > > Did you find a solution ? > > > >> We run zfs so fsck is not of much help, but none of the pools report a > >> problem. > > > > We too, run FreeBSD + jail + ZFS (two, one for the mailbox on mechanical > > disk and another on ssd). > > > > No solution found so far, still get "IOERROR: conversations_audit on > store" on this account, tried all switches for ctl_conversationsdb. > Currently out of ideas. > > > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Bron Gondwana, CEO, FastMail Pty Ltd brong at fastmailteam.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From Albert.Shih at obspm.fr Mon Oct 15 07:15:28 2018 From: Albert.Shih at obspm.fr (Albert Shih) Date: Mon, 15 Oct 2018 13:15:28 +0200 Subject: Problem with sync In-Reply-To: <20181008083610.GA19743@io.chezmoi.fr> References: <20181008083610.GA19743@io.chezmoi.fr> Message-ID: <20181015111528.GB1678@io.chezmoi.fr> Le 08/10/2018 ? 10:36:10+0200, Albert Shih a ?crit Hi everyone, > I still got my problem : > I got two level of synchro: > > master --- sync --> imap-mirror-1 --- sync --> imap-mirror-2 > > The first level work fine, the second level (imap-mirror-1 --> imap-mirror-2) > crash sometime ago. > > Now I try to restart the sync, and currently I'm not sure it's working, in > the imap-mirror-1 I got > > [root at imap-mirror-1 /var/imap/sync/log]# ls -l > total 133519 > -rw------- 1 cyrus cyrus 16080624 Oct 8 10:27 log > -rw------- 1 cyrus cyrus 256631941 Oct 3 16:39 log-run > [root at imap-mirror-1 /var/imap/sync/log]# top > > When I restart the imap daemon on both mirror, I see a sync_client on the > imap-mirror-1 and imapd on the imap-mirror-2 at 100% during sometime ( 4 > days), now I still got the > > /var/imap/sync/log/log-run > > file and the > > /var/imap/sync/log/log > > who still growing up, but very few activity on those mirror. But it seem the sync working, but it's very slow and stop very often. So how can I speedup me sync, knowning at this speed I will never end the sync because the /var/imap/sync/log/log still growing up. So if it wait the end of /var/imap/sync/log/log-run I don't see how this going to happen. So is they are any way to synchronise what's in the /var/imap/sync/log/log-run manually without waiting the imap daemon to launch the sync. Other question, at this time (during the sync of log-run) can I restart the imap daemon without breaking everything ? Regards. -- Albert SHIH DIO b?timent 15 Observatoire de Paris xmpp: jas at obspm.fr Heure local/Local time: Mon Oct 15 13:12:05 CEST 2018 From kawazu428 at gmail.com Tue Oct 23 05:58:03 2018 From: kawazu428 at gmail.com (Kristian Rink) Date: Tue, 23 Oct 2018 11:58:03 +0200 Subject: Debugging A5 BAD Unexpected extra arguments to Search ? Message-ID: <501fa202-51a8-7367-b815-85989d7e2291@gmail.com> Hi all; we're running a Java (and Java Mail) based application interacting with our cyrus imapd server. Trying to use complex search terms (searching for multiple criterias, such as message-id and message date), we regularly (all the time?) end up with messages such as A5 BAD Unexpected extra arguments to Search or A7 BAD Unexpected extra arguments to Search Questions here: What's the difference between A5 and A7? Any way to debug what exactly is wrong about our search terms here? TIA and all the best, Kristian From ellie at fastmail.com Tue Oct 23 21:27:06 2018 From: ellie at fastmail.com (ellie timoney) Date: Wed, 24 Oct 2018 12:27:06 +1100 Subject: Debugging A5 BAD Unexpected extra arguments to Search ? In-Reply-To: <501fa202-51a8-7367-b815-85989d7e2291@gmail.com> References: <501fa202-51a8-7367-b815-85989d7e2291@gmail.com> Message-ID: <1540344426.2462107.1552457368.27272452@webmail.messagingengine.com> Hi Kristian, The "A5" and "A7" are just the "tags" associated with the client's search commands, they have no semantic value. The client prefixes each command with some tag, and then the server uses the same tag in the response (so that you can identify which response applies to which command). In these two cases, your client has chosen "A5" and "A7" as tags. Tags are described in detail here: https://tools.ietf.org/html/rfc3501#section-2.2.1 Your Java/Java Mail application hopefully has a feature to log the commands it's sending. If you turn this on, you'll be able to see which bad search commands were resulting in the error (by matching response tags and command tags), and then compare the command syntax to the expected syntax from the relevant RFCs. Cheers, ellie On Tue, Oct 23, 2018, at 8:58 PM, Kristian Rink wrote: > Hi all; > > we're running a Java (and Java Mail) based application interacting with > our cyrus imapd server. Trying to use complex search terms (searching > for multiple criterias, such as message-id and message date), we > regularly (all the time?) end up with messages such as > > A5 BAD Unexpected extra arguments to Search or > A7 BAD Unexpected extra arguments to Search > > Questions here: What's the difference between A5 and A7? Any way to > debug what exactly is wrong about our search terms here? > > TIA and all the best, > Kristian From kawazu428 at gmail.com Wed Oct 24 09:31:47 2018 From: kawazu428 at gmail.com (Kristian Rink) Date: Wed, 24 Oct 2018 15:31:47 +0200 Subject: Debugging A5 BAD Unexpected extra arguments to Search ? In-Reply-To: <1540344426.2462107.1552457368.27272452@webmail.messagingengine.com> References: <501fa202-51a8-7367-b815-85989d7e2291@gmail.com> <1540344426.2462107.1552457368.27272452@webmail.messagingengine.com> Message-ID: <673db8f0-90e1-95f2-a9ec-7ecf9b3f77c8@gmail.com> Hi Ellie; ok, thanks for explaining. So I'll dive into the Java application and see whether debugging the search terms gets me any further. Cheers, Kristian Am 24.10.18 um 03:27 schrieb ellie timoney: > > The "A5" and "A7" are just the "tags" associated with the client's > search commands, they have no semantic value. The client prefixes > each command with some tag, and then the server uses the same tag in > the response (so that you can identify which response applies to > which command). In these two cases, your client has chosen "A5" and > "A7" as tags. Tags are described in detail here: > https://tools.ietf.org/html/rfc3501#section-2.2.1 > From kawazu428 at gmail.com Wed Oct 24 09:34:24 2018 From: kawazu428 at gmail.com (Kristian Rink) Date: Wed, 24 Oct 2018 15:34:24 +0200 Subject: Transfer folder subscriptions Message-ID: <3120e6d3-1775-79db-07c5-321e4f2dec40@gmail.com> Folks; I'm in the process of moving mails between two cyrus installations in order to at some point switch over all users. So far using imapsync and doing so as cyrus user met most of my needs, however I so far fail to transfer each users folder subscriptions. Given user names and folder hierarchy are exactly the same on both systems, is there any way to transfer the list of folders each user has subscribed without resetting all their passwords and running a tool such as imapsync on a per-user level? Thanks in advance and all the best, Kristian From steffo76 at gmx.de Wed Oct 24 15:38:53 2018 From: steffo76 at gmx.de (Stephan) Date: Wed, 24 Oct 2018 21:38:53 +0200 Subject: Debugging A5 BAD Unexpected extra arguments to Search ? In-Reply-To: <501fa202-51a8-7367-b815-85989d7e2291@gmail.com> References: <501fa202-51a8-7367-b815-85989d7e2291@gmail.com> Message-ID: <7e6c9760-a079-8203-8562-c4c457740399@gmx.de> Am 23.10.2018 um 11:58 schrieb Kristian Rink: > Hi all; > > we're running a Java (and Java Mail) based application interacting with > our cyrus imapd server. Trying to use complex search terms (searching > for multiple criterias, such as message-id and message date), we > regularly (all the time?) end up with messages such as > > A5 BAD Unexpected extra arguments to Search??? or > A7 BAD Unexpected extra arguments to Search > > Questions here: What's the difference between A5 and A7? Any way to > debug what exactly is wrong about our search terms here? > > TIA and all the best, > Kristian > ---- Couldn't this be debugged by enabling telemetry logging ? Or is this feature missing from the newer releases ? https://www.cyrusimap.org/2.5/imap/faqs/o-telemetry.html From kawazu428 at gmail.com Thu Oct 25 02:31:50 2018 From: kawazu428 at gmail.com (Kristian Rink) Date: Thu, 25 Oct 2018 08:31:50 +0200 Subject: Debugging A5 BAD Unexpected extra arguments to Search ? In-Reply-To: <7e6c9760-a079-8203-8562-c4c457740399@gmx.de> References: <501fa202-51a8-7367-b815-85989d7e2291@gmail.com> <7e6c9760-a079-8203-8562-c4c457740399@gmx.de> Message-ID: Am 24.10.18 um 21:38 schrieb Stephan: > > Couldn't this be debugged by enabling telemetry logging ? Or is this > feature missing from the newer releases ? > > https://www.cyrusimap.org/2.5/imap/faqs/o-telemetry.html I will have a look at this, thanks. :) I'm currently moving from cyrus 2.2 (old Ubuntu 10.04) to a "newer" (cyrus 2.4) version on Ubuntu 16.04. Maybe telemetry still works in these versions... :) Cheers, Kristian From matsl at irf.se Thu Oct 25 07:01:55 2018 From: matsl at irf.se (Mats Luspa) Date: Thu, 25 Oct 2018 13:01:55 +0200 Subject: timsieved does not recognize CHECKSCRIPT Message-ID: Hello! It seems that the version "Cyrus timsieved git2.4.17+0-Debian-2.4.17+nocaldav-0+deb8u2" does not recognize the CHECKSCRIPT command. I encountered it when I installed a software and SIEVE scripts remotely didn't work. The followong protocol was followed: telnet mail 4190 Trying 2001:6b0:27:ff::c... Connected to mail.irf.se. Escape character is '^]'. "IMPLEMENTATION" "Cyrus timsieved git2.4.17+0-Debian-2.4.17+nocaldav-0+deb8u2" "SASL" "PLAIN" "SIEVE" "comparator-i;ascii-numeric fileinto reject vacation imapflags notify include env" "STARTTLS" "UNAUTHENTICATE" OK AUTHENTICATE "PLAIN" {24+} AG1hdHNsAHZsJSZBbTQ1Ng== OK LISTSCRIPTS "ingo" "rainloop.user" "default" OK CHECKSCRIPT {266+} NO "Expected a command. Got something else." Can this be a bug? /Regards Mats -- Mats Luspa Phone: +46 (0)980 79 022 Cellular phone: +46 (0)725813330 Institutet f?r rymdfysik?????????????? Fax: +46 (0)980 79 050 Swedish Institute of Space Physics????? email: matsl at irf.se Visiting/Delivery address: Rymdcampus 1, SE-981 92 Kiruna Postal address: Box 812, SE-981 28 Kiruna -- PGP Public Key: https://www.irf.se/pgp/matsl??????? From michael.menge at zdv.uni-tuebingen.de Thu Oct 25 09:09:40 2018 From: michael.menge at zdv.uni-tuebingen.de (Michael Menge) Date: Thu, 25 Oct 2018 15:09:40 +0200 Subject: squatter not used after upgrade to cyrus 3.0.8 In-Reply-To: <20180917171814.Horde.d0cTBJKuZdVAj2dx5sWRBef@webmail.uni-tuebingen.de> References: <20180917140152.Horde.W8VK46keZMD6i3ABlUUhwzX@webmail.uni-tuebingen.de> <20180917123123.GA24774@io.chezmoi.fr> <20180917171814.Horde.d0cTBJKuZdVAj2dx5sWRBef@webmail.uni-tuebingen.de> Message-ID: <20181025150940.Horde.ZSBGZc-bdo4Ww0nHKDroxYH@webmail.uni-tuebingen.de> Hi, Quoting Michael Menge : > Hi, > > Quoting Albert Shih : > >> Le 17/09/2018 ? 14:01:52+0200, Michael Menge a ?crit >>> Hi, >>> >>> we recently upgrade from Cyrus-Imapd 2.4.x to 3.0.8. After some initial >>> problems >>> which we could fix cyrus imapd 3.0.8 is running stable. The one remaining >>> problem >>> we receive reports about is, that the search is not working/too slow. >>> >>> First we discovered that one of the options for Squatter are not supported >>> anymore, "-s Skip mailboxes whose index file is older than their current >>> squat file (within a small time delta)." and that squatter does not like >>> "-r" in combination with the source "user" >>> >>> > squatter -C /etc/imapd_be.conf -r user >>> fatal error: Internal error: assertion failed: lib/cyrusdb_twoskip.c: >>> 2339: keylen >>> >>> >>> But after reindexing all mailboxes the search is still slow. I tried to >>> debug this and >>> found with strace that cyrus didn't try to open the cyrus.squat >>> file for the >>> mailbox. >>> >>> I suspect that I am missed some configuration change. So here is our >>> imapd.conf for our backends >> >> If I'm correct you need : >> >> search_fuzzy_always: on >> >> in your config. >> >> If you can tell me if it's work...I would really appreciate. Because I >> activated that but I'm not able to see if it's work really. >> > Thanks for your help. > > I did tried it on my test server and it seems to be a bit faster, > but that could be due to caching. Strace still didn't show any access > to the cyrus.squat file. > > For information: We only use squatter. No Xapia. And we had much faster > search with Cyrus-Imapd 2.3.x and 2.4.x. I don't have the timings form > the old system but our users are complaining and they receive timeouts > in our horde/imp webmailer, which they did't receive before. > > Any other ideas are appreciated. I still have the problem that search in cyrus imap 3.0.8 with search engine squatter is slow compared to 2.4.20. I try to figure out if the squatter search engine is working in cyurs imapd 3.0 and I messed up my configuration, or if my configuration should work but squatter is broken. I did set up a test environment to compare the old and new versions. To verify that the search is indeed slower with 3.0.8 I used two different searches 'B SEARCH TEXT "Test"' and 'B SEARCH HEADER X-comment Unirundmail' === 2.4.20 === SEARCH TEXT A SELECT INBOX * 4321 EXISTS * 4321 RECENT * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] Ok * OK [UNSEEN 1] Ok * OK [UIDVALIDITY 1540372444] Ok * OK [UIDNEXT 93369] Ok * OK [HIGHESTMODSEQ 2] Ok * OK [URLMECH INTERNAL] Ok A OK [READ-WRITE] Completed B SEARCH TEXT "Test" * SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 33 34 35 36 37 38 39 .... 4292 4294 4295 4296 4298 4299 4300 4301 4303 4306 4307 4309 4310 4315 4316 4317 4318 4321 B OK Completed (1996 msgs in 0.690 secs) === 3.0.8 === SEARCH TEXT a SELECT INBOX * 4321 EXISTS * 0 RECENT * FLAGS (\Answered \Flagged \Draft \Deleted \Seen NonJunk $Forwarded $mdnsent $label1 $label2 $label3 hordetest testflag) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen NonJunk $Forwarded $mdnsent $label1 $label2 $label3 hordetest testflag \*)] Ok * OK [UNSEEN 3737] Ok * OK [UIDVALIDITY 1238498084] Ok * OK [UIDNEXT 93373] Ok * OK [HIGHESTMODSEQ 98491] Ok * OK [URLMECH INTERNAL] Ok * OK [ANNOTATIONS 65536] Ok a OK [READ-WRITE] Completed B SEARCH TEXT "Test" * SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 33 34 35 36 37 38 39 .... 4274 4275 4276 4277 4278 4279 4285 4286 4287 4288 4292 4294 4295 4296 4298 4299 4300 4301 4303 4306 4307 4309 4310 4315 4316 4317 4318 4321 B OK Completed (1935 msgs in 2.580 secs) ==== 2.4.20 === SEARCH HEADER a SELECT INBOX * 4321 EXISTS * 0 RECENT * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] Ok * OK [UNSEEN 1] Ok * OK [UIDVALIDITY 1540372444] Ok * OK [UIDNEXT 93369] Ok * OK [HIGHESTMODSEQ 2] Ok * OK [URLMECH INTERNAL] Ok a OK [READ-WRITE] Completed b SEARCH HEADER X-comment Unirundmail * SEARCH 4283 4291 4313 4319 4320 b OK Completed (5 msgs in 0.020 secs) ==== 3.0.8 === SEARCH HEADER a SELECT INBOX * 4321 EXISTS * 0 RECENT * FLAGS (\Answered \Flagged \Draft \Deleted \Seen NonJunk $Forwarded $mdnsent $label1 $label2 $label3 hordetest testflag) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen NonJunk $Forwarded $mdnsent $label1 $label2 $label3 hordetest testflag \*)] Ok * OK [UNSEEN 3737] Ok * OK [UIDVALIDITY 1238498084] Ok * OK [UIDNEXT 93373] Ok * OK [HIGHESTMODSEQ 98491] Ok * OK [URLMECH INTERNAL] Ok * OK [ANNOTATIONS 65536] Ok a OK [READ-WRITE] Completed b SEARCH HEADER X-comment Unirundmail * SEARCH 4283 4291 4313 4319 4320 b OK Completed (5 msgs in 0.370 secs) === There is also a big discrepancy between time indicated in the "OK Completed" and the time from sending the search command till the return of the result, which is 0.890 sec compared to ~30 sec on the production system. I used strace on the imapd processes while searching to verify that the squat file was used in 2.4 but not in 3.0. I could see open events for the squat file and the messages that where found for 2.4.20 and no open event (not even a failed one) to the squat file but instead open events for all message files in that folder for 3.0.8 I read the documentation and source code and tried to figure out if i am missing some config options, or if i could pinpoint a function where the search was turning the wrong way. I used "perf -g" the see the call graphs and to figure out where the call graphs change I can see that the same functions are called up to "index_search", and that the called functions change at that point. I know that the search code got restructured to support different search engines and that some functions got renamed. I have attached the perf report output, so that someone with a better understanding of the code can see whats going on. I can provide the perf.data files if it helps. Can someone confirm or refute that the squatter search engine is working with cyrus imapd 3.0.x? Is "search_engine: squat" in imapd.conf and a "squatter" event in cyrus.conf is sufficient to use the squatter search index in 3.0 or are there other options steps required. Regards Michael Menge PS. link to my original post with my imapd.conf https://lists.andrew.cmu.edu/pipermail/info-cyrus/2018-September/040395.html -------------------------------------------------------------------------------- M.Menge Tel.: (49) 7071/29-70316 Universit?t T?bingen Fax.: (49) 7071/29-5912 Zentrum f?r Datenverarbeitung mail: michael.menge at zdv.uni-tuebingen.de W?chterstra?e 76 72074 T?bingen -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: perf_TEXT_SEARCH_2.4.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: perf_TEXT_SEARCH_3.0.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: perf_HEADER_SEARCH_2.4.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: perf_HEADER_SEARCH_3.0.txt URL: