From ae at dal.ca Mon Jun 1 10:21:57 2009 From: ae at dal.ca (Aidan Evans) Date: Mon, 1 Jun 2009 11:21:57 -0300 (ADT) Subject: service imap pid nnnn in BUSY state: terminated abnormally Message-ID: I sent this last Friday but it never appeared (and I did not get any non-delivery report), so I'm trying again. I am attempting to upgrade a Cyrus Murder from 2.3.12p2 to 2.3.14. The mupdate and backend servers are happy at 2.3.14, but on the frontends while I can login successfully, attempting to select a folder fails with messages like May 29 12:13:59 kil-imap-13 master[15558]: process 15584 exited, signaled to death by 11 May 29 12:13:59 kil-imap-13 master[15558]: service imap pid 15584 in BUSY state: terminated abnormally This is on Red Hat Linux 5 (2.6.18-128.1.6.el5PAE kernel). I have not found anything in the Cyrus mailing list archive that looks like this. A Google search did find a similar problem from just a couple of weeks ago with a Debian installation which was attributed to "packages sasl2-bin, libsasl2-2 and libsasl2-modules". I am using the Red Hat SASL RPMs, version 2.1.22-4 which appears to be the latest. This has worked so far, so I have not installed SASL from Cyrus itself. The Cyrus SASL 2.1.23 announcement only talks about a "potential" buffer overflow which suggests to me that the change there is not relevant Should I switch to Cyrus SASL? The mailbox database is skiplist, seen and subscription are flat, and quota is legacy; others are Berkeley. cyradm "version" is name : Cyrus IMAPD version : v2.3.14 2009/03/25 10:41:00 vendor : Project Cyrus support-url: http://cyrusimap.web.cmu.edu os : Linux os-version : 2.6.18-128.el5PAE environment: Built w/Cyrus SASL 2.1.22 Running w/Cyrus SASL 2.1.22 Built w/Sleepycat Software: Berkeley DB 4.3.29: (September 12, 2006) Running w/Sleepycat Software: Berkeley DB 4.3.29: (September 12, 2006) Built w/OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008 Running w/OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008 CMU Sieve 2.3 TCP Wrappers mmap = shared lock = fcntl nonblock = fcntl idle = poll Aidan Evans | Networks & Systems (902)494-3332 | Dalhousie University, Halifax, N.S., Canada From simon.matter at invoca.ch Mon Jun 1 10:33:25 2009 From: simon.matter at invoca.ch (Simon Matter) Date: Mon, 1 Jun 2009 16:33:25 +0200 Subject: service imap pid nnnn in BUSY state: terminated abnormally In-Reply-To: References: Message-ID: <498f401f3221cdca4368011af5ccfff4.squirrel@webmail.bi.corp.invoca.ch> > I sent this last Friday but it never appeared (and I did not get any > non-delivery report), so I'm trying again. > > I am attempting to upgrade a Cyrus Murder from 2.3.12p2 to 2.3.14. The > mupdate and backend servers are happy at 2.3.14, but on the frontends > while > I can login successfully, attempting to select a folder fails with > messages > like > > May 29 12:13:59 kil-imap-13 master[15558]: process 15584 exited, signaled > to > death by 11 > May 29 12:13:59 kil-imap-13 master[15558]: service imap pid 15584 in BUSY > state: terminated abnormally I'm not sure that could be related but there is a bug in 2.3.14 which can result in unexpected behaviour. It may be unrelated but you better check it out: http://www.mail-archive.com/info-cyrus at lists.andrew.cmu.edu/msg37575.html > > This is on Red Hat Linux 5 (2.6.18-128.1.6.el5PAE kernel). > > I have not found anything in the Cyrus mailing list archive that looks > like > this. A Google search did find a similar problem from just a couple of > weeks > ago with a Debian installation which was attributed to "packages > sasl2-bin, > libsasl2-2 and libsasl2-modules". > > I am using the Red Hat SASL RPMs, version 2.1.22-4 which appears to be the > latest. This has worked so far, so I have not installed SASL from Cyrus > itself. The Cyrus SASL 2.1.23 announcement only talks about a "potential" > buffer overflow which suggests to me that the change there is not relevant >From what I know the sasl rpms from RHEL5 are fine. I don't remember anyone having problem with them. Regards, Simon From ae at dal.ca Mon Jun 1 17:37:15 2009 From: ae at dal.ca (Aidan Evans) Date: Mon, 1 Jun 2009 18:37:15 -0300 (ADT) Subject: service imap pid nnnn in BUSY state: terminated abnormally In-Reply-To: <498f401f3221cdca4368011af5ccfff4.squirrel@webmail.bi.corp.invoca.ch> References: <498f401f3221cdca4368011af5ccfff4.squirrel@webmail.bi.corp.invoca.ch> Message-ID: On Mon, 1 Jun 2009 at 16:33 Simon Matter wrote to Aidan Evans and... > I'm not sure that could be related but there is a bug in 2.3.14 which can > result in unexpected behaviour. It may be unrelated but you better check > it out: > http://www.mail-archive.com/info-cyrus at lists.andrew.cmu.edu/msg37575.html Thank you, I had noticed that and thought it was a bit outside what I was experiencing, but I'm going to have another look. P.S. Both my posts, this morning's and Friday's, are in the list archive, but I did not receive a copy, nor did the list copy of your email come appear. I checked my list settings and I am supposed to get a list copy. Aidan Evans | Networks & Systems (902)494-3332 | Dalhousie University, Halifax, N.S., Canada From Hagedorn at uni-koeln.de Tue Jun 2 03:49:26 2009 From: Hagedorn at uni-koeln.de (Sebastian Hagedorn) Date: Tue, 02 Jun 2009 09:49:26 +0200 Subject: Self-healing mailbox? In-Reply-To: <20090531124152.GA28931@brong.net> References: <4A22316D.4010401@gbif.org> <139F629B628639B2FEE835AD@G5.local> <20090531124152.GA28931@brong.net> Message-ID: --On 31. Mai 2009 22:41:52 +1000 Bron Gondwana wrote: >> I interpret the "System I/O error" to be the IOERROR from the line >> before, i.e. not actually an I/O error but rather a corrupt file. The >> error does not show on any of the previous days, nor does it show today. > > Sounds to me like it was copying the record to the cyrus.expunge.NEW > file each time for the earlier days (and failing, so skipping the > entire mailbox), but now the record has actually expired, so it > doesn't need to copy the cache record to the cyrus.cache.NEW file, > and hence never looks at it. > > Meaning: yes, it did heal itself! Thanks for the explanation! -- .:.Sebastian Hagedorn - RZKR-R1 (Geb?ude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universit?t zu K?ln / Cologne University - ? +49-221-478-5587.:. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 194 bytes Desc: not available Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090602/2641f6f0/attachment.bin From Bhavesh.Lad at disneyanimation.com Tue Jun 2 18:28:35 2009 From: Bhavesh.Lad at disneyanimation.com (Bhavesh Lad) Date: Tue, 02 Jun 2009 15:28:35 -0700 Subject: Load testing Cyrus IMAP Message-ID: <4A25A793.3080903@disney.com> Were currently running Cyrus on a single RHEL 4 (64 bit) and are planning on a migrating to 3 HP DL380 G5's running RHEL 5 (64 bit) with a fiber channel backend. I want to test the performance but not sure what to use. Does anybody have any recommendations on how to test the load via a script with variable users? Thanks! -Bhavesh From ktm at rice.edu Tue Jun 2 18:33:20 2009 From: ktm at rice.edu (Kenneth Marshall) Date: Tue, 2 Jun 2009 17:33:20 -0500 Subject: Load testing Cyrus IMAP In-Reply-To: <4A25A793.3080903@disney.com> References: <4A25A793.3080903@disney.com> Message-ID: <20090602223320.GO18879@it.is.rice.edu> mstone is worth checking out. Cheers, Ken On Tue, Jun 02, 2009 at 03:28:35PM -0700, Bhavesh Lad wrote: > Were currently running Cyrus on a single RHEL 4 (64 bit) and are > planning on a migrating to 3 HP DL380 G5's running RHEL 5 (64 bit) with > a fiber channel backend. I want to test the performance but not sure > what to use. Does anybody have any recommendations on how to test the > load via a script with variable users? > > Thanks! > > -Bhavesh > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > From thomas.cataldo at linagora.com Wed Jun 3 18:11:46 2009 From: thomas.cataldo at linagora.com (Thomas Cataldo) Date: Thu, 4 Jun 2009 00:11:46 +0200 Subject: Cyrus APIs ? In-Reply-To: <20090529005855.GA6708@brong.net> References: <20090529005855.GA6708@brong.net> Message-ID: 2009/5/29 Bron Gondwana : >> ? - custom authentification mechanism (for single sign-on purpose, >> because kerberos doesn't fit everywhere) > > BYO saslauthd protocol daemon. ?We have one written in Perl that does > all sorts of clever. ?Just put this in your imapd.conf > > sasl_pwcheck_method: saslauthd > > And have your daemon listen on a unix socket at: > > /var/state/saslauthd/mux > > You need to speak the saslauthd protocol, which is a packed string > format. ?We parse it in Perl like this: > > ?my $LoginName = get_counted_string($Self->{server}{client}); > ?my $Password = get_counted_string($Self->{server}{client}); > ?my $Service = lc get_counted_string($Self->{server}{client}); > ?my $Realm = get_counted_string($Self->{server}{client}); > > And return one of: > > ?use constant SASL_SUCC_RESP ?=> pack("nA3", 2, "OK\000"); > ?use constant SASL_FAIL_RESP ?=> pack("nA3", 2, "NO\000"); > > (with this function - slightly ugly code, but it works) > > sub get_counted_string { > ?my $fh = shift; > > ?my ($rd, $data); > > ?($rd = sysread($fh, $data, 2) ? ?) == 2 > ? ?or die "Unable to read counted string size ($rd != 2) ($!)"; > > ?my $size = unpack("n", $data); > > ?$data = ''; $rd = 0; my $this_data = ''; my $rem_size = $size; > ?while (my $this_rd = sysread($fh, $this_data, $rem_size)) { > ? ?$rd += $this_rd; > ? ?$rem_size -= $this_rd; > ? ?$data .= $this_data; > ?} > ?die "Unable to read counted string data ($rd != $size) ($!)" > ? ?unless ($rd ?== $size); > > ?return unpack("A$size", $data); > } > Thank you very much. This was so obvious but we might have sought that replacing saslauthd would be complicated. Thanks a lot, we'll probably solve everything with an homebrew saslauthd. From hans.moser at ofd-sth.niedersachsen.de Fri Jun 5 05:11:24 2009 From: hans.moser at ofd-sth.niedersachsen.de (Marc Patermann) Date: Fri, 05 Jun 2009 11:11:24 +0200 Subject: Migrate from 2.2. to 2.3 with ldap Message-ID: <4A28E13C.7050605@ofd-sth.niedersachsen.de> Hi, I'm a bit stuck. I want to migrate from 2.2.12 to a recent 2.3.x server. cyrus.conf includes ptloader cmd="ptloader" listen="/mail/imap/ptclient/ptsock" prefork=1 imapd.conf includes ldap_* parameters sasl_ldap* parameters (with auxprop ldapdb) Current install is with ldap enabled, build with --with-ldap=/opt/freeware --with-pts=ldap \ --with-auth=pts In 2.3. there is only "with-ldap" left. Is this enough to build with the intended ldap support? The available rpms from openSuSE Build Service (openSuSE 11.1 or SLES 11) form SLES 11 all do not have the with-ldap option set. I cannot find any ptloader binary after install. Do I not need any ptloader for ldap anymore? Marc From pcravero at as2594.net Fri Jun 5 10:40:52 2009 From: pcravero at as2594.net (Paolo Cravero) Date: Fri, 05 Jun 2009 16:40:52 +0200 Subject: murder and autocreate (I know it is not supported) Message-ID: <4A292E74.2080304@as2594.net> Hi. I'm setting up a test environment with frontend, backend, murder (currently 1, 1, 1 respectively). All using RPMs on recent RedHat systems. I have read in the docs that the autocreate patch does not work with murder, and I have experienced it too. But... ... if I leave the autocreate ON on backends only, and then simulate an IMAP access or mail delivery right at the backend level, I should not get into troubles, right? Just to ease the creation of mailboxes, especially during this deployment test. Or perhaps even after if our complex tools won't like the cyradm Perl interface. (yes, we will have a look at gosa and korreio) TIA, Paolo From lists at oliver-block.eu Mon Jun 8 17:29:26 2009 From: lists at oliver-block.eu (lists at oliver-block.eu) Date: Mon, 08 Jun 2009 23:29:26 +0200 Subject: =?UTF-8?Q?sasl=5Fpwcheck=5Fmethod?= Message-ID: <20090608212926.3833.qmail@h1599057.stratoserver.net> Hello everybody, I configured cyrus imapd on a Opensuse 11 machine following the recommedation in a README file. Now I discovered the following - for me odd behavior - which might depend on a "misconfiguration". /etc/imap.conf: sasl_pwcheck_method: saslauthd /etc/sysconfig/saslauthd: SASLAUTHD_AUTHMECH=pam If a user logs into cyrus (I used mtest from uw-imap because of it's debug messages) it takes 4 trials (3 with CRAM-MD5 and a final with plain password) before the login succeeds. By chance I've found a tutorial which recommends adding a user to sasldb2. I tried that and without any additional changes to the configuration the first login attempt succeeds. I wonder if someone could tell me 1. Why did it take 4 attempts using the system credentials 2. Why did it succeed with one attempts after a user with the same username and different password was added to sasldb2 3. Why did the sasldb2 approach succedd at all without any configuration changes. Your help is appreciated. Best Regards, Oliver Block -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090608/2c70d624/attachment.html From dwhite at olp.net Mon Jun 8 18:32:12 2009 From: dwhite at olp.net (Dan White) Date: Mon, 08 Jun 2009 17:32:12 -0500 Subject: sasl_pwcheck_method In-Reply-To: <20090608212926.3833.qmail@h1599057.stratoserver.net> References: <20090608212926.3833.qmail@h1599057.stratoserver.net> Message-ID: <4A2D916C.3060101@olp.net> lists at oliver-block.eu wrote: > > Hello everybody, > > I configured cyrus imapd on a Opensuse 11 machine following the > recommedation in a README file. Now I discovered the following - for > me odd behavior - which might depend on a "misconfiguration". > > /etc/imap.conf: > sasl_pwcheck_method: saslauthd > > /etc/sysconfig/saslauthd: > SASLAUTHD_AUTHMECH=pam > > If a user logs into cyrus (I used mtest from uw-imap because of it's > debug messages) it takes 4 trials (3 with CRAM-MD5 and a final with > plain password) before the login succeeds. > > By chance I've found a tutorial which recommends adding a user to > sasldb2. I tried that and without any additional changes to the > configuration the first login attempt succeeds. > > I wonder if someone could tell me > 1. Why did it take 4 attempts using the system credentials > 2. Why did it succeed with one attempts after a user with the same > username and different password was added to sasldb2 > 3. Why did the sasldb2 approach succedd at all without any > configuration changes. > When authenticating via CRAM-MD5, the pwcheck_method will be ignored. Your chosen pwcheck_method should only be referenced when authenticating via a 'plaintext' authentication mechanism - LOGIN or PLAIN. The fact that mtest attempted to authenticate via CRAM-MD5 probably means that you are advertising CRAM-MD5 support within imapd.conf. When authenticating via a mechanism which utilizes a shared secret, such as CRAM-MD5, your auxprop configuration will be used (sasl_auxprop_plugin). The default auxprop plugin is sasldb. If you are advertising CRAM-MD5 support in /etc/imapd.conf, but do not have the user configured in an auxprop store, then CRAM-MD5 should always fail. > 1. Why did it take 4 attempts using the system credentials mtest is probably falling back to PLAIN after 3 unsuccessful CRAM-MD5 login attempts. > 2. Why did it succeed with one attempts after a user with the same username and different password was added to sasldb2 > 3. Why did the sasldb2 approach succedd at all without any configuration changes. Because adding the user to your (default) auxprop store allowed CRAM-MD5 to succeed. If you are planning to support CRAM-MD5, you'll want to use: sasl_pwcheck_method: auxprop which will provide some consistency between PLAIN logins and CRAM-MD5 logins. It will not allow you to use PAM and you'll need to configure your users in /etc/sasldb2. If you don't care about supporting CRAM-MD5, then remove it from your 'sasl_mech_list', and you can stick with saslauthd and PAM. - Dan From lists at oliver-block.eu Mon Jun 8 19:19:49 2009 From: lists at oliver-block.eu (lists at oliver-block.eu) Date: Tue, 09 Jun 2009 01:19:49 +0200 Subject: =?UTF-8?Q?sasl=5Fpwcheck=5Fmethod?= Message-ID: <20090608231949.9387.qmail@h1599057.stratoserver.net> Dan White schrieb: > When authenticating via CRAM-MD5, the pwcheck_method will be ignored. > Your chosen pwcheck_method should only be referenced when > authenticating > via a 'plaintext' authentication mechanism - LOGIN or PLAIN. Good to know. I must have omitted this part of the manual.:-) > The fact > that mtest attempted to authenticate via CRAM-MD5 probably means that > you are advertising CRAM-MD5 support within imapd.conf. Actually cyrus seems to do that by his own!? Adding sasl_mech_list: PLAIN LOGIN to imapd.conf stops advertising it. As cyrus on this server will only be used by system users and with a secure connection, I think I will use it with PLAIN and pam. Thanks for help. Best Regards, Oliver Block -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090609/b4a4831a/attachment.html From pcravero at as2594.net Tue Jun 9 03:20:48 2009 From: pcravero at as2594.net (Paolo Cravero) Date: Tue, 09 Jun 2009 09:20:48 +0200 Subject: murder and cross-backend mailbox sharing Message-ID: <4A2E0D50.4090907@as2594.net> Hi. I have a setup with 1 frontend, 1 murder, 2 backends. I am playing with cross-backend mailbox sharing, one of the features we value most. Problem. While I can access shared mailboxes that reside on the same backend just fine, with Thunderbird (1.5/linux) I can see shared folders on other backends, but when I subscribe nothing is subscribed. Thunderbird always connects to the frontend machine. All cyrus servers have the same Invoca RPMs (v2.3.14-Invoca-RPM-2.3.14-2). Authentication via PAM LDAP, if that matters. I use virtdomains but all accounts are under the same @domain . ACL are "anyone lrs". Summarizing: I can access shared mailboxes on the same backend, but on different backends they cannot be subscribed, Something I should look for? TIA, Paolo From michael.menge at zdv.uni-tuebingen.de Tue Jun 9 03:56:27 2009 From: michael.menge at zdv.uni-tuebingen.de (Michael Menge) Date: Tue, 09 Jun 2009 09:56:27 +0200 Subject: murder and cross-backend mailbox sharing In-Reply-To: <4A2E0D50.4090907@as2594.net> References: <4A2E0D50.4090907@as2594.net> Message-ID: <20090609095627.15303nsk1uflwlgr@webmail.uni-tuebingen.de> Hi, Quoting Paolo Cravero : > Hi. > I have a setup with 1 frontend, 1 murder, 2 backends. I am playing with > cross-backend mailbox sharing, one of the features we value most. > > Problem. While I can access shared mailboxes that reside on the same backend > just fine, with Thunderbird (1.5/linux) I can see shared folders on other > backends, but when I subscribe nothing is subscribed. > The subscibtion is managed on the backend of the user. Cyrus checks if the Folder you whant to subscrib exists. This Check is done only on the backend. You have to use allowallsubscribe: 1 Allow subscription to nonexistent mailboxes. This option is typi- cally used on backend servers in a Murder so that users can sub- scribe to mailboxes that don't reside on their "home" server. This option can also be used as a workaround for IMAP clients which don't play well with nonexistent or unselectable mailboxes (eg. Microsoft Outlook). -------------------------------------------------------------------------------- M.Menge Tel.: (49) 7071/29-70316 Universit?t T?bingen Fax.: (49) 7071/29-5912 Zentrum f?r Datenverarbeitung mail: michael.menge at zdv.uni-tuebingen.de W?chterstra?e 76 72074 T?bingen -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5339 bytes Desc: S/MIME krytographische Unterschrift Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090609/e39926d0/attachment.bin From simon.matter at invoca.ch Tue Jun 9 04:05:47 2009 From: simon.matter at invoca.ch (Simon Matter) Date: Tue, 9 Jun 2009 10:05:47 +0200 Subject: murder and cross-backend mailbox sharing In-Reply-To: <4A2E0D50.4090907@as2594.net> References: <4A2E0D50.4090907@as2594.net> Message-ID: > Hi. > I have a setup with 1 frontend, 1 murder, 2 backends. I am playing with > cross-backend mailbox sharing, one of the features we value most. > > Problem. While I can access shared mailboxes that reside on the same > backend > just fine, with Thunderbird (1.5/linux) I can see shared folders on other > backends, but when I subscribe nothing is subscribed. > > Thunderbird always connects to the frontend machine. All cyrus servers > have > the same Invoca RPMs (v2.3.14-Invoca-RPM-2.3.14-2). Authentication via PAM > LDAP, if that matters. > > I use virtdomains but all accounts are under the same @domain . ACL are > "anyone lrs". I'm not sure it's related but you really should upgrade to the 2.3.14-6 package. 2.3.14 has a bug which is patched in the 2.3.14-6 RPM. Regards, Simon From pcravero at as2594.net Tue Jun 9 04:10:38 2009 From: pcravero at as2594.net (Paolo Cravero) Date: Tue, 09 Jun 2009 10:10:38 +0200 Subject: murder and cross-backend mailbox sharing In-Reply-To: References: <4A2E0D50.4090907@as2594.net> Message-ID: <4A2E18FE.8000404@as2594.net> Simon Matter wrote: > I'm not sure it's related but you really should upgrade to the 2.3.14-6 > package. 2.3.14 has a bug which is patched in the 2.3.14-6 RPM. Thank you Simon. I'm in the staging phase, so still looking for features rather than bugs :) BTW, thank you for providing those wonderful RPMs. Anyway Michael pointed out the missing configuration parameter. Now x-backend sharing works and I've also seen in the log why autocreate and murder don't work well together. Will post a reply in the proper thread I opened on June 5th. Paolo From hans.moser at ofd-sth.niedersachsen.de Tue Jun 9 12:00:43 2009 From: hans.moser at ofd-sth.niedersachsen.de (Marc Patermann) Date: Tue, 09 Jun 2009 18:00:43 +0200 Subject: Migrate from 2.2. to 2.3 with ldap In-Reply-To: References: Message-ID: <4A2E872B.5020202@ofd-sth.niedersachsen.de> Hi, Marc Patermann schrieb: > I'm a bit stuck. I want to migrate from 2.2.12 to a recent 2.3.x server. Does no one use 2.3 with ldap and can point in the right direction? > Do I not need any ptloader for ldap anymore? If I not wrong, cyrus 2.3 complains about not being able to connect to ptloader, so it seems to be necessary ... Marc From michael.menge at zdv.uni-tuebingen.de Tue Jun 9 12:38:45 2009 From: michael.menge at zdv.uni-tuebingen.de (Michael Menge) Date: Tue, 09 Jun 2009 18:38:45 +0200 Subject: Migrate from 2.2. to 2.3 with ldap In-Reply-To: <4A2E872B.5020202@ofd-sth.niedersachsen.de> References: <4A2E872B.5020202@ofd-sth.niedersachsen.de> Message-ID: <20090609183845.12303kbimqdcdyj9@webmail.uni-tuebingen.de> Hi, Quoting Marc Patermann : > Hi, > > Marc Patermann schrieb: > >> I'm a bit stuck. I want to migrate from 2.2.12 to a recent 2.3.x server. > Does no one use 2.3 with ldap and can point in the right direction? > We use Cyrus 2.3 with pam mand pam_ldap, if you don't need support for group permissions, this works fine without ptloader. As we don't use group permissions i don't know if this is possible without ptloader, or how to make it work. >> Do I not need any ptloader for ldap anymore? > If I not wrong, cyrus 2.3 complains about not being able to connect to > ptloader, so it seems to be necessary ... > > > Marc > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > -------------------------------------------------------------------------------- M.Menge Tel.: (49) 7071/29-70316 Universit?t T?bingen Fax.: (49) 7071/29-5912 Zentrum f?r Datenverarbeitung mail: michael.menge at zdv.uni-tuebingen.de W?chterstra?e 76 72074 T?bingen -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5339 bytes Desc: S/MIME krytographische Unterschrift Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090609/81adf73c/attachment.bin From Duncan.Gibb at SiriusIT.co.uk Tue Jun 9 12:55:29 2009 From: Duncan.Gibb at SiriusIT.co.uk (Duncan Gibb) Date: Tue, 09 Jun 2009 17:55:29 +0100 Subject: murder and autocreate (I know it is not supported) In-Reply-To: <4A292E74.2080304@as2594.net> References: <4A292E74.2080304@as2594.net> Message-ID: <4A2E9401.8080706@SiriusIT.co.uk> Paolo Cravero wrote: PC> I have read in the docs that the autocreate patch does not work PC> with murder, and I have experienced it too. But... PC> ... if I leave the autocreate ON on backends only, and then PC> simulate an IMAP access or mail delivery right at the backend PC> level, I should not get into troubles, right? You might be interested in the attached patch, which is something I cooked up last year for our internal Cyrus builds. It's a fairly dirty hack to provide frontends with a way to decide which backend should they go to for a mailbox that doesn't yet exist. The basic idea is to catch lookups by proxyd and lmtpproxyd which would normally return IMAP_MAILBOX_NONEXISTENT and call out to an administrator-defined helper script (roughly modelled on the way Squid auth helpers work) which decides whether and where to assume the mailbox should exist. Then normal auto* on the backends kicks in... There are many ways in which this could be improved. Most obviously admins shouldn't be required to maintain the helper script across all the FEs - the decision logic should be at a central location such as the MUPDATE host. And it would be good to extend the information passed to the helper (eg authenticated sender info in the LMTP case), and back (eg a way to set mailbox ACLs or other config). The latter would require much better integration with the UoA patches. The protocol spoken between the Cyrus components and the helper app should resemble some sane IMAP-related one... Ideas and improvements welcome ;-) Cheers Duncan -- Duncan Gibb - Technical Director Sirius Corporation plc - control through freedom http://www.siriusit.co.uk/ t: +44 870 608 0063 || m: +44 7977 441 515 -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 95-automurder-frontend.dpatch Url: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090609/5afe39a8/attachment-0001.ksh From dave64 at andrew.cmu.edu Tue Jun 9 13:11:39 2009 From: dave64 at andrew.cmu.edu (Dave McMurtrie) Date: Tue, 09 Jun 2009 13:11:39 -0400 Subject: murder and autocreate (I know it is not supported) In-Reply-To: <4A2E9401.8080706@SiriusIT.co.uk> References: <4A292E74.2080304@as2594.net> <4A2E9401.8080706@SiriusIT.co.uk> Message-ID: <4A2E97CB.5070902@andrew.cmu.edu> Duncan Gibb wrote: > Paolo Cravero wrote: > > PC> I have read in the docs that the autocreate patch does not work > PC> with murder, and I have experienced it too. But... > > PC> ... if I leave the autocreate ON on backends only, and then > PC> simulate an IMAP access or mail delivery right at the backend > PC> level, I should not get into troubles, right? > > You might be interested in the attached patch, which is something I > cooked up last year for our internal Cyrus builds. It's a fairly dirty > hack to provide frontends with a way to decide which backend should they > go to for a mailbox that doesn't yet exist. Ken is actually working on some code now that will integrate this functionality into Cyrus. I'm short on details right now (exactly how it will work and exactly when it will be done) but I thought it would interest you to know that it's being worked on. Thanks, Dave -- Dave McMurtrie, SPE Email Systems Team Leader Carnegie Mellon University, Computing Services From Duncan.Gibb at SiriusIT.co.uk Tue Jun 9 13:30:31 2009 From: Duncan.Gibb at SiriusIT.co.uk (Duncan Gibb) Date: Tue, 09 Jun 2009 18:30:31 +0100 Subject: murder and autocreate (I know it is not supported) In-Reply-To: <4A2E97CB.5070902@andrew.cmu.edu> References: <4A292E74.2080304@as2594.net> <4A2E9401.8080706@SiriusIT.co.uk> <4A2E97CB.5070902@andrew.cmu.edu> Message-ID: <4A2E9C37.1080702@SiriusIT.co.uk> Dave McMurtrie wrote: DMcM> Ken is actually working on some code now that will integrate DMcM> this functionality into Cyrus. I'm short on details right now DMcM> (exactly how it will work and exactly when it will be done) I don't think you're allowed to specify both of those ;-) DMcM> but I thought it would interest you to know that it's being DMcM> worked on. Definitely. Anything that anyone else could contribute to? Top of our list obviously is arbitrary LDAP-based decision making (which we've done by outsourcing to perl). Setting ACLs at create time would come next, and might still get implemented in our patch. Cheers Duncan -- Duncan Gibb - Technical Director Sirius Corporation plc - control through freedom http://www.siriusit.co.uk/ || t: +44 870 608 0063 Debian Cyrus Team - https://alioth.debian.org/projects/pkg-cyrus-imapd/ From steph at nuken.net Tue Jun 9 13:46:23 2009 From: steph at nuken.net (Stephane Kenn) Date: Tue, 09 Jun 2009 13:46:23 -0400 Subject: libsasl2 Message-ID: <20090609134623.58102o0a7eeh5quc@192.168.1.8> I keep seeing this message in my logs: Could not find a dlname line in .la file: libsasl2.la Running FreeBSD 7.2, perl 5-10 and cyrus-sasl 2.1.23, cyrus-imap 2.3.12. I see the file in /usr/local/lib and here are its content: # libsasl2.la - a libtool library file # Generated by ltmain.sh - GNU libtool 1.3.5 (1.385.2.206 2000/05/27 11:12:27) # # Please DO NOT delete this file! # It is necessary for linking the library. # The name that we can dlopen(3). dlname='' # Names of this library. library_names='libsasl2.so.2.23 libsasl2.so.2.23' # The name of the static archive. old_library='' # Libraries that this one depends upon. dependency_libs='' # Version information for libsasl2. current=2 age=0 revision=23 # Is this an already installed library? installed=no # Directory that this library needs to be installed in: libdir='/usr/local/lib' ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From brong at fastmail.fm Tue Jun 9 19:36:28 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Wed, 10 Jun 2009 09:36:28 +1000 Subject: murder and autocreate (I know it is not supported) In-Reply-To: <4A2E9C37.1080702@SiriusIT.co.uk> References: <4A292E74.2080304@as2594.net> <4A2E9401.8080706@SiriusIT.co.uk> <4A2E97CB.5070902@andrew.cmu.edu> <4A2E9C37.1080702@SiriusIT.co.uk> Message-ID: <20090609233628.GA14510@brong.net> On Tue, Jun 09, 2009 at 06:30:31PM +0100, Duncan Gibb wrote: > Dave McMurtrie wrote: > > DMcM> Ken is actually working on some code now that will integrate > DMcM> this functionality into Cyrus. I'm short on details right now > DMcM> (exactly how it will work and exactly when it will be done) > > I don't think you're allowed to specify both of those ;-) No, no, you are. You're just not allowed to specify how _correctly_ it will work at the same time :) Bron ( for some definition of "done" ) From woods-cyrus at weird.com Tue Jun 9 21:32:35 2009 From: woods-cyrus at weird.com (Greg A. Woods) Date: Tue, 09 Jun 2009 21:32:35 -0400 Subject: sasl_pwcheck_method In-Reply-To: <20090608231949.9387.qmail@h1599057.stratoserver.net> References: <20090608231949.9387.qmail@h1599057.stratoserver.net> Message-ID: At Tue, 09 Jun 2009 01:19:49 +0200, lists at oliver-block.eu wrote: Subject: Re: Re: sasl_pwcheck_method > > Dan White schrieb: > > When authenticating via CRAM-MD5, the pwcheck_method will be ignored. > > Your chosen pwcheck_method should only be referenced when > > authenticating > > via a 'plaintext' authentication mechanism - LOGIN or PLAIN. > > Good to know. I must have omitted this part of the manual.:-) > > > > The fact > > that mtest attempted to authenticate via CRAM-MD5 probably means that > > you are advertising CRAM-MD5 support within imapd.conf. > > Actually cyrus seems to do that by his own!? Adding sasl_mech_list: PLAIN LOGIN to imapd.conf stops advertising it. I've had the following in my template imapd.conf file for years now: # Use these SASL authentication mechanisms. # # Don't use CRAM-MD5 or DIGEST-MD5 if you don't have a local sasldb # and you start saslauthd with "-a getpwent" # # Don't use OTP or ANONYMOUS unless you really need them -- it causes some # clients to prefer it, such as "cyradm". # # Don't put PLAIN before LOGIN -- it buggers Mozilla. # sasl_mech_list: LOGIN PLAIN I'm not sure why Mozilla was confused, or whether current versions would still be confused, but suffice it to say that no current clients I've encountered in relatively large user populations have had problems with the order being "LOGIN PLAIN". -- Greg A. Woods +1 416 218-0098 VE3TCP RoboHack Planix, Inc. Secrets of the Weird From pcravero at as2594.net Wed Jun 10 03:53:45 2009 From: pcravero at as2594.net (Paolo Cravero) Date: Wed, 10 Jun 2009 09:53:45 +0200 Subject: murder and autocreate (I know it is not supported) In-Reply-To: <4A292E74.2080304@as2594.net> References: <4A292E74.2080304@as2594.net> Message-ID: <4A2F6689.4000309@as2594.net> Thank you Duncan for the info about the patch. I would try it, but I'm sure it would not be accepted by the tech stuff as it is not official. As I said in another thread, this is what happens when autocreate is ON in a murder setup with several backends, and a user accesses a shared mailbox residing on a different server. This is an excerpt of backend1 maillog: canonified backend1 -> backend1 mupdate NO response: mailbox already exists MUPDATE: can't reserve mailbox entry for 'example.com!user.onBE2' autocreateinbox: User onBE2 at example.com, INBOX failed. unable to reserve mailbox on mupdate server seen_db: user onBE2 at example.com opened /var/lib/imap/domain/e/example.com/user/z/onBE2.seen open: user onBE2 at example.com opened user/onBE1/778899 In words, the backend not owning the mailbox tries to create it, but receives a deny from the murder server. Then it opens the target mailbox (onBE1/778899) and creates the .seen structure. So far this means a messy maillog file. I will use autocreate just to populate my backends from the backend itself (with some imtest iteration, or whatever) and then switch it off. Looking forward to a MUPDATE protocol update or cyrus official patch. Paolo From hans.moser at ofd-sth.niedersachsen.de Wed Jun 10 07:38:22 2009 From: hans.moser at ofd-sth.niedersachsen.de (Marc Patermann) Date: Wed, 10 Jun 2009 13:38:22 +0200 Subject: Migrate from 2.2. to 2.3 with ldap In-Reply-To: <20090609183845.12303kbimqdcdyj9@webmail.uni-tuebingen.de> References: <4A2E872B.5020202@ofd-sth.niedersachsen.de> <20090609183845.12303kbimqdcdyj9@webmail.uni-tuebingen.de> Message-ID: <4A2F9B2E.5060309@ofd-sth.niedersachsen.de> Hi, Michael Menge schrieb: >> Marc Patermann schrieb: >> >>> I'm a bit stuck. I want to migrate from 2.2.12 to a recent 2.3.x server. >> Does no one use 2.3 with ldap and can point in the right direction? > We use Cyrus 2.3 with pam mand pam_ldap, if you don't need > support for group permissions, this works fine without ptloader. I'm not sure, is this the sasl part with saslauthd I supose or for auth_mech: unix? I don't want to use PAM anyway. I tried auth_mech: pts pts_module: ldap So I think I need ptloader with ldap. Without the ptloader service configured in cyrus.conf, cyrus nevertheless creates a ptcache.db. ptload() tries to ping ptloader, what fails. "can't connect to ptloader server: no such file or directory No data available at all from ptload()" With the service cyrus complains about not being able to start is (because the is no such binary). Marc From dave64 at andrew.cmu.edu Wed Jun 10 07:38:31 2009 From: dave64 at andrew.cmu.edu (Dave McMurtrie) Date: Wed, 10 Jun 2009 07:38:31 -0400 Subject: murder and autocreate (I know it is not supported) In-Reply-To: <4A2F6689.4000309@as2594.net> References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> Message-ID: <4A2F9B37.3040009@andrew.cmu.edu> Paolo Cravero wrote: > In words, the backend not owning the mailbox tries to create it, but receives > a deny from the murder server. Then it opens the target mailbox (onBE1/778899) > and creates the .seen structure. > > So far this means a messy maillog file. I will use autocreate just to populate > my backends from the backend itself (with some imtest iteration, or whatever) > and then switch it off. What Ken is working on isn't specific to autocreate. Rather, he's working to integrate IDM into our environment. We need a way for our identity management system to be able to simply connect to a Cyrus frontend and issue a create command and have something useful happen. I believe that Ken's work will involve a default server/partition instead of the current partition-default that assumes either a single server environment or a backend server is being connected to. Also, I believe he's setting up an annotation to determine which backend server has the most free space available. If I'm wrong on any of this, Ken will correct me. > > Looking forward to a MUPDATE protocol update or cyrus official patch. Buy Ken lots of beer. It makes him work faster. :) Thanks, Dave -- Dave McMurtrie, SPE Email Systems Team Leader Carnegie Mellon University, Computing Services From hans.moser at ofd-sth.niedersachsen.de Wed Jun 10 12:23:56 2009 From: hans.moser at ofd-sth.niedersachsen.de (Marc Patermann) Date: Wed, 10 Jun 2009 18:23:56 +0200 Subject: Migrate from 2.2. to 2.3 with ldap In-Reply-To: References: <4A2E872B.5020202@ofd-sth.niedersachsen.de> <20090609183845.12303kbimqdcdyj9@webmail.uni-tuebingen.de> Message-ID: <4A2FDE1C.10606@ofd-sth.niedersachsen.de> Hi, Marc Patermann schrieb: I tried a walk with CenOS 5.3 now, which provides ldapdb and cyrus with ldap and ptloader (that is what ldd ptloader says). I configured imapd.conf and cyrus.conf. ps shows ptloader is running. The defined socket is created. srwxrwxrwx 1 root root 0 10. Jun 17:00 /var/lib/imap/socket/ptclient Nevertheless cyrus complains: > "can't connect to ptloader server: no such file or directory > No data available at all from ptload()" Marc From schweizer.martin at gmail.com Thu Jun 11 01:42:59 2009 From: schweizer.martin at gmail.com (Martin Schweizer) Date: Thu, 11 Jun 2009 07:42:59 +0200 Subject: Auth mech for sync_client/sync_server Message-ID: <380ccfd60906102242m41cb970bk4d52258efdab0f54@mail.gmail.com> Hello I have two server which are up to date with Cyrus IMAP v2.3.14 on FreeBSD 7.1. I sync them with the following config options: Server2 (sync_client) /usr/local/etc/imapd.conf [snip] sync_host: server4 sync_authname: user sync_password: password sync_machineid: 1 sync_log: 1 sync_repeat_interval: 1 [snip] Server4 (sync_master) /usr/local/etc/imapd.conf [snip] sync_machineid: 2 sync_repeat_interval: 1 [snip] So far all works well. In /var/log/messages I see regulary the following message: Jun 11 07:15:20 server4 syncserver[79013]: login: server2 [192.168.20.3] cyrus DIGEST-MD5 User logged in This means that Server2 logs in Server4 and download the data. Can I change the mech DIGEST-MD5 in any way? Or has the parameter sasl_mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 in /usr/local/etc/imapd.conf any action to the sync_client/sync_server communication? I ask this because my clients do not longer need DIGEST-MD5. Kind regards, -- Martin Schweizer schweizer.martin at gmail.com Tel.: +41 32 512 48 54 (VoIP) Fax: +1 619 3300587 From woods-cyrus at weird.com Thu Jun 11 18:38:31 2009 From: woods-cyrus at weird.com (Greg A. Woods) Date: Thu, 11 Jun 2009 18:38:31 -0400 Subject: murder and autocreate (I know it is not supported) In-Reply-To: <4A2F9B37.3040009@andrew.cmu.edu> References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> Message-ID: At Wed, 10 Jun 2009 07:38:31 -0400, Dave McMurtrie wrote: Subject: Re: murder and autocreate (I know it is not supported) > > What Ken is working on isn't specific to autocreate. Rather, he's > working to integrate IDM into our environment. We need a way for our > identity management system to be able to simply connect to a Cyrus > frontend and issue a create command and have something useful happen. I've never understood this idea of why anything that has anything to do with user management should _ever_ have anything to do with mailbox creation. Mailboxes (i.e. the default first INBOX for every user) should always be created _automatically_ as needed for every valid user. The easiest way to do this is to trust the mail delivery system to have already verified the existence of every authorized mail user. This is quite safe to do because if there is any reason you can't do so then your mail system is broken, by definition, anyway. (i.e. if your MTA cannot be trusted to only accept and to deliver mail for valid users then it will no doubt be generating backscatter and it will be abused by those who do such dastardly things) Why make everything far more complicated than it needs to be? Especially things related to user management? -- Greg A. Woods +1 416 218-0098 VE3TCP RoboHack Planix, Inc. Secrets of the Weird From duncan.gibb at siriusit.co.uk Thu Jun 11 18:57:18 2009 From: duncan.gibb at siriusit.co.uk (Duncan Gibb) Date: Thu, 11 Jun 2009 23:57:18 +0100 Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> Message-ID: <4A318BCE.9070400@siriusit.co.uk> Greg A. Woods wrote: DMcM> What Ken is working on isn't specific to autocreate. Rather, DMcM> he's working to integrate IDM into our environment. We need DMcM> a way for our identity management system to be able to simply DMcM> connect to a Cyrus frontend and issue a create command and DMcM> have something useful happen. GAW> I've never understood this idea of why anything that has GAW> anything to do with user management should _ever_ have GAW> anything to do with mailbox creation. I would tend to agree, actually. An identity management system's job is to create the context within which mail may potentially be delivered or a user may potentially authenticate. IMHO mailbox creation should be the consequence of /that/ action, not something the management system does or does to the store. GAW> Mailboxes (i.e. the default first INBOX for every user) should GAW> always be created _automatically_ as needed for every valid user. Which is what makes UoA autocreate so handy... GAW> The easiest way to do this is to trust the mail delivery system GAW> to have already verified the existence of every authorized mail GAW> user. Yep, but the MTA (deliberately) doesn't know everything that goes on inside the store. One of the things the MTA doesn't and shouldn't know is which physical backend box inside a murder the user (or shared mailbox) should be created on - hence the hack. Cheers Duncan -- Duncan Gibb - Technical Director Sirius Corporation plc - control through freedom http://www.siriusit.co.uk/ || t: +44 870 608 0063 Debian Cyrus Team - https://alioth.debian.org/projects/pkg-cyrus-imapd/ From dave64 at andrew.cmu.edu Thu Jun 11 18:59:00 2009 From: dave64 at andrew.cmu.edu (Dave McMurtrie) Date: Thu, 11 Jun 2009 18:59:00 -0400 Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> Message-ID: On Jun 11, 2009, at 6:38 PM, "Greg A. Woods" wrote: > At Wed, 10 Jun 2009 07:38:31 -0400, Dave McMurtrie > wrote: > Subject: Re: murder and autocreate (I know it is not supported) >> >> What Ken is working on isn't specific to autocreate. Rather, he's >> working to integrate IDM into our environment. We need a way for our >> identity management system to be able to simply connect to a Cyrus >> frontend and issue a create command and have something useful happen. > > I've never understood this idea of why anything that has anything to > do > with user management should _ever_ have anything to do with mailbox > creation. > > Mailboxes (i.e. the default first INBOX for every user) should > always be > created _automatically_ as needed for every valid user. > > The easiest way to do this is to trust the mail delivery system to > have > already verified the existence of every authorized mail user. This is > quite safe to do because if there is any reason you can't do so then > your mail system is broken, by definition, anyway. (i.e. if your MTA > cannot be trusted to only accept and to deliver mail for valid users > then it will no doubt be generating backscatter and it will be > abused by > those who do such dastardly things) > > Why make everything far more complicated than it needs to be? > Especially things related to user management? > A valid point to mailbox creation, but what would delete the mailbox when a student graduates? Dave From brong at fastmail.fm Thu Jun 11 20:25:54 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Fri, 12 Jun 2009 10:25:54 +1000 Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> Message-ID: <20090612002554.GA7164@brong.net> On Thu, Jun 11, 2009 at 06:59:00PM -0400, Dave McMurtrie wrote: > > On Jun 11, 2009, at 6:38 PM, "Greg A. Woods" > wrote: > > Why make everything far more complicated than it needs to be? > > Especially things related to user management? > > A valid point to mailbox creation, but what would delete the mailbox > when a student graduates? You ;) Seriously though, you would run some sort of audit every so often that compares mailboxes to your authentication database. We also delete mailboxes after a fixed amount of time with no logins for our free accounts, or a time after which they stopped paying for our paid accounts, which I guess is similar to graduating... I guess you could add an option to cyr_expire to query the authentication database for each user.* mailbox and optionally delete it if the user was no longer present. Dangerous if your authentication daemon is offline though! For that matter, does sasl let you check if a user exists without knowing their password? Bron. From morgan at orst.edu Thu Jun 11 20:37:34 2009 From: morgan at orst.edu (Andrew Morgan) Date: Thu, 11 Jun 2009 17:37:34 -0700 (PDT) Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> Message-ID: On Thu, 11 Jun 2009, Dave McMurtrie wrote: >>> What Ken is working on isn't specific to autocreate. Rather, he's >>> working to integrate IDM into our environment. We need a way for our >>> identity management system to be able to simply connect to a Cyrus >>> frontend and issue a create command and have something useful happen. >> >> I've never understood this idea of why anything that has anything to >> do >> with user management should _ever_ have anything to do with mailbox >> creation. >> >> Mailboxes (i.e. the default first INBOX for every user) should >> always be >> created _automatically_ as needed for every valid user. >> >> The easiest way to do this is to trust the mail delivery system to >> have >> already verified the existence of every authorized mail user. This is >> quite safe to do because if there is any reason you can't do so then >> your mail system is broken, by definition, anyway. (i.e. if your MTA >> cannot be trusted to only accept and to deliver mail for valid users >> then it will no doubt be generating backscatter and it will be >> abused by >> those who do such dastardly things) >> >> Why make everything far more complicated than it needs to be? >> Especially things related to user management? >> > > A valid point to mailbox creation, but what would delete the mailbox > when a student graduates? It is really quite trivial to write small scripts (perl, php, python, etc) to manage Cyrus mailboxes. I don't know why folks do all this work by hand... I don't like the thought of Cyrus creating mailboxes on its own. One can simply add mailbox creation to all the other steps of provisioning a new account (creating an LDAP entry, making a home directory, setting quotas, etc). Even if I were to make accounts "by hand", I would still write a script to do all the steps so it is repeatable! :) Andy From matteo at corti.li Fri Jun 12 06:04:38 2009 From: matteo at corti.li (Matteo Corti) Date: Fri, 12 Jun 2009 12:04:38 +0200 Subject: procmail, deliver and failure (65) Message-ID: Hi, I use procmail to filter emails tagged by spamassassin and use deliver to put them in the mailbox as follows: :0 w | /usr/lib/cyrus-imapd/deliver -a $USER -m user.$USER after a recent update I am getting an exit code 65 from deliver which, if I am not wrong, means a wrong formatting. Following several suggestions I removed the From header from the email: :0f | formail -I "From " :0 w | /usr/lib/cyrus-imapd/deliver -a $USER -m user.$USER but I always get the same error. I then saved emails to see what the problem could be. :0f | formail -I "From " :0 c | cat > /tmp/backup :0 w | /usr/lib/cyrus-imapd/deliver -a $USER -m user.$USER This is what I get: $ cat /tmp/backup Return-Path: Received: by matteocorti.ch (Postfix, from userid 0) id 8339B2EC003; Fri, 12 Jun 2009 12:01:47 +0200 (CEST) Date: Fri, 12 Jun 2009 12:01:47 +0200 To: corti at matteocorti.ch Subject: Testsubject User-Agent: Heirloom mailx 12.4 7/29/08 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20090612100147.8339B2EC003 at matteocorti.ch> From: root at matteocorti.ch (root) Test text END $ Do you have any hint at what could be the problem? I am using version 2.3.14 Many thanks, Matteo -- Matteo Corti http://matteocorti.ch From hans.moser at ofd-sth.niedersachsen.de Fri Jun 12 07:38:28 2009 From: hans.moser at ofd-sth.niedersachsen.de (Marc Patermann) Date: Fri, 12 Jun 2009 13:38:28 +0200 Subject: Migrate from 2.2. to 2.3 with ldap In-Reply-To: References: <4A2E872B.5020202@ofd-sth.niedersachsen.de> <20090609183845.12303kbimqdcdyj9@webmail.uni-tuebingen.de> Message-ID: <4A323E34.9020904@ofd-sth.niedersachsen.de> Hi, Marc Patermann schrieb: > Marc Patermann schrieb: Back to my little monolog here. :) > I tried a walk with CenOS 5.3 now, which provides ldapdb and cyrus with > ldap and ptloader (that is what ldd ptloader says). > > I configured imapd.conf and cyrus.conf. > ps shows ptloader is running. The defined socket is created. > srwxrwxrwx 1 root root 0 10. Jun 17:00 /var/lib/imap/socket/ptclient I found out, that I forgot ptloader_socket in imapd.conf http://cyrusimap.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=44383 auth_mech: pts pts_module: ldap ptloader_socket: /var/lib/imap/socket/ptclient Why do I need to note it twice - it is already in cyrus.conf!? On CentOS it ist working now. Now that I have a working config, I'll go back to SuSE to try again ... Marc From wes at umich.edu Fri Jun 12 20:28:43 2009 From: wes at umich.edu (Wesley Craig) Date: Fri, 12 Jun 2009 20:28:43 -0400 Subject: Migrate from 2.2. to 2.3 with ldap In-Reply-To: <4A2F9B2E.5060309@ofd-sth.niedersachsen.de> References: <4A2E872B.5020202@ofd-sth.niedersachsen.de> <20090609183845.12303kbimqdcdyj9@webmail.uni-tuebingen.de> <4A2F9B2E.5060309@ofd-sth.niedersachsen.de> Message-ID: <6CEE81BA-E68F-4BBD-8B9C-96109BE75F44@umich.edu> On 10 Jun 2009, at 07:38, Marc Patermann wrote: > Michael Menge schrieb: >>> Marc Patermann schrieb: >>> >>>> I'm a bit stuck. I want to migrate from 2.2.12 to a recent 2.3.x >>>> server. >>> Does no one use 2.3 with ldap and can point in the right direction? >> We use Cyrus 2.3 with pam mand pam_ldap, if you don't need >> support for group permissions, this works fine without ptloader. > I'm not sure, is this the sasl part with saslauthd I supose or for > auth_mech: unix? Are you trying to use LDAP for authentication of users or for population of group membership? :wes From wes at umich.edu Fri Jun 12 20:34:46 2009 From: wes at umich.edu (Wesley Craig) Date: Fri, 12 Jun 2009 20:34:46 -0400 Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> Message-ID: <18265F8D-0755-4BB2-8501-B02E80932EF6@umich.edu> On 11 Jun 2009, at 18:59, Dave McMurtrie wrote: > A valid point to mailbox creation, but what would delete the mailbox > when a student graduates? There is no mailbox location problem for deletes. IDM simply connects to any frontend, permits itself, and then deletes the mailbox hierarchy. :wes From baconm at email.unc.edu Sat Jun 13 16:22:03 2009 From: baconm at email.unc.edu (Michael Bacon) Date: Sat, 13 Jun 2009 16:22:03 -0400 Subject: MUPDATE database problems -- help greatly appreciated Message-ID: <00BFAADC8FC3B60673A4103D@dhcp00032.its.unc.edu> Hello all, We're in the middle of trying to move from our single server installation to a new murder installation on all new hardware. We're getting into the late stages of setup, when we've run into a killer problem with getting the old server to sync up with the MUPDATE server so that we can migrate off of it. We're under a deadline to get the expensive new hardware rolled out into production, so any help would be enormously appreciated. The test installation with a test backend of, oh, a couple dozen mailboxes worked flawlessly. Syncing happened just as it was supposed to, and everything looked good for production. The next step was to start the old server syncing its database with the MUPDATE server, and that's where we're stuck. The initial sync from the old backend works just fine. During the second sync, however (ctl_mboxlist -m), the backend connects to the MUPDATE server, executes a LIST , and then the server returns somewhere between 2500-10,000 lines (of a 830k+ mailboxes database), and freezes. A combination of telemetry logs and truss output shows that the server records itself as having sent more data than the client receives, but truss'ing the client shows the client expectantly waiting in a read state. (The server continues to spin in a fstat/stat/fcntl/fcntl cycle on the mailboxes database, which as far as I can tell is normal behavior for the skiplist driver, but still looks really weird in a truss.) Now, here's where it gets even weirder: if I connect using mupdatetest and issue the same LIST command and let it run, the command runs to completion without error. However, if I at some point use flow control on my ssh session and hit ^S, then a ^Q, the scrolling continues briefly, and then the server hangs in a very similar way as above. To make things even odder, when I run a super-aggressive truss on the process (truss -aeflE -v all), the error never occurs. It's as if slowing down the mupdate process keeps it out of whatever error state it gets into. To make matters stranger, when I used the berkeley-hash driver on the MUPDATE mboxlist, the MUPDATE server fails to return anything from a LIST command, even when its database is full of matching entries. When ctl_mboxlist -m is run, an assert() fails and the process exits without performing any work. Because of all of this, I suspect something going wrong with a buffer filling up ungracefully somewhere. The spot I'm attacking right now is the 64-bit build -- I'm spending the weekend in the office rebuilding everything as 32 bit instead (libraries from the ground up), in case there's some problem with a different interpretation of size_t or some such thing in the 64-bit world. I'll share any findings in a few days, but I wanted to get this out earlier. We've eliminated hardware, OS, network, and compiler-specific errors by trying uploading the same database from numerous different clients to numerous different servers. (See the combinations tried below). I'm open to any and all suggestions at this point. Michael Bacon ITS Messaging UNC Chapel Hill Current system information: Hardware: Sun T5220s (Sparc CoolThreads architecture) running Solaris 10 Build: 64-bit binaries built using the Sun SPro compiler (to get CoolThreads optimizations) Configuration: tlscache, duplicate, and mboxlist_db all defined to skiplist Combinations tried: (backend client -> mupdate server) (all builds currently 64 bit 2.3.13) Sun 6800+Sol 9+gcc build -> Sun 5220+Sol 10+spro build Sun 6800+Sol 9+gcc build -> Sun 5120+Sol 10+spro build Sun 280R+Sol 9+gcc build -> Sun 5220+Sol 10+spro build Sun 280R+Sol 9+gcc build -> Same machine, separate cyrus install over localhost Sun 5220+Sol 10+spro build -> Sun 5220+Sol 10+spro build Sun 5220+Sol 10+spro build -> Sun 280R+Sol 9+gcc build We tried others too, but this covers most of the important combinations, I think. From mhlavink at redhat.com Mon Jun 15 05:23:18 2009 From: mhlavink at redhat.com (Michal Hlavinka) Date: Mon, 15 Jun 2009 11:23:18 +0200 Subject: Is mupdate_port not supported/broken? Message-ID: <200906151123.18697.mhlavink@redhat.com> Hi, we've tried to use mupdate_port in config file, but it seems it's not supported or at least doesn't work. We've specified mupdate_port, but it was still trying to connect to default port. When I look at source code, there is mupdate_port in configuration structure, but it seems it's not used anywhere. There seems to be only two lines in the whole sources ( grep -i -r mupdate_port cyrus-imapd/ ): cyrus-imapd-2.3.14/lib/imapopts.h: IMAPOPT_MUPDATE_PORT, ^^^ just an item in enum cyrus-imapd-2.3.14/lib/imapopts.c: { IMAPOPT_MUPDATE_PORT, "mupdate_port", 0, (union config_value)((long) 3905), OPT_INT, { { NULL, IMAP_ENUM_ZERO } } }, and mupdate-client.c contains: backend_connect(NULL, server, &mupdate_protocol,.... -> getaddrinfo(server, mupdate_protocol->service, &hints, &res0); where mupdate_protocol->service = "mupdate" set in mupdate-client.c, so this is reason default port for service is used. There is no assignment to mupdate_protocol->service which is used for getaddrinfo. So is this broken/not supported or I've just missed something? Regards, Michal Hlavinka From dave64 at andrew.cmu.edu Mon Jun 15 07:42:14 2009 From: dave64 at andrew.cmu.edu (Dave McMurtrie) Date: Mon, 15 Jun 2009 07:42:14 -0400 Subject: murder and autocreate (I know it is not supported) In-Reply-To: <18265F8D-0755-4BB2-8501-B02E80932EF6@umich.edu> References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> <18265F8D-0755-4BB2-8501-B02E80932EF6@umich.edu> Message-ID: <4A363396.1070201@andrew.cmu.edu> Wesley Craig wrote: > On 11 Jun 2009, at 18:59, Dave McMurtrie wrote: >> A valid point to mailbox creation, but what would delete the mailbox >> when a student graduates? > > There is no mailbox location problem for deletes. IDM simply connects > to any frontend, permits itself, and then deletes the mailbox hierarchy. Exactly. The point I was trying to make is that we already have a need for some system to be able to connect to our IMAP server for the purpose of deleting mailboxes, so having that same system connect to our IMAP server to create mailboxes seems to make perfect sense. Thanks, Dave -- Dave McMurtrie, SPE Email Systems Team Leader Carnegie Mellon University, Computing Services From baconm at email.unc.edu Mon Jun 15 10:07:34 2009 From: baconm at email.unc.edu (Michael Bacon) Date: Mon, 15 Jun 2009 10:07:34 -0400 Subject: Repeat recovers on databases Message-ID: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> This appears to be an issue in addition to the freeze-ups we're having. Given all the dumping and undumping I'm doing in the name of debugging, this may not be surprising, but I keep seeing instances where a database gets into some state wherein any process that opens it decides to run a recover on it before doing anything. Running a ctl_cyrusdb -r, even with all other processes stopped, does not seem to change this behavior. The next time a cyrus process starts up, whether it's an imapd, mupdate, or ctl_mboxlist, the process goes and does a recover before doing anything else. Has anyone else seen this? I've seen it on brand-new, newly "undumped" databases in the past week. Michael Bacon ITS Messaging UNC Chapel Hill From baconm at email.unc.edu Tue Jun 16 10:50:06 2009 From: baconm at email.unc.edu (Michael Bacon) Date: Tue, 16 Jun 2009 10:50:06 -0400 Subject: Cyrus in Solaris zones In-Reply-To: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> References: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> Message-ID: <3A9C8911AFBCA68657314A4E@dhcp00032.its.unc.edu> Okay, failing that, since I'm wondering if this is an issue with Solaris zones... Is anyone else running Cyrus inside of a non-global zone on Solaris? If so, have you run into any odd problems with it? Thanks, Michael Bacon ITS Messaging UNC Chapel Hill --On June 15, 2009 10:07:34 AM -0400 Michael Bacon wrote: > This appears to be an issue in addition to the freeze-ups we're having. > > Given all the dumping and undumping I'm doing in the name of debugging, > this may not be surprising, but I keep seeing instances where a database > gets into some state wherein any process that opens it decides to run a > recover on it before doing anything. Running a ctl_cyrusdb -r, even with > all other processes stopped, does not seem to change this behavior. The > next time a cyrus process starts up, whether it's an imapd, mupdate, or > ctl_mboxlist, the process goes and does a recover before doing anything > else. > > Has anyone else seen this? I've seen it on brand-new, newly "undumped" > databases in the past week. > > Michael Bacon > ITS Messaging > UNC Chapel Hill > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html From dick at nagual.nl Tue Jun 16 11:30:41 2009 From: dick at nagual.nl (dick hoogendijk) Date: Tue, 16 Jun 2009 17:30:41 +0200 Subject: Cyrus in Solaris zones In-Reply-To: <3A9C8911AFBCA68657314A4E@dhcp00032.its.unc.edu> References: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> <3A9C8911AFBCA68657314A4E@dhcp00032.its.unc.edu> Message-ID: <20090616173041.063bb433.dick@nagual.nl> On Tue, 16 Jun 2009 10:50:06 -0400 Michael Bacon wrote: > Okay, failing that, since I'm wondering if this is an issue with > Solaris zones... > > Is anyone else running Cyrus inside of a non-global zone on Solaris? > If so, have you run into any odd problems with it? I run cyrus inside a solaris 10 zone for quite some time now. Since U3 I think.. No issues whatsoever. -- Dick Hoogendijk -- PGP/GnuPG key: 01D2433D + http://nagual.nl/ | nevada / OpenSolaris 2009.06 release + All that's really worth doing is what we do for others (Lewis Carrol) From dbecker at cpicorp.com Tue Jun 16 12:08:23 2009 From: dbecker at cpicorp.com (Derek Chen-Becker) Date: Tue, 16 Jun 2009 11:08:23 -0500 Subject: Bizarre error with lmtpd Message-ID: <4A37C377.5060103@cpicorp.com> I'm working on installing Cyrus 2.3.14 and I've run into a weird issue. When postfix goes to deliver a message via lmtpunix, I get a segfault: Jun 12 16:06:30 ssmail lmtpunix[24401]: [ID 921384 local6.debug] accepted connection Jun 12 16:06:30 ssmail lmtpunix[24401]: [ID 685068 local6.debug] lmtp connection preauth'd as postman Jun 12 16:06:30 ssmail master[27855]: [ID 970914 local6.error] process 24401 exited, signaled to death by 11 Jun 12 16:06:30 ssmail master[27855]: [ID 621917 local6.debug] service lmtpunix pid 24401 in BUSY state: terminated abnormally But if I rebuild imap with "CFLAGS=-g" then it delivers normally: Jun 16 10:52:43 ssmail lmtpunix[6838]: [ID 518349 local6.debug] executed Jun 16 10:52:43 ssmail lmtpunix[6838]: [ID 921384 local6.debug] accepted connection Jun 16 10:52:43 ssmail lmtpunix[6838]: [ID 685068 local6.debug] lmtp connection preauth'd as postman Jun 16 10:52:43 ssmail lmtpunix[6838]: [ID 802986 local6.debug] IOERROR: fstating sieve script /var/imap/sieve/d/dbecker/defaultbc: No such file or directory Jun 16 10:52:43 ssmail lmtpunix[6838]: [ID 100061 local6.debug] duplicate_check: <4A37BC0E.5020409 at cpicorp.com> user.dbecker 0 Jun 16 10:52:44 ssmail last message repeated 1 time Jun 16 10:52:44 ssmail lmtpunix[6838]: [ID 964586 local6.info] Delivered: <4A37B C0E.5020409 at cpicorp.com> to mailbox: user.dbecker I had turned on the "-g" flag to try and debug the failure. Based on output from truss (this is Solaris 10), the segfault was coming when lmtpd tried to mmap and access the cyrus.header file for a given user. The cyrus.header file was correct, and even after I nuked the cyrus.header file and did a reconstruct it still crashed. I've done 4 builds now, two with "-g" and two without, and the behavior is consistent. Ideas? Thanks, Derek -- ---------------------------------------------------------------------- Derek Chen-Becker Senior Network Engineer, Security Architect CPI Corp, Inc. 1706 Washington Ave St. Louis, MO 63103 Phone: 314-231-7711 x6455 Fax: 314-613-6724 dbecker at cpicorp.com PGP Key available from public key servers Fingerprint: E4C4 26C0 8588 E80A C29F 636D 1FBE 0FE3 2871 4AE8 ---------------------------------------------------------------------- From simon.matter at invoca.ch Tue Jun 16 12:18:44 2009 From: simon.matter at invoca.ch (Simon Matter) Date: Tue, 16 Jun 2009 18:18:44 +0200 Subject: Bizarre error with lmtpd In-Reply-To: <4A37C377.5060103@cpicorp.com> References: <4A37C377.5060103@cpicorp.com> Message-ID: <16b95fae18f95bded94034531ec58cd3.squirrel@webmail.bi.corp.invoca.ch> > I'm working on installing Cyrus 2.3.14 and I've run into a weird issue. > When postfix goes to deliver a message via lmtpunix, I get a segfault: Please make sure you add this patch before any more debugging: http://github.com/brong/cyrus-imapd/commit/ec1bfcf6a1db9c86cbf55b9c25d7eb044dbbe51b#diff-0 Regards, Simon > > Jun 12 16:06:30 ssmail lmtpunix[24401]: [ID 921384 local6.debug] > accepted connection > Jun 12 16:06:30 ssmail lmtpunix[24401]: [ID 685068 local6.debug] lmtp > connection preauth'd as postman > Jun 12 16:06:30 ssmail master[27855]: [ID 970914 local6.error] process > 24401 exited, signaled to death by 11 > Jun 12 16:06:30 ssmail master[27855]: [ID 621917 local6.debug] service > lmtpunix pid 24401 in BUSY state: terminated abnormally > > But if I rebuild imap with "CFLAGS=-g" then it delivers normally: > > Jun 16 10:52:43 ssmail lmtpunix[6838]: [ID 518349 local6.debug] executed > Jun 16 10:52:43 ssmail lmtpunix[6838]: [ID 921384 local6.debug] accepted > connection > Jun 16 10:52:43 ssmail lmtpunix[6838]: [ID 685068 local6.debug] lmtp > connection preauth'd as postman > Jun 16 10:52:43 ssmail lmtpunix[6838]: [ID 802986 local6.debug] IOERROR: > fstating sieve script /var/imap/sieve/d/dbecker/defaultbc: No such file > or directory > Jun 16 10:52:43 ssmail lmtpunix[6838]: [ID 100061 local6.debug] > duplicate_check: <4A37BC0E.5020409 at cpicorp.com> user.dbecker > 0 > Jun 16 10:52:44 ssmail last message repeated 1 time > Jun 16 10:52:44 ssmail lmtpunix[6838]: [ID 964586 local6.info] > Delivered: <4A37B > C0E.5020409 at cpicorp.com> to mailbox: user.dbecker > > I had turned on the "-g" flag to try and debug the failure. Based on > output from truss (this is Solaris 10), the segfault was coming when > lmtpd tried to mmap and access the cyrus.header file for a given user. > The cyrus.header file was correct, and even after I nuked the > cyrus.header file and did a reconstruct it still crashed. I've done 4 > builds now, two with "-g" and two without, and the behavior is > consistent. Ideas? > > Thanks, > > Derek > > -- > ---------------------------------------------------------------------- > Derek Chen-Becker > Senior Network Engineer, Security Architect > CPI Corp, Inc. > 1706 Washington Ave > St. Louis, MO 63103 > Phone: 314-231-7711 x6455 > Fax: 314-613-6724 > dbecker at cpicorp.com > PGP Key available from public key servers > Fingerprint: E4C4 26C0 8588 E80A C29F 636D 1FBE 0FE3 2871 4AE8 > ---------------------------------------------------------------------- > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > From dbecker at cpicorp.com Tue Jun 16 13:08:31 2009 From: dbecker at cpicorp.com (Derek Chen-Becker) Date: Tue, 16 Jun 2009 12:08:31 -0500 Subject: Bizarre error with lmtpd In-Reply-To: <16b95fae18f95bded94034531ec58cd3.squirrel@webmail.bi.corp.invoca.ch> References: <4A37C377.5060103@cpicorp.com> <16b95fae18f95bded94034531ec58cd3.squirrel@webmail.bi.corp.invoca.ch> Message-ID: <4A37D18F.4040307@cpicorp.com> Simon Matter wrote: >> I'm working on installing Cyrus 2.3.14 and I've run into a weird issue. >> When postfix goes to deliver a message via lmtpunix, I get a segfault: > > Please make sure you add this patch before any more debugging: > http://github.com/brong/cyrus-imapd/commit/ec1bfcf6a1db9c86cbf55b9c25d7eb044dbbe51b#diff-0 Thanks! That fixed it. For what it's worth, I noticed that without specifying the "CFLAGS=-g", the CFLAGS var was actually being set to "CFLAGS=-g -O2", so what I was really doing was disabling optimization. Derek -- ---------------------------------------------------------------------- Derek Chen-Becker Senior Network Engineer, Security Architect CPI Corp, Inc. 1706 Washington Ave St. Louis, MO 63103 Phone: 314-231-7711 x6455 Fax: 314-613-6724 dbecker at cpicorp.com PGP Key available from public key servers Fingerprint: E4C4 26C0 8588 E80A C29F 636D 1FBE 0FE3 2871 4AE8 ---------------------------------------------------------------------- From arbatovevgeniy at gmail.com Wed Jun 17 08:51:49 2009 From: arbatovevgeniy at gmail.com (Evgeniy Arbatov) Date: Wed, 17 Jun 2009 15:51:49 +0300 Subject: Cyrus IMAP SASL authentication failure Message-ID: <56c989d50906170551y34b823bfrb63c78ae3af25d70@mail.gmail.com> Hello, I have a problem with Cyrus IMAP SASL authentication. When I try to login to create Cyrus IMAP mailboxes, I see the following: $ cyradm --user cyrus --auth login localhost IMAP Password: Login failed: generic failure at /usr/lib/perl5/Cyrus/IMAP/Admin.pm line 119 cyradm: cannot authenticate to server with login as cyrus This message appears even when I enter incorrect password. Can I be missing some of the packages (I am running Ubuntu 9.04)? I use sasldb authentication mechanism for saslauthd. Here is an extract from my imapd.conf file: admins: cyrus imap_admins: cyrus sasl_mech_list: LOGIN sasl_minimum_layer: 1 sasl_maximum_layer: 256 sasl_pwcheck_method: saslauthd This is what I see in my /var/log/mail.log cyrus/imap[7257]: badlogin: localhost [::1] plaintext cyrus SASL(-1): generic failure: checkpass failed If I try to telnet on imap port I have: Connected to localhost. Escape character is '^]'. * OK computer Cyrus IMAP4 v2.2.13-Debian-2.2.13-14ubuntu3 server ready imap login cyrus password imap NO Login failed: generic failure An interesting thing is that I have already setup Postfix sasldb authentication on the same host and it works fine. Any advice is much apreciated! Thank you in advance! Regards, Evgeniy -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090617/cad6aaff/attachment.html From vova at edu.yar.ru Wed Jun 17 09:02:27 2009 From: vova at edu.yar.ru (Vladimir Vassiliev) Date: Wed, 17 Jun 2009 17:02:27 +0400 Subject: Cyrus IMAP SASL authentication failure In-Reply-To: <56c989d50906170551y34b823bfrb63c78ae3af25d70@mail.gmail.com> References: <56c989d50906170551y34b823bfrb63c78ae3af25d70@mail.gmail.com> Message-ID: <200906171702.27712.vova@edu.yar.ru> > Here is an extract from my imapd.conf file: > > admins: cyrus > imap_admins: cyrus > sasl_mech_list: LOGIN > sasl_minimum_layer: 1 > sasl_maximum_layer: 256 > sasl_pwcheck_method: saslauthd Maybe it's because of sasl_minimum_layer: 1 LOGIN gives you no security layer. -- Vladimir Vassiliev From list at joreybump.com Wed Jun 17 09:12:51 2009 From: list at joreybump.com (Jorey Bump) Date: Wed, 17 Jun 2009 09:12:51 -0400 Subject: Cyrus IMAP SASL authentication failure In-Reply-To: <200906171702.27712.vova@edu.yar.ru> References: <56c989d50906170551y34b823bfrb63c78ae3af25d70@mail.gmail.com> <200906171702.27712.vova@edu.yar.ru> Message-ID: <4A38EBD3.3070502@joreybump.com> Vladimir Vassiliev wrote, at 06/17/2009 09:02 AM: >> Here is an extract from my imapd.conf file: >> >> admins: cyrus >> imap_admins: cyrus >> sasl_mech_list: LOGIN >> sasl_minimum_layer: 1 >> sasl_maximum_layer: 256 >> sasl_pwcheck_method: saslauthd > > Maybe it's because of sasl_minimum_layer: 1 > LOGIN gives you no security layer. > Indeed. Try: cyradm --user cyrus --auth login localhost -tls From baconm at email.unc.edu Wed Jun 17 09:44:16 2009 From: baconm at email.unc.edu (Michael Bacon) Date: Wed, 17 Jun 2009 09:44:16 -0400 Subject: MUPDATE database problems -- the importance of thread safety In-Reply-To: <00BFAADC8FC3B60673A4103D@dhcp00032.its.unc.edu> References: <00BFAADC8FC3B60673A4103D@dhcp00032.its.unc.edu> Message-ID: <74CFE7AD3EF3987A9FE96DAB@dhcp00032.its.unc.edu> It turns out that this was an issue with mupdate being a multi-threaded daemon, and in a critical place in the non-blocking prot code (in prot_flush_internal()), the behavior relies on the value of errno. If it's EAGAIN, the write will try again, otherwise it sets s->error and quits. Naturally, being a global variable normally, errno doesn't work terribly well in multi-threaded code unless the necessary thread safety switch is passed to the compiler. Hence, when thread #5 was getting a -1 from the write(2) system call, it was reading errno as 0, rather than EAGAIN as it should have been. The solution, should anyone else run into this, is as simple as recompiling with the thread safety switch. (In the case of Sun's SPro, it's -mt. I think it's -mthread for gcc, but I'm not sure.) Maddening that the fix was that simple, as I spent two solid weeks hunting for the dratted bug. I have two requests to the CVS maintainers out there. First, the below patch to current CVS isn't terribly comprehensive, and doesn't narrow it down from about a dozen places s->error could be set, but at least would have given SOME kind of indication on the server that something had gone wrong, and might have saved me about a week of hunting. Secondly, I am very weak in the ways of autoconf, but it strikes me that since Cyrus now builds mupdate as multithreaded by default (good decision, IMO), autoconf should make some attempt to figure out what thread safety switch is appropriate and add it to CFLAGS. Regards, Michael Bacon ITS Messaging UNC Chapel Hill --- prot.c 23 Apr 2009 17:10:07 -0000 1.97 +++ prot.c 17 Jun 2009 13:34:26 -0000 @@ -1038,6 +1038,8 @@ /* If we are exiting with an error, we should clear our memory buffer * and set our return code */ if(s->error) { + syslog(LOG_DEBUG, "prot_flush_internal: Error -- %s", s->error); + s->ptr = s->buf; s->cnt = s->maxplain; return EOF; --On June 13, 2009 4:22:03 PM -0400 Michael Bacon wrote: > Hello all, > > We're in the middle of trying to move from our single server installation > to a new murder installation on all new hardware. We're getting into the > late stages of setup, when we've run into a killer problem with getting > the old server to sync up with the MUPDATE server so that we can migrate > off of it. We're under a deadline to get the expensive new hardware > rolled out into production, so any help would be enormously appreciated. > > The test installation with a test backend of, oh, a couple dozen > mailboxes worked flawlessly. Syncing happened just as it was supposed > to, and everything looked good for production. The next step was to > start the old server syncing its database with the MUPDATE server, and > that's where we're stuck. > > The initial sync from the old backend works just fine. During the second > sync, however (ctl_mboxlist -m), the backend connects to the MUPDATE > server, executes a LIST , and then the server returns > somewhere between 2500-10,000 lines (of a 830k+ mailboxes database), and > freezes. A combination of telemetry logs and truss output shows that > the server records itself as having sent more data than the client > receives, but truss'ing the client shows the client expectantly waiting > in a read state. (The server continues to spin in a > fstat/stat/fcntl/fcntl cycle on the mailboxes database, which as far as > I can tell is normal behavior for the skiplist driver, but still looks > really weird in a truss.) > > Now, here's where it gets even weirder: if I connect using mupdatetest > and issue the same LIST command and let it run, the command runs to > completion without error. However, if I at some point use flow control > on my ssh session and hit ^S, then a ^Q, the scrolling continues > briefly, and then the server hangs in a very similar way as above. To > make things even odder, when I run a super-aggressive truss on the > process (truss -aeflE -v all), the error never occurs. It's as if > slowing down the mupdate process keeps it out of whatever error state it > gets into. > > To make matters stranger, when I used the berkeley-hash driver on the > MUPDATE mboxlist, the MUPDATE server fails to return anything from a LIST > command, even when its database is full of matching entries. When > ctl_mboxlist -m is run, an assert() fails and the process exits without > performing any work. > > Because of all of this, I suspect something going wrong with a buffer > filling up ungracefully somewhere. The spot I'm attacking right now is > the 64-bit build -- I'm spending the weekend in the office rebuilding > everything as 32 bit instead (libraries from the ground up), in case > there's some problem with a different interpretation of size_t or some > such thing in the 64-bit world. I'll share any findings in a few days, > but I wanted to get this out earlier. > > We've eliminated hardware, OS, network, and compiler-specific errors by > trying uploading the same database from numerous different clients to > numerous different servers. (See the combinations tried below). I'm > open to any and all suggestions at this point. > > Michael Bacon > ITS Messaging > UNC Chapel Hill > > > > Current system information: > Hardware: Sun T5220s (Sparc CoolThreads architecture) running Solaris 10 > Build: 64-bit binaries built using the Sun SPro compiler (to get > CoolThreads optimizations) > Configuration: tlscache, duplicate, and mboxlist_db all defined to > skiplist > > > Combinations tried: (backend client -> mupdate server) > (all builds currently 64 bit 2.3.13) > > Sun 6800+Sol 9+gcc build -> Sun 5220+Sol 10+spro build > Sun 6800+Sol 9+gcc build -> Sun 5120+Sol 10+spro build > Sun 280R+Sol 9+gcc build -> Sun 5220+Sol 10+spro build > Sun 280R+Sol 9+gcc build -> Same machine, separate cyrus install over > localhost > Sun 5220+Sol 10+spro build -> Sun 5220+Sol 10+spro build > Sun 5220+Sol 10+spro build -> Sun 280R+Sol 9+gcc build > We tried others too, but this covers most of the important combinations, > I think. > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html From schweizer.martin at gmail.com Wed Jun 17 11:29:50 2009 From: schweizer.martin at gmail.com (Martin Schweizer) Date: Wed, 17 Jun 2009 17:29:50 +0200 Subject: Fwd: Auth mech for sync_client/sync_server In-Reply-To: <380ccfd60906102242m41cb970bk4d52258efdab0f54@mail.gmail.com> References: <380ccfd60906102242m41cb970bk4d52258efdab0f54@mail.gmail.com> Message-ID: <380ccfd60906170829v67c3a610sf50c0b999ba72cd6@mail.gmail.com> Hello One week a go I sent the attached mail and never get an answer. Nobody has an idea? Regards, ---------- Forwarded message ---------- From: Martin Schweizer Date: 2009/6/11 Subject: Auth mech for sync_client/sync_server To: info-cyrus at lists.andrew.cmu.edu Hello I have two server which are up to date with Cyrus IMAP v2.3.14 on FreeBSD 7.1. I sync them with the following config options: Server2 (sync_client) /usr/local/etc/imapd.conf [snip] sync_host: server4 sync_authname: user sync_password: password sync_machineid: 1 sync_log: 1 sync_repeat_interval: 1 [snip] Server4 (sync_master) /usr/local/etc/imapd.conf [snip] sync_machineid: 2 sync_repeat_interval: 1 [snip] So far all works well. In /var/log/messages I see ?regulary the following message: Jun 11 07:15:20 server4 syncserver[79013]: login: server2 [192.168.20.3] cyrus DIGEST-MD5 User logged in This means that Server2 logs in Server4 and download the data. Can I change the mech DIGEST-MD5 in any way? Or has the parameter sasl_mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 in /usr/local/etc/imapd.conf any action to the sync_client/sync_server communication? I ask this because my clients do not longer need DIGEST-MD5. -- Martin Schweizer schweizer.martin at gmail.com Tel.: +41 32 512 48 54 (VoIP) Fax: +1 619 3300587 From wes at umich.edu Wed Jun 17 14:13:52 2009 From: wes at umich.edu (Wesley Craig) Date: Wed, 17 Jun 2009 14:13:52 -0400 Subject: Auth mech for sync_client/sync_server In-Reply-To: <380ccfd60906170829v67c3a610sf50c0b999ba72cd6@mail.gmail.com> References: <380ccfd60906102242m41cb970bk4d52258efdab0f54@mail.gmail.com> <380ccfd60906170829v67c3a610sf50c0b999ba72cd6@mail.gmail.com> Message-ID: On 17 Jun 2009, at 11:29, Martin Schweizer wrote: > Jun 11 07:15:20 server4 syncserver[79013]: login: server2 > [192.168.20.3] cyrus DIGEST-MD5 User logged in > > This means that Server2 logs in Server4 and download the data. Can I > change the mech DIGEST-MD5 in any way? Or has the parameter > > sasl_mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 > > in /usr/local/etc/imapd.conf any action to the sync_client/sync_server > communication? I ask this because my clients do not longer need > DIGEST-MD5. What do you want it to be? PLAIN and LOGIN aren't available via unencrypted channels unless you permit it. :wes From wes at umich.edu Wed Jun 17 14:16:44 2009 From: wes at umich.edu (Wesley Craig) Date: Wed, 17 Jun 2009 14:16:44 -0400 Subject: MUPDATE database problems -- the importance of thread safety In-Reply-To: <74CFE7AD3EF3987A9FE96DAB@dhcp00032.its.unc.edu> References: <00BFAADC8FC3B60673A4103D@dhcp00032.its.unc.edu> <74CFE7AD3EF3987A9FE96DAB@dhcp00032.its.unc.edu> Message-ID: Please open a report in bugzilla and mark it was a "blocker". Thanks for finding the issue. :wes On 17 Jun 2009, at 09:44, Michael Bacon wrote: > It turns out that this was an issue with mupdate being a multi- > threaded daemon, and in a critical place in the non-blocking prot > code (in prot_flush_internal()), the behavior relies on the value > of errno. If it's EAGAIN, the write will try again, otherwise it > sets s->error and quits. Naturally, being a global variable > normally, errno doesn't work terribly well in multi-threaded code > unless the necessary thread safety switch is passed to the > compiler. Hence, when thread #5 was getting a -1 from the write(2) > system call, it was reading errno as 0, rather than EAGAIN as it > should have been. > > The solution, should anyone else run into this, is as simple as > recompiling with the thread safety switch. (In the case of Sun's > SPro, it's -mt. I think it's -mthread for gcc, but I'm not sure.) > Maddening that the fix was that simple, as I spent two solid weeks > hunting for the dratted bug. > > I have two requests to the CVS maintainers out there. First, the > below patch to current CVS isn't terribly comprehensive, and > doesn't narrow it down from about a dozen places s->error could be > set, but at least would have given SOME kind of indication on the > server that something had gone wrong, and might have saved me about > a week of hunting. > > Secondly, I am very weak in the ways of autoconf, but it strikes me > that since Cyrus now builds mupdate as multithreaded by default > (good decision, IMO), autoconf should make some attempt to figure > out what thread safety switch is appropriate and add it to CFLAGS. From pegasus at nerv.eu.org Thu Jun 18 04:18:31 2009 From: pegasus at nerv.eu.org (Jure =?UTF-8?B?UGXEjWFy?=) Date: Thu, 18 Jun 2009 10:18:31 +0200 Subject: apple.mail and shared folders Message-ID: <20090618101831.71c00729.pegasus@nerv.eu.org> Hello, I think the problem with apple mail client not displaying shared folders in the subscription list is well known ... My question is: can something be done on the cyrus side to work around its problems? Google seems to have implemented something ... -- Jure Pe?ar http://jure.pecar.org From robm at fastmail.fm Thu Jun 18 05:36:52 2009 From: robm at fastmail.fm (Rob Mueller) Date: Thu, 18 Jun 2009 19:36:52 +1000 Subject: apple.mail and shared folders References: <20090618101831.71c00729.pegasus@nerv.eu.org> Message-ID: > I think the problem with apple mail client not displaying shared folders > in the subscription list is well known ... > > My question is: can something be done on the cyrus side to work around its > problems? Google seems to have implemented something ... Use the "altnamespace" option. You can even run a separate instance of imapd on a different port. http://blog.fastmail.fm/2008/08/11/alternate-namespace-imap-port-may-help-outlook-ol-express-apple-mail-and-bis-users/ Rob From arbatovevgeniy at gmail.com Thu Jun 18 05:42:52 2009 From: arbatovevgeniy at gmail.com (Evgeniy Arbatov) Date: Thu, 18 Jun 2009 12:42:52 +0300 Subject: Cyrus IMAP SASL authentication failure In-Reply-To: <4A38EBD3.3070502@joreybump.com> References: <56c989d50906170551y34b823bfrb63c78ae3af25d70@mail.gmail.com> <200906171702.27712.vova@edu.yar.ru> <4A38EBD3.3070502@joreybump.com> Message-ID: <56c989d50906180242w2d2c0885tdcf3aa731b83c083@mail.gmail.com> Thank you for your suggestions! I figured out what was the problem in my case. This was the OPTIONS setting in /etc/deafault/saslauthd. Since I run my Postfix chrooted I had: OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd" In order for cyradm to identify users using I saslauthd, I also added default OPTIONS setting to /etc/default/saslauthd file. So, my final settings look like this: OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd" OPTIONS="-c -m /var/run/saslauthd" Regards, Evgeniy On Wed, Jun 17, 2009 at 4:12 PM, Jorey Bump wrote: > Vladimir Vassiliev wrote, at 06/17/2009 09:02 AM: > >> Here is an extract from my imapd.conf file: > >> > >> admins: cyrus > >> imap_admins: cyrus > >> sasl_mech_list: LOGIN > >> sasl_minimum_layer: 1 > >> sasl_maximum_layer: 256 > >> sasl_pwcheck_method: saslauthd > > > > Maybe it's because of sasl_minimum_layer: 1 > > LOGIN gives you no security layer. > > > > Indeed. Try: > > cyradm --user cyrus --auth login localhost -tls > > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090618/24b26dde/attachment.html From schweizer.martin at gmail.com Thu Jun 18 08:12:32 2009 From: schweizer.martin at gmail.com (Martin Schweizer) Date: Thu, 18 Jun 2009 14:12:32 +0200 Subject: Auth mech for sync_client/sync_server In-Reply-To: References: <380ccfd60906102242m41cb970bk4d52258efdab0f54@mail.gmail.com> <380ccfd60906170829v67c3a610sf50c0b999ba72cd6@mail.gmail.com> Message-ID: <380ccfd60906180512n14f96e1cld0e93a765d16b250@mail.gmail.com> Hello Wes 2009/6/17 Wesley Craig : > On 17 Jun 2009, at 11:29, Martin Schweizer wrote: >> >> Jun 11 07:15:20 server4 syncserver[79013]: login: server2 >> [192.168.20.3] cyrus DIGEST-MD5 User logged in >> >> This means that Server2 logs in Server4 and download the data. Can I >> change the mech DIGEST-MD5 in any way? Or has the parameter >> >> sasl_mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 >> >> in /usr/local/etc/imapd.conf any action to the sync_client/sync_server >> communication? I ask this because my clients do not longer need >> DIGEST-MD5. > > What do you want it to be? ?PLAIN and LOGIN aren't available via unencrypted > channels unless you permit it. Does the line sasl_mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 any impact on the syncserver process? My goal is to reduce the mech list to absolut minimum which I need in my environment. Kind regards, -- Martin Schweizer schweizer.martin at gmail.com Tel.: +41 32 512 48 54 (VoIP) Fax: +1 619 3300587 From wes at umich.edu Thu Jun 18 13:06:06 2009 From: wes at umich.edu (Wesley Craig) Date: Thu, 18 Jun 2009 13:06:06 -0400 Subject: Auth mech for sync_client/sync_server In-Reply-To: <380ccfd60906180512n14f96e1cld0e93a765d16b250@mail.gmail.com> References: <380ccfd60906102242m41cb970bk4d52258efdab0f54@mail.gmail.com> <380ccfd60906170829v67c3a610sf50c0b999ba72cd6@mail.gmail.com> <380ccfd60906180512n14f96e1cld0e93a765d16b250@mail.gmail.com> Message-ID: <7916A6DF-813F-4C47-92E2-7D753A722619@umich.edu> On 18 Jun 2009, at 08:12, Martin Schweizer wrote: > Does the line sasl_mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 any > impact on the syncserver process? My goal is to reduce the mech list > to absolut minimum which I need in my environment. It ought to, yes. :wes From woods-cyrus at weird.com Thu Jun 18 14:29:01 2009 From: woods-cyrus at weird.com (Greg A. Woods) Date: Thu, 18 Jun 2009 14:29:01 -0400 Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> Message-ID: At Thu, 11 Jun 2009 17:37:34 -0700 (PDT), Andrew Morgan wrote: Subject: Re: murder and autocreate (I know it is not supported) > > >> Why make everything far more complicated than it needs to be? > >> Especially things related to user management? > >> > > > > A valid point to mailbox creation, but what would delete the mailbox > > when a student graduates? > > It is really quite trivial to write small scripts (perl, php, python, etc) > to manage Cyrus mailboxes. I don't know why folks do all this work by > hand... Who said anything about doing anything by hand? (or mailbox deletion, for that matter :-)) > I don't like the thought of Cyrus creating mailboxes on its own. One can > simply add mailbox creation to all the other steps of provisioning a new > account (creating an LDAP entry, making a home directory, setting quotas, > etc). Cyrus autocreate isn't creating mailboxes "on its own" -- it's creating them at the demand of, and under the guidance of, the MTA So, if something screwed up, as things inevitably do, even with all kinds of fancy special local script hacks that are supposed to be doing this mailbox creation, and the MTA is able to see that an account is valid and it should accept mail for it, but the screwup means that Cyrus doesn't have a mailbox waiting to receive the mail that the MTA just accepted on good authority of the authentication database. User management tools should NEVER _ever_ have anything to do with mailbox _creation_. The (modern) MTA _must_ validate the addresses. Since it already has to do this job the LDA really must just trust it, else the problem solved by the MTA's validation of addresses is effectively dissolved and broken. Therefore Cyrus _must_ create mailboxes automatically for addresses presented to it by the MTA. I suppose for the paranoid Cyrus could also validate the existence of the user account, but it's hardly necessary if your MTA/LDA/Cyrus implementation is secure. I'm really not sure why anyone would worry about Cyrus creating mailboxes. Things have worked this way for nearly forever in Unix systems. The mailer always creates mailboxes automatically for users who are known to exist. -- Greg A. Woods +1 416 218-0098 VE3TCP RoboHack Planix, Inc. Secrets of the Weird From woods-cyrus at weird.com Thu Jun 18 14:56:11 2009 From: woods-cyrus at weird.com (Greg A. Woods) Date: Thu, 18 Jun 2009 14:56:11 -0400 Subject: murder and autocreate (I know it is not supported) In-Reply-To: <4A363396.1070201@andrew.cmu.edu> References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> <18265F8D-0755-4BB2-8501-B02E80932EF6@umich.edu> <4A363396.1070201@andrew.cmu.edu> Message-ID: At Mon, 15 Jun 2009 07:42:14 -0400, Dave McMurtrie wrote: Subject: Re: murder and autocreate (I know it is not supported) > > Exactly. The point I was trying to make is that we already have a need > for some system to be able to connect to our IMAP server for the purpose > of deleting mailboxes, so having that same system connect to our IMAP > server to create mailboxes seems to make perfect sense. That's all fine and well for those who wish to automate deletion of mailboxes. Note that I'm not saying anything about preventing or removing the ability to manually or programmatically create mailboxes -- just that Cyrus in many (even most by numbers?) environments _must_ have the ability to automatically create mailboxes on demand if it's to be easily managed without having a vast majority of those installations also have to craft or find some additional custom mailbox management tool that's more likely to be an ugly hack than a secure and clean design. Even if you do have your user management system create mailboxes You still need to have your MTA validate addresses. And, unless you do mailbox creation before user creation, you could still end up with a window of vulnerability where the authentication database contains the user account and the MTA presents a message to Cyrus before there's a mailbox ready to receive it. Perhaps this will normally be a tiny window but I have actually seen an ISP account management system reliably generate the welcome message to a new user before it finished creating the new mailbox and thus the welcome message always bounced. Yes it was a stupid design, but that's what happens when the underlying systems are not first designed cleanly and elegantly. Not having a built-in basic way for Cyrus to automatically create mailboxes for valid users is also a poor design. Several good well-tested patches to enable this feature had been available for a very long time (years?) before the "murder" feature was added in the first place, and I have a firm conviction that if the autocreate feature had been added to Cyrus when it was first made available then the design of the clustering mechanisms would have supported it properly from the beginning too. I'm not even sure I understand the difficulty with it now -- if the cluster front-end knows how to direct a user access to the appropriate backend, then so can it direct an initial delivery for mailbox creation. -- Greg A. Woods +1 416 218-0098 VE3TCP RoboHack Planix, Inc. Secrets of the Weird From morgan at orst.edu Thu Jun 18 15:22:20 2009 From: morgan at orst.edu (Andrew Morgan) Date: Thu, 18 Jun 2009 12:22:20 -0700 (PDT) Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> Message-ID: On Thu, 18 Jun 2009, Greg A. Woods wrote: > Cyrus autocreate isn't creating mailboxes "on its own" -- it's creating > them at the demand of, and under the guidance of, the MTA > > So, if something screwed up, as things inevitably do, even with all > kinds of fancy special local script hacks that are supposed to be doing > this mailbox creation, and the MTA is able to see that an account is > valid and it should accept mail for it, but the screwup means that Cyrus > doesn't have a mailbox waiting to receive the mail that the MTA just > accepted on good authority of the authentication database. > > User management tools should NEVER _ever_ have anything to do with > mailbox _creation_. NEVER use absolutes! ;) > The (modern) MTA _must_ validate the addresses. Since it already has to > do this job the LDA really must just trust it, else the problem solved > by the MTA's validation of addresses is effectively dissolved and broken. > > Therefore Cyrus _must_ create mailboxes automatically for addresses > presented to it by the MTA. I suppose for the paranoid Cyrus could also > validate the existence of the user account, but it's hardly necessary if > your MTA/LDA/Cyrus implementation is secure. > > I'm really not sure why anyone would worry about Cyrus creating > mailboxes. Things have worked this way for nearly forever in Unix > systems. The mailer always creates mailboxes automatically for users > who are known to exist. Unix systems don't automatically create accounts when someone tries to login... Nor do they automatically create home directories for an existing user, or set disk quotas, or populate the home directory with skeleton shell init scripts. That is the job of the user management tool ("useradd" on many systems). In our case, the list of valid email addresses on our MTA is generated from the list of mailboxes in Cyrus. Obviously there are multiple ways to do account/mailbox creation. I have no problem if you want to let Cyrus create mailboxes automatically. I prefer to have more direct control, but to each their own. Andy From morgan at orst.edu Thu Jun 18 15:36:07 2009 From: morgan at orst.edu (Andrew Morgan) Date: Thu, 18 Jun 2009 12:36:07 -0700 (PDT) Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> <18265F8D-0755-4BB2-8501-B02E80932EF6@umich.edu> <4A363396.1070201@andrew.cmu.edu> Message-ID: On Thu, 18 Jun 2009, Greg A. Woods wrote: > Not having a built-in basic way for Cyrus to automatically create > mailboxes for valid users is also a poor design. Several good > well-tested patches to enable this feature had been available for a very > long time (years?) before the "murder" feature was added in the first > place, and I have a firm conviction that if the autocreate feature had > been added to Cyrus when it was first made available then the design of > the clustering mechanisms would have supported it properly from the > beginning too. I'm not even sure I understand the difficulty with it > now -- if the cluster front-end knows how to direct a user access to the > appropriate backend, then so can it direct an initial delivery for > mailbox creation. When the mailbox already exists, the murder frontend looks up which backend contains the mailbox and proxies requests to it. In the case of new mailbox creation, something must decide which backend/partition to use. However, the frontends don't even know which backends/partitions exist. Nothing in the cyrus configuration on the frontend lists the available backends/partitions. The mailboxes.db file contains a list of backends/partitions, in a way, because it lists all mailboxes. Here is an example entry: user.morgan 1 cyrus-be2.onid.oregonstate.edu!default morgan lrswipcda This says that my mailbox exists on the backend "cyrus-be2.onid.oregonstate.edu" in the "default" partition. I suppose the frontend could generate a list of all backends and partitions from the mailboxes.db, but when you add a new backend or partition, it wouldn't exist there either. I'm trying to think of a way the frontend could determine which backends/partitions exist, but I don't think it can currently. Let's assume it could though... How would it pick which backend/partition to create the mailbox on? At random? Andy From duncan.gibb at siriusit.co.uk Thu Jun 18 17:22:31 2009 From: duncan.gibb at siriusit.co.uk (Duncan Gibb) Date: Thu, 18 Jun 2009 22:22:31 +0100 Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> Message-ID: <4A3AB017.6000400@siriusit.co.uk> Andrew Morgan wrote: GAW> User management tools should NEVER _ever_ have anything to GAW> do with mailbox _creation_. AM> NEVER use absolutes! ;) Let's talk about "common deployment scenarios", then ;-) GAW> The (modern) MTA _must_ validate the addresses. AM> Unix systems don't automatically create accounts when someone AM> tries to login... Depends what you mean by "create". If the Unix box is hooked up to nss_ldap and pam_ldap, then the login will work as soon as the LDAP account is active - without anyone or anything touching the Unix box itself. Autocreate makes Cyrus equally easy to manage, which is why so many people would like it or an equivalent bit of functionality in the main tree. AM> Nor do they automatically create home directories for an AM> existing user, or set disk quotas, or populate the home AM> directory with skeleton shell init scripts. That is the AM> job of the user management tool ("useradd" on many systems). Or if the sysadmin prefers, it's the job of pam_mkhomedir (and pam_quota if you're using FreeBSD), which may well be in the stack with pam_ldap. AM> I have no problem if you want to let Cyrus create mailboxes AM> automatically. I prefer to have more direct control, but to AM> each their own. I don't think anyone wants autocreate to be _compulsory_ - or even enabled by default - just available without having to maintain a medium-sized external patch. Cheers Duncan -- Duncan Gibb - Technical Director Sirius Corporation plc - control through freedom http://www.siriusit.co.uk/ || t: +44 870 608 0063 Debian Cyrus Team - https://alioth.debian.org/projects/pkg-cyrus-imapd/ From duncan.gibb at siriusit.co.uk Thu Jun 18 17:22:38 2009 From: duncan.gibb at siriusit.co.uk (Duncan Gibb) Date: Thu, 18 Jun 2009 22:22:38 +0100 Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> <18265F8D-0755-4BB2-8501-B02E80932EF6@umich.edu> <4A363396.1070201@andrew.cmu.edu> Message-ID: <4A3AB01E.8030200@siriusit.co.uk> Andrew Morgan wrote: AM> In the case of new mailbox creation, something must decide AM> which backend/partition to use. AM> I'm trying to think of a way the frontend could determine AM> which backends/partitions exist, but I don't think it can AM> currently. AM> Let's assume it could though... How would it pick which AM> backend/partition to create the mailbox on? At random? That is indeed the question that needs to be answered, and I think you're right that it's too complex and too deployment-specific a problem to be solved inside of Cyrus. The patch I posted earlier in this thread: http://lists.andrew.cmu.edu/pipermail/info-cyrus/2009-June/031173.html is a rough-and-ready means for the frontend to farm the decision out to something that _is_ competent to decide. In my world, that's pretty much always a perl script which interrogates OpenLDAP, but other people's scenarios may differ ;-) Cheers Duncan -- Duncan Gibb - Technical Director Sirius Corporation plc - control through freedom http://www.siriusit.co.uk/ || t: +44 870 608 0063 Debian Cyrus Team - https://alioth.debian.org/projects/pkg-cyrus-imapd/ From baconm at email.unc.edu Thu Jun 18 17:44:19 2009 From: baconm at email.unc.edu (Michael Bacon) Date: Thu, 18 Jun 2009 17:44:19 -0400 Subject: Repeat recovers on databases In-Reply-To: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> References: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> Message-ID: <0E5A945BF34FF898C9F1E390@dhcp00032.its.unc.edu> Another one stomped here. This time, it's a 32/64 bit issue. myinit in cyrusdb_skiplist.c assumes that type_t is 4 bytes long, and writes out that many from the current timestamp when creating $confdir/db/skipstamp. On 64-bit Solaris, time_t is 8 bytes (it's typedef'ed as a long). I'm forgetting my Who's Who of big and little endian chips, but my guess is that on x86 systems, the first four bytes are the ones with the real data in them, so there's actually meaningful data that gets written out. On Sparc, though, no such luck. So, when ctl_cyrusdb decides to recover the database, it writes out four bytes of data, all of which happen to be zeroes. Henceforth, every process that looks at the database goes, "oh, look, the database needs recovering!" then spends 55 seconds recovering it before it does any meaningful work, then proceeds to write out 4 bytes of zeroes into the skipstamp file. The next process comes along, reads the skipstamp file, and goes, "oh, look, the database needs recovering!" The fix for it is below. I will also open a bugzilla issue for this. Always remember boys and girls, when you ASS-UM-E the bit size of types, you make lots of ASSemblers go "UM...." exponentially. Michael Bacon ITS Messaging UNC Chapel Hill =================================================================== RCS file: /cvs/src/cyrus/lib/cyrusdb_skiplist.c,v retrieving revision 1.64 diff -u -r1.64 cyrusdb_skiplist.c --- cyrusdb_skiplist.c 8 Oct 2008 15:47:08 -0000 1.64 +++ cyrusdb_skiplist.c 18 Jun 2009 21:42:30 -0000 @@ -239,7 +239,7 @@ if (r != -1) r = ftruncate(fd, 0); a = htonl(global_recovery); - if (r != -1) r = write(fd, &a, 4); + if (r != -1) r = write(fd, &a, sizeof(time_t)); if (r != -1) r = close(fd); if (r == -1) { @@ -252,7 +252,7 @@ fd = open(sfile, O_RDONLY, 0644); if (fd == -1) r = -1; - if (r != -1) r = read(fd, &a, 4); + if (r != -1) r = read(fd, &a, sizeof(time_t)); if (r != -1) r = close(fd); if (r == -1) { --On June 15, 2009 10:07:34 AM -0400 Michael Bacon wrote: > This appears to be an issue in addition to the freeze-ups we're having. > > Given all the dumping and undumping I'm doing in the name of debugging, > this may not be surprising, but I keep seeing instances where a database > gets into some state wherein any process that opens it decides to run a > recover on it before doing anything. Running a ctl_cyrusdb -r, even with > all other processes stopped, does not seem to change this behavior. The > next time a cyrus process starts up, whether it's an imapd, mupdate, or > ctl_mboxlist, the process goes and does a recover before doing anything > else. > > Has anyone else seen this? I've seen it on brand-new, newly "undumped" > databases in the past week. > > Michael Bacon > ITS Messaging > UNC Chapel Hill > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html From brong at fastmail.fm Thu Jun 18 19:47:53 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Fri, 19 Jun 2009 09:47:53 +1000 Subject: Repeat recovers on databases In-Reply-To: <0E5A945BF34FF898C9F1E390@dhcp00032.its.unc.edu> References: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> <0E5A945BF34FF898C9F1E390@dhcp00032.its.unc.edu> Message-ID: <20090618234753.GA4196@brong.net> On Thu, Jun 18, 2009 at 05:44:19PM -0400, Michael Bacon wrote: > Another one stomped here. This time, it's a 32/64 bit issue. myinit in > cyrusdb_skiplist.c assumes that type_t is 4 bytes long, and writes out > that many from the current timestamp when creating $confdir/db/skipstamp. > On 64-bit Solaris, time_t is 8 bytes (it's typedef'ed as a long). I'm > forgetting my Who's Who of big and little endian chips, but my guess is > that on x86 systems, the first four bytes are the ones with the real data > in them, so there's actually meaningful data that gets written out. On > Sparc, though, no such luck. Er, yeah. Ouch. Damn. I want to make it an 8 bit value, but that would be an incompatible format change to skiplists. At which time I would do a bunch of other stuff too. I do have a cyrusdb_skiplist2.c file floating around somewhere that does it (checksums for one thing). I was even thinking of doing something really evil with ordering on checkpoint, but I never got around to running the numbers to see if it made point. Basically instead of: level: 1 2 1 3 2 1 1 2 key : aaa bbb ccc ddd eee fff ggg hhh It would lay the records out like this: level: 3 2 2 2 1 1 1 1 key : ddd bbb eee hhh aaa ccc fff ggg The advantage being that for a lookup, the "next record" at the same level would be directly after the current one, so readahead would be more likely to hit the next node for the search case. It would be a fair bit more random for enumerating though, so I don't know if it's really sane (and of course as you make changes, it all gets more random until the next checkpoint anyway) So anyway, will definitely fix the immediate issue! Thanks, Bron. From brong at fastmail.fm Thu Jun 18 20:09:16 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Fri, 19 Jun 2009 10:09:16 +1000 Subject: Repeat recovers on databases In-Reply-To: <0E5A945BF34FF898C9F1E390@dhcp00032.its.unc.edu> References: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> <0E5A945BF34FF898C9F1E390@dhcp00032.its.unc.edu> Message-ID: <20090619000916.GB5674@brong.net> On Thu, Jun 18, 2009 at 05:44:19PM -0400, Michael Bacon wrote: > The fix for it is below. I will also open a bugzilla issue for this. I think this is actually a better fix that keeps things in the right type on to disk. Can you please test it on your platform. Thanks, Bron. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Use-correctly-sized-variable-for-recovery-time.patch Type: text/x-diff Size: 2927 bytes Desc: not available Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090619/1465c6bf/attachment.bin From brong at fastmail.fm Thu Jun 18 19:57:03 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Fri, 19 Jun 2009 09:57:03 +1000 Subject: Repeat recovers on databases In-Reply-To: <0E5A945BF34FF898C9F1E390@dhcp00032.its.unc.edu> References: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> <0E5A945BF34FF898C9F1E390@dhcp00032.its.unc.edu> Message-ID: <20090618235703.GA5674@brong.net> On Thu, Jun 18, 2009 at 05:44:19PM -0400, Michael Bacon wrote: > Another one stomped here. This time, it's a 32/64 bit issue. myinit in > cyrusdb_skiplist.c assumes that type_t is 4 bytes long, and writes out > that many from the current timestamp when creating $confdir/db/skipstamp. Actually, reading the code, that's not strictly true: > a = htonl(global_recovery); > - if (r != -1) r = write(fd, &a, 4); > + if (r != -1) r = write(fd, &a, sizeof(time_t)); It writes "a", which is the result of calling htonl on global_recovery. If htonl isn't returning a 32 bit value of the lower order bytes of the value that it's given, then this bug is going to be causing a LOT more problems than just this. We assume this works in quite a few other places in the code, including the timestamp value in the skiplist header itself, and in places throughout the mailbox code too. "htonl" => "host to net long" by my reading. There's also htonll for 64 bit values. Is your platform creating net longlongs? time_t a; There's the actual bug. That should be bit32 a; Bron. From gombasg at sztaki.hu Fri Jun 19 00:47:14 2009 From: gombasg at sztaki.hu (Gabor Gombas) Date: Fri, 19 Jun 2009 06:47:14 +0200 Subject: Repeat recovers on databases In-Reply-To: <20090619000916.GB5674@brong.net> References: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> <0E5A945BF34FF898C9F1E390@dhcp00032.its.unc.edu> <20090619000916.GB5674@brong.net> Message-ID: <20090619044714.GA2143@boogie.lpds.sztaki.hu> On Fri, Jun 19, 2009 at 10:09:16AM +1000, Bron Gondwana wrote: > @@ -192,6 +192,18 @@ struct db_list { > static time_t global_recovery = 0; > static struct db_list *open_db = NULL; > > +#define BIT32_MAX 4294967295U > + > +#if UINT_MAX == BIT32_MAX > +typedef unsigned int bit32; > +#elif ULONG_MAX == BIT32_MAX > +typedef unsigned long bit32; > +#elif USHRT_MAX == BIT32_MAX > +typedef unsigned short bit32; > +#else > +#error dont know what to use for bit32 > +#endif > + If you're touching this code, why not use standard stdint.h types like uint32_t here? Gabor -- --------------------------------------------------------- MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences --------------------------------------------------------- From brong at fastmail.fm Fri Jun 19 01:46:41 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Fri, 19 Jun 2009 15:46:41 +1000 Subject: Repeat recovers on databases In-Reply-To: <20090619044714.GA2143@boogie.lpds.sztaki.hu> References: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu><0E5A945BF34FF898C9F1E390@dhcp00032.its.unc.edu><20090619000916.GB5674@brong.net> <20090619044714.GA2143@boogie.lpds.sztaki.hu> Message-ID: <1245390401.8807.1321145869@webmail.messagingengine.com> On Fri, 19 Jun 2009 06:47 +0200, "Gabor Gombas" wrote: > On Fri, Jun 19, 2009 at 10:09:16AM +1000, Bron Gondwana wrote: > > > @@ -192,6 +192,18 @@ struct db_list { > > static time_t global_recovery = 0; > > static struct db_list *open_db = NULL; > > > > +#define BIT32_MAX 4294967295U > > + > > +#if UINT_MAX == BIT32_MAX > > +typedef unsigned int bit32; > > +#elif ULONG_MAX == BIT32_MAX > > +typedef unsigned long bit32; > > +#elif USHRT_MAX == BIT32_MAX > > +typedef unsigned short bit32; > > +#else > > +#error dont know what to use for bit32 > > +#endif > > + > > If you're touching this code, why not use standard stdint.h types like > uint32_t here? Yeah, I was just thinking that actually. Mostly because, well - that's what's already there! I'll do a stdint rewrite of it all some time soon :) Bron. -- Bron Gondwana brong at fastmail.fm From arisg at noc.uoa.gr Fri Jun 19 07:29:06 2009 From: arisg at noc.uoa.gr (Aristotelis) Date: Fri, 19 Jun 2009 14:29:06 +0300 Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> <18265F8D-0755-4BB2-8501-B02E80932EF6@umich.edu> <4A363396.1070201@andrew.cmu.edu> Message-ID: <4A3B7682.3060508@noc.uoa.gr> Hello all, It is an interesting discussion about the autocreate feature in cyrus imapd. I would like to point you to a rather old discussion (but still very relevant) that was on cyrus-devel: http://marc.info/?l=cyrus-devel&m=109206301904447&w=2 It outlines some design choises and some evaluation of these. I still believe that the best way to tackle the issue would be the mupdate server that can have an overview of the whole process, but in order for this to happen a change in the MUPDATE protocol must be made, to add the extra functionality. So perhaps people whould have a look also on this thread. While having a look in the issue a function was created that was running on proxyd (no unified systems) that was going through the list of the available backend servers (the list was given as a configuration option) and picking one randomly. Then it was making a connection back to the backend server, and there the autocreate patch was kicking in and creating the folder. This solution had some shortcomings, so it wasn't released. (and since we don't use a murder environment it couldn't be properly tested). Best regards, Aristotelis From baconm at email.unc.edu Fri Jun 19 09:40:17 2009 From: baconm at email.unc.edu (Michael Bacon) Date: Fri, 19 Jun 2009 09:40:17 -0400 Subject: Repeat recovers on databases In-Reply-To: <20090619000916.GB5674@brong.net> References: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> <0E5A945BF34FF898C9F1E390@dhcp00032.its.unc.edu> <20090619000916.GB5674@brong.net> Message-ID: Right, right, I suppose changing database formats is somehow "bad..." :) This fix also works -- thanks. -Michael --On June 19, 2009 10:09:16 AM +1000 Bron Gondwana wrote: > On Thu, Jun 18, 2009 at 05:44:19PM -0400, Michael Bacon wrote: >> The fix for it is below. I will also open a bugzilla issue for this. > > I think this is actually a better fix that keeps things in the > right type on to disk. Can you please test it on your platform. > > Thanks, > > Bron. From nodens2099 at gmail.com Fri Jun 19 13:27:52 2009 From: nodens2099 at gmail.com (nodens2099) Date: Fri, 19 Jun 2009 19:27:52 +0200 Subject: murder and autocreate (I know it is not supported) In-Reply-To: <4A3B7682.3060508@noc.uoa.gr> References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> <18265F8D-0755-4BB2-8501-B02E80932EF6@umich.edu> <4A363396.1070201@andrew.cmu.edu> <4A3B7682.3060508@noc.uoa.gr> Message-ID: <4A3BCA98.1060002@gmail.com> Aristotelis a ?crit : > > While having a look in the issue a function was created that was running > on proxyd (no unified systems) that was going through the list of the > available backend servers (the list was given as a configuration option) > and picking one randomly. Then it was making a connection back to the > backend server, and there the autocreate patch was kicking in and > creating the folder. This solution had some shortcomings, so it wasn't > released. (and since we don't use a murder environment it couldn't be > properly tested). > How about supporting using adding a hostname in the "defaultpartition:" configuration value on the murder server ? Something like "hostname:default", the same as the renamemailbox command would make sense. Ok, it wouldn't give a way to say "such domain on this backend, such domaine on the other one" but it would be a start (just change this value when the backend is full), and better than random picking. Regards, -- Cl?ment Hermann From dave64 at andrew.cmu.edu Fri Jun 19 14:12:34 2009 From: dave64 at andrew.cmu.edu (Dave McMurtrie) Date: Fri, 19 Jun 2009 14:12:34 -0400 Subject: improved_mboxlist_sort option Message-ID: <4A3BD512.2090006@andrew.cmu.edu> Hi, Historically, the mailbox sorting code in Cyrus used ascii to sort, which would occasionally cause odd problems with mailboxes containing non-alphabetic characters. Not too long ago, Ken added some code to deal with this problem but he didn't want to force it on everyone and risk breaking a ton of stuff, so he made it a configurable option. If you set "improved_mboxlist_sort: 1" it uses Ken's improved sorting code. I'm curious whether anyone is running this in production. Thanks, Dave -- Dave McMurtrie, SPE Email Systems Team Leader Carnegie Mellon University, Computing Services From woods-cyrus at weird.com Fri Jun 19 14:24:29 2009 From: woods-cyrus at weird.com (Greg A. Woods) Date: Fri, 19 Jun 2009 14:24:29 -0400 Subject: murder and autocreate (I know it is not supported) In-Reply-To: References: <4A292E74.2080304@as2594.net> <4A2F6689.4000309@as2594.net> <4A2F9B37.3040009@andrew.cmu.edu> Message-ID: At Thu, 18 Jun 2009 12:22:20 -0700 (PDT), Andrew Morgan wrote: Subject: Re: murder and autocreate (I know it is not supported) > > Unix systems don't automatically create accounts when someone tries to > login... But unix mailers do automatically create mailboxes for existing users whenever those mailboxes need to be created... Also, as you allude to, there's a reason why modern systems include a properly integrated, built-in to the fundamental base system, method of doing all the things needed to create a usable user account. Unfortunately Cyrus is not a fully integrated subsystem in any existing modern base system, so the base system useradd or equivalent cannot automatically initialize Cyrus mailboxes too. (I'm sure I'm also not alone in thinking that many folks wouldn't mind if there were a standard way to do all the other mundane parts of account setup upon first login -- it would obviously be easy enough to do by simply setting the initial default shell to some script that did all the right things. Perhaps it's already been done. Maybe I'll do it myself in my own custom NetBSD releases now that you've prompted me to think about it again. Dim memories even suggest it has already been done -- I seem to remember being asked which shell I wanted to use when I first logged onto some systems long ago.) > In our case, the list of valid email addresses on our MTA is generated > from the list of mailboxes in Cyrus. I knew someone would end up doing it "backwards" -- it was inevitable! :-) (Your way is of course one good way to completely uncouple the mail system AAA from the underlying OS AAA when using something like Cyrus for mailbox management and storage. I prefer not to do that, but with Cyrus it's already half done anyway and there's no option to undo it.) > Obviously there are multiple ways to do account/mailbox creation. I have > no problem if you want to let Cyrus create mailboxes automatically. I > prefer to have more direct control, but to each their own. Exactly -- we agree entirely. There's nothing wrong with having administrative control over when and how mailboxes are created, but at the same time I think it is also a fundamental requirement in many environments that the mail system automatically honour all valid existing accounts even if that means automatically creating mailboxes (and perhaps also setting default quota) when necessary. -- Greg A. Woods +1 416 218-0098 VE3TCP RoboHack Planix, Inc. Secrets of the Weird From baconm at email.unc.edu Fri Jun 19 15:43:43 2009 From: baconm at email.unc.edu (Michael Bacon) Date: Fri, 19 Jun 2009 15:43:43 -0400 Subject: Repeat recovers on databases In-Reply-To: <20090618235703.GA5674@brong.net> References: <272BC2FE0F18300DCCDCDA04@dhcp00032.its.unc.edu> <0E5A945BF34FF898C9F1E390@dhcp00032.its.unc.edu> <20090618235703.GA5674@brong.net> Message-ID: <12A1FE1EA0F010454F35FE96@dhcp00032.its.unc.edu> --On June 19, 2009 9:57:03 AM +1000 Bron Gondwana wrote: > On Thu, Jun 18, 2009 at 05:44:19PM -0400, Michael Bacon wrote: >> Another one stomped here. This time, it's a 32/64 bit issue. myinit in >> cyrusdb_skiplist.c assumes that type_t is 4 bytes long, and writes out >> that many from the current timestamp when creating >> $confdir/db/skipstamp. > > Actually, reading the code, that's not strictly true: > >> a = htonl(global_recovery); >> - if (r != -1) r = write(fd, &a, 4); >> + if (r != -1) r = write(fd, &a, sizeof(time_t)); > > It writes "a", which is the result of calling htonl on global_recovery. > > If htonl isn't returning a 32 bit value of the lower order bytes of the > value that it's given, then this bug is going to be causing a LOT more > problems than just this. We assume this works in quite a few other > places in the code, including the timestamp value in the skiplist header > itself, and in places throughout the mailbox code too. > > "htonl" => "host to net long" by my reading. There's also htonll for 64 > bit values. Is your platform creating net longlongs? Good question -- this may be a Solaris bug after all. Solaris clearly defines in the man page that htonl is supposed to return a uint32_t from htonl, but looking at sys/byteorder.h, that's um, not being enforced... #if defined(_BIG_ENDIAN) && !defined(ntohl) && !defined(__lint) /* big-endian */ #define ntohl(x) (x) #define ntohs(x) (x) #define htonl(x) (x) #define htons(x) (x) #elif !defined(ntohl) /* little-endian */ I think I may give our friends out in CA a call here... -Michael From freggy at gmail.com Mon Jun 22 15:06:37 2009 From: freggy at gmail.com (Frederik) Date: Mon, 22 Jun 2009 21:06:37 +0200 Subject: cyradm --authz --user selecting wrong mailbox on Mac OS X Message-ID: <28d495d10906221206s74359c10m4ea8ea8d9dedf7fd@mail.gmail.com> I run this command on a Mac OS X 10.5 server running cyrus (version 2.3.8 IIRC): /usr/bin/cyrus/admin/cyradm -user cyradm -authz user -auth plain localhost This should select the mailbox for "user" and not the one from cyradm, right? Because this does not seem to work on this system: lm shows the mailboxes for cyradm and the mail folders for user are visible in "Other Users/". Of course I have a line admins: cyradm in imapd.conf. What could be wrong? Does not this work in Cyrus as included in Mac OS X 10.5 server? -- Frederik From Valery.Brasseur at atosorigin.com Tue Jun 23 04:30:49 2009 From: Valery.Brasseur at atosorigin.com (=?utf-8?B?QnJhc3NldXIgVmFsw6lyeQ==?=) Date: Tue, 23 Jun 2009 10:30:49 +0200 Subject: error in localdelete Message-ID: I have a murder configuration, where I try to move some user with xfer. the xfer command hang with the result : " Syntax error in parameters" what's wrong ? here is the stack trace : #0 0xbfffe402 in ?? () #1 0xb7cf3b73 in __read_nocancel () from /lib/libc.so.6 #2 0x080b1b55 in prot_fill (s=0x8244940) at prot.c:471 #3 0x080b2c9b in prot_fgets (buf=0xbfd19f40 "D01 NO Syntax error in parameters\r\n", size=4095, s=0x8244940) at prot.c:1188 #4 0x0805b64b in getresult (p=0x8244940, tag=0x8174c56 "LD1") at imapd.c:7889 #5 0x0805bcfe in do_xfer_single (toserver=0x823fe18 "dev-cyrus-agence.lclmail.priv.atos.fr", topart=0x8174c3a "LD1 LOCALDELETE {%d+}\r\n%s\r\n", name=0xbfd1b420 "user.99999.AGENT_A at agence.lcl.fr.cly", mailboxname=0xbfd1b650 "agence.xxx.xxx.cly!user.99999.AGENT_A", mbflags=1, path=0xfffffe00
, mpath=0xfffffe00
, part=0x8247dc8 "dev-cyrus.xxx.priv.xxxx!parta", acl=0x8246bc8 "99999 at agence.xxx.xxx.cly\tlrswipkxtecda\tcyrusadmin\tlrswipkxtecda\tanyone\tp\t", prereserved=0, h_in=0x8240aa0, be_in=0x82444b8) at imapd.c:8301 #6 0x0805de83 in xfer_user_cb (name=0xbfd1b650 "agence.xxx.xxx.cly!user.99999.yyyy", matchlen=36, maycreate=1, rock=0xfffffe00) at imapd.c:8371 #7 0x08082f1c in find_cb (rockp=0xbfd1bd00, key=0xb688a498 "agence.xxx.xxx.cly!user.99999.yyyy", keylen=36, data=0xb688a4c0 "1 dev-cyrus.xxx.priv.xxxx!parta 99999 at agence.lcl.fr.cly\tlrswipkxtecda\tcyrusadmin\tlrswipkxtecda\tanyone\tp\t", datalen=111) at mboxlist.c:2006 #8 0x080b98f6 in myforeach (db=0x8237670, prefix=0xbfd1b920 "agence.xxx.xxx.cly!user.99999.*", prefixlen=29, goodp=0x8082b00 , cb=0x8082e20 , rock=0xbfd1bd00, tid=0x0) at cyrusdb_skiplist.c:998 #9 0x0808350b in mboxlist_findall (namespace=0x0, pattern=0xbfd1bfe0 "agence.xxx.xxx.cly!user.99999.*", isadmin=-512, userid=0x8082b00 "U\211?\201?(\002", auth_state=0xfffffe00, proc=0x805dcc0 , rock=0xbfd1bdd0) at mboxlist.c:2198 #10 0x0805ce4b in cmd_xfer (tag=0x823fb88 ".", name=0x823fc68 "user.99999 at agence.lcl.fr.cly", toserver=0x823fe18 " dev-cyrus.xxx.priv.xxxx", topart=0x0) at imapd.c:8522 #11 0x08066fbe in cmdloop () at imapd.c:1869 #12 0x080674d6 in service_main (argc=1, argv=0x8235008, envp=0xbfd1e6c4) at imapd.c:797 thanks -- The least expensive, most bug-free line of code is the one you didn't have to write -- Steve Jobs Ce message et les pi?ces jointes sont confidentiels et r?serv?s ? l'usage exclusif de ses destinataires. Il peut ?galement ?tre prot?g? par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir imm?diatement l'exp?diteur et de le d?truire. L'int?grit? du message ne pouvant ?tre assur?e sur Internet, la responsabilit? du groupe Atos Origin ne pourra ?tre recherch?e quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'exp?diteur ne donne aucune garantie ? cet ?gard et sa responsabilit? ne saurait ?tre recherch?e pour tout dommage r?sultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. From ae at dal.ca Tue Jun 23 11:35:44 2009 From: ae at dal.ca (Aidan Evans) Date: Tue, 23 Jun 2009 12:35:44 -0300 (ADT) Subject: service imap pid nnnn in BUSY state: terminated abnormally In-Reply-To: References: Message-ID: On Fri, 29 May 2009 at 14:47 Aidan Evans wrote to info-cyrus at lists.andrew.c... > I am attempting to upgrade a Cyrus Murder from 2.3.12p2 to 2.3.14. The > mupdate and backend servers are happy at 2.3.14, but on the frontends while > I can login successfully, attempting to select a folder fails with messages > like > > May 29 12:13:59 kil-imap-13 master[15558]: process 15584 exited, signaled to > death by 11 > May 29 12:13:59 kil-imap-13 master[15558]: service imap pid 15584 in BUSY > state: terminated abnormally > > This is on Red Hat Linux 5 (2.6.18-128.1.6.el5PAE kernel). It appears this is ultimately a problem with the xstrdup function in xmalloc.c, or with its usage. The imap service failure occurs in backend.c in the following code starting about line 424 in the 2.3.14 source /* now need to authenticate to backend server, unless we're doing LMTP/CSYNC on a UNIX socket (deliver/sync_client) */ if ((server[0] != '/') || (strcmp(prot->sasl_service, "lmtp") && strcmp(prot->sasl_service, "csync"))) { char *mlist = xstrdup(mechlist); /* backend_auth is destructive */ if ((r = backend_authenticate(ret, prot, &mlist, userid, cb, auth_status))) { In the call to xstrdup, the value of mechlist happens to be NULL. xstrdup is char *xstrdup(const char* str) { char *p = xmalloc(strlen(str)+1); strcpy(p, str); return p; } and the strlen(str) when str is NULL produces a segmentation fault. If I change char *mlist = xstrdup(mechlist); /* backend_auth is destructive */ to char *mlist = mechlist ? xstrdup(mechlist) : NULL; /* backend_auth is destructive */ a "SELECT folder" no longer crashes. This leads me to conclude thqt either one must ensure that xstrdup is never called with a NULL string pointer, or maybe xstrdup should check for this and return NULL in that case (this would be relieve callers from having to check). Of course the bigger question is why I am seeing this problem and apparently no one else is. Aidan Evans | Networks & Systems (902)494-3332 | Dalhousie University, Halifax, N.S., Canada From rudy.gevaert at ugent.be Wed Jun 24 03:13:05 2009 From: rudy.gevaert at ugent.be (Rudy Gevaert) Date: Wed, 24 Jun 2009 09:13:05 +0200 Subject: lmtp delivery if over quota Message-ID: <20090624091305.60574gkuzyr9nv29@langoest.ugent.be> Hi, I was wondering if it is possible to deliver an email to a mailbox even when the mailbox is over quota. We sometimes have users emailing us 'we are over quota', but we can't reply because they are over quota :). We then temporary increase the quota till the users cleans up his mailbox. However it would be easy to still deliver email in certain cases. Any ideas? Thanks! -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert Rudy.Gevaert at UGent.be tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- From rudy.gevaert at ugent.be Wed Jun 24 03:21:38 2009 From: rudy.gevaert at ugent.be (Rudy Gevaert) Date: Wed, 24 Jun 2009 09:21:38 +0200 Subject: lmtp delivery if over quota In-Reply-To: <20090624091305.60574gkuzyr9nv29@langoest.ugent.be> References: <20090624091305.60574gkuzyr9nv29@langoest.ugent.be> Message-ID: <20090624092138.81697cog6odjgnwi@langoest.ugent.be> Citeren Rudy Gevaert : > Hi, > > I was wondering if it is possible to deliver an email to a mailbox > even when the mailbox is over quota. > > We sometimes have users emailing us 'we are over quota', but we can't > reply because they are over quota :). We then temporary increase the > quota till the users cleans up his mailbox. > > However it would be easy to still deliver email in certain cases. > > Any ideas? And now I see that lmtp has an IGNOREQUOTA extension! :) Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert Rudy.Gevaert at UGent.be tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- From juergen.wolf at idmt.fraunhofer.de Wed Jun 24 09:49:15 2009 From: juergen.wolf at idmt.fraunhofer.de (Juergen Wolf) Date: Wed, 24 Jun 2009 15:49:15 +0200 Subject: service imap pid nnnn in BUSY state: terminated abnormally In-Reply-To: References: Message-ID: <20090624154915.2f9bc7a5@il007.idmt.fraunhofer.de> On Tue, 23 Jun 2009 12:35:44 -0300 (ADT) Aidan Evans wrote: . . . > > Of course the bigger question is why I am seeing this problem and > apparently no one else is. Hello, I have absolutely the same problem here. In addition to the crashing imapd I see the lmtpd and the lmtpproxyd crashing as well when trying to authenticate to a backend on a solaris10 cluster. SunOS imailserv01 5.10 Generic_141414-01 sun4v sparc SUNW,SPARC-Enterprise-T2000 Jun 24 06:25:19 imailserv01 master[11233]: about to exec /opt/local/bin/lmtpd Jun 24 06:25:36 imailserv01 master[11089]: process 11132 exited, signaled to death by 11 Jun 24 06:25:36 imailserv01 master[11089]: service lmtp pid 11132 in BUSY state: terminated abnormally SunOS imailserv-frontend 5.10 Generic_141414-01 sun4v sparc SUNW,SPARC-Enterprise-T2000 Jun 24 14:48:00 imailserv-frontend master[6406]: about to exec /opt/local/bin/lmtpproxyd Jun 24 14:48:01 imailserv-frontend master[6388]: process 6406 exited, signaled to death by 11 Jun 24 14:48:01 imailserv-frontend master[6388]: service lmtpunix pid 6406 in BUSY state: terminated abnormally So it seems this is a more general problem in version 2.3.14 as 2.3.13 runs fine. Cyradm Version Output: name : Cyrus IMAPD version : v2.3.14-openpkg 2009/03/25 10:41:00 vendor : Project Cyrus support-url: http://cyrusimap.web.cmu.edu os : SunOS os-version : 5.10 environment: Built w/Cyrus SASL 2.1.23 Running w/Cyrus SASL 2.1.23 Built w/Berkeley DB 4.7.25: (May 15, 2008) Running w/Berkeley DB 4.7.25: (May 15, 2008) Built w/OpenSSL 0.9.8i 15 Sep 2008 Running w/OpenSSL 0.9.8i 15 Sep 2008 CMU Sieve 2.3 mmap = shared lock = fcntl nonblock = fcntl idle = poll Kind Regards, J?rgen Wolf -- email: Juergen.Wolf at idmt.fraunhofer.de gilb: Fraunhofer-Institut fuer Digitale Medientechnologie IDMT 98693 Ilmenau, Ehrenbergstr. 31 Tel.: +49 3677 467-234 Fax: +49 3677 467-467 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 1965 bytes Desc: not available Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090624/d509c370/attachment.bin From simon.matter at invoca.ch Wed Jun 24 10:00:08 2009 From: simon.matter at invoca.ch (Simon Matter) Date: Wed, 24 Jun 2009 16:00:08 +0200 Subject: service imap pid nnnn in BUSY state: terminated abnormally In-Reply-To: <20090624154915.2f9bc7a5@il007.idmt.fraunhofer.de> References: <20090624154915.2f9bc7a5@il007.idmt.fraunhofer.de> Message-ID: <70555a475c21dd01ce78177f5fb8514d.squirrel@webmail.bi.corp.invoca.ch> > On Tue, 23 Jun 2009 12:35:44 -0300 (ADT) > Aidan Evans wrote: > > . > . > . > >> >> Of course the bigger question is why I am seeing this problem and >> apparently no one else is. > > Hello, > > I have absolutely the same problem here. In addition to the crashing > imapd I see the lmtpd and the lmtpproxyd crashing as well when trying > to authenticate to a backend on a solaris10 cluster. Hi, Did you apply this patch? http://github.com/brong/cyrus-imapd/commit/ec1bfcf6a1db9c86cbf55b9c25d7eb044dbbe51b#diff-0 You seem to be running openpkg rpms but the spec here http://cvs.openpkg.org/fileview?f=openpkg-src/imapd/imapd.spec doesn't seem to include that patch. Regards, Simon > > > SunOS imailserv01 5.10 Generic_141414-01 sun4v sparc > SUNW,SPARC-Enterprise-T2000 > > Jun 24 06:25:19 imailserv01 master[11233]: about to exec > /opt/local/bin/lmtpd > Jun 24 06:25:36 imailserv01 master[11089]: process 11132 exited, > signaled to death by 11 > Jun 24 06:25:36 imailserv01 master[11089]: service lmtp pid 11132 > in BUSY state: terminated abnormally > > > SunOS imailserv-frontend 5.10 Generic_141414-01 sun4v sparc > SUNW,SPARC-Enterprise-T2000 > > Jun 24 14:48:00 imailserv-frontend master[6406]: about to exec > /opt/local/bin/lmtpproxyd > Jun 24 14:48:01 imailserv-frontend master[6388]: process 6406 > exited, signaled to death by 11 > Jun 24 14:48:01 imailserv-frontend master[6388]: service lmtpunix > pid 6406 in BUSY state: terminated abnormally > > So it seems this is a more general problem in version 2.3.14 as 2.3.13 > runs fine. > > Cyradm Version Output: > > > name : Cyrus IMAPD > version : v2.3.14-openpkg 2009/03/25 10:41:00 > vendor : Project Cyrus > support-url: http://cyrusimap.web.cmu.edu > os : SunOS > os-version : 5.10 > environment: Built w/Cyrus SASL 2.1.23 > Running w/Cyrus SASL 2.1.23 > Built w/Berkeley DB 4.7.25: (May 15, 2008) > Running w/Berkeley DB 4.7.25: (May 15, 2008) > Built w/OpenSSL 0.9.8i 15 Sep 2008 > Running w/OpenSSL 0.9.8i 15 Sep 2008 > CMU Sieve 2.3 > mmap = shared > lock = fcntl > nonblock = fcntl > idle = poll > > Kind Regards, > J?rgen Wolf > > -- > email: Juergen.Wolf at idmt.fraunhofer.de > gilb: Fraunhofer-Institut fuer Digitale Medientechnologie IDMT > 98693 Ilmenau, Ehrenbergstr. 31 > Tel.: +49 3677 467-234 Fax: +49 3677 467-467 > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html From garry at glendown.de Wed Jun 24 12:13:18 2009 From: garry at glendown.de (Garry Glendown) Date: Wed, 24 Jun 2009 18:13:18 +0200 Subject: Auto-creating accounts? Message-ID: <4A42509E.7070400@glendown.de> Hi, I have a cyrus setup that is working fine, with LDAP auth and the other 9 yards ... I will start a 600+ user migration pretty soon ... for now, I plan on setting up a mirror setup on a VM to let the users get used to the new system ... I was planning on forwarding all live mail from the old system to the test system ... problem is, all the cyrus accounts aren't there yet, and won't be until I do the live migration ... which means I can't receive mails on the system until the user loggs in once ... is there a way to get Deliver to create the accounts, in contrast to IMAP which already does it? Tnx, -garry From morgan at orst.edu Wed Jun 24 13:27:29 2009 From: morgan at orst.edu (Andrew Morgan) Date: Wed, 24 Jun 2009 10:27:29 -0700 (PDT) Subject: lmtp delivery if over quota In-Reply-To: <20090624092138.81697cog6odjgnwi@langoest.ugent.be> References: <20090624091305.60574gkuzyr9nv29@langoest.ugent.be> <20090624092138.81697cog6odjgnwi@langoest.ugent.be> Message-ID: On Wed, 24 Jun 2009, Rudy Gevaert wrote: > > Citeren Rudy Gevaert : > >> Hi, >> >> I was wondering if it is possible to deliver an email to a mailbox >> even when the mailbox is over quota. >> >> We sometimes have users emailing us 'we are over quota', but we can't >> reply because they are over quota :). We then temporary increase the >> quota till the users cleans up his mailbox. >> >> However it would be easy to still deliver email in certain cases. >> >> Any ideas? We kinda bruteforce it here: open(OUT, "| ssh root\@$location /usr/local/cyrus/bin/deliver -q -r osuhelpdesk\@oregonstate.edu $username"); (where $location is the backend holding $username's mailbox) Then we pipe in our prepared overquota warning message to it. > And now I see that lmtp has an IGNOREQUOTA extension! :) Is anyone using this? I don't have a script which speaks LMTP, so some example code would be useful. Andy From nodens2099 at gmail.com Wed Jun 24 14:10:02 2009 From: nodens2099 at gmail.com (=?UTF-8?Q?Cl=C3=A9ment_Hermann_=28nodens=29?=) Date: Wed, 24 Jun 2009 20:10:02 +0200 Subject: Auto-creating accounts? In-Reply-To: <73c61560906241106j274d7fe0s7b07dc8bd81ee742@mail.gmail.com> References: <4A42509E.7070400@glendown.de> <73c61560906241101h7ab77fc3m20515862367a4eb@mail.gmail.com> <73c61560906241106j274d7fe0s7b07dc8bd81ee742@mail.gmail.com> Message-ID: <73c61560906241110v37ad2569tdebe293e5e553cfd@mail.gmail.com> Seems like it does not know how to reply to a mailing list either ... Clement Hermann (nodens) On Jun 24, 2009 8:06 PM, "Cl?ment Hermann (nodens)" wrote: I would use a script to mass-create the user top mailboxes. Imapcreate.pl can do that, reading a list from standard input. You'll find it on the cyrus wiki. (Sorry about top-posting, the crappy gmail application on my phone doesn't know how to quote properly) Clement Hermann (nodens) > > On Jun 24, 2009 6:16 PM, "Garry Glendown" wrote: > > Hi, > > I have a cyrus... -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090624/5461ec78/attachment.html From Duncan.Gibb at SiriusIT.co.uk Wed Jun 24 14:16:42 2009 From: Duncan.Gibb at SiriusIT.co.uk (Duncan Gibb) Date: Wed, 24 Jun 2009 19:16:42 +0100 Subject: Auto-creating accounts? In-Reply-To: <4A42509E.7070400@glendown.de> References: <4A42509E.7070400@glendown.de> Message-ID: <4A426D8A.7080307@SiriusIT.co.uk> Garry Glendown wrote: GG> is there a way to get Deliver to create the accounts, in contrast GG> to IMAP which already does it? Putting createonpost: yes in imapd.conf will do that (assuming you're using the UoA autocreate patch set much-discussed recently). I'm not sure that's what you really want in your scenario, though. Surely you need to copy over the contents of any existing mailboxes before you start dual-delivery. Unless this is just for testing, of course... Cheers Duncan -- Duncan Gibb - Technical Director Sirius Corporation plc - control through freedom http://www.siriusit.co.uk/ || t: +44 870 608 0063 Debian Cyrus Team - https://alioth.debian.org/projects/pkg-cyrus-imapd/ From simon.matter at invoca.ch Wed Jun 24 14:42:38 2009 From: simon.matter at invoca.ch (Simon Matter) Date: Wed, 24 Jun 2009 20:42:38 +0200 Subject: lmtp delivery if over quota In-Reply-To: References: <20090624091305.60574gkuzyr9nv29@langoest.ugent.be> <20090624092138.81697cog6odjgnwi@langoest.ugent.be> Message-ID: <27741874a5233aa719643cff7fbe955a.squirrel@webmail.bi.corp.invoca.ch> > On Wed, 24 Jun 2009, Rudy Gevaert wrote: > >> >> Citeren Rudy Gevaert : >> >>> Hi, >>> >>> I was wondering if it is possible to deliver an email to a mailbox >>> even when the mailbox is over quota. >>> >>> We sometimes have users emailing us 'we are over quota', but we can't >>> reply because they are over quota :). We then temporary increase the >>> quota till the users cleans up his mailbox. >>> >>> However it would be easy to still deliver email in certain cases. >>> >>> Any ideas? > > We kinda bruteforce it here: > > open(OUT, "| ssh root\@$location /usr/local/cyrus/bin/deliver -q > -r osuhelpdesk\@oregonstate.edu $username"); > > (where $location is the backend holding $username's mailbox) > > Then we pipe in our prepared overquota warning message to it. > >> And now I see that lmtp has an IGNOREQUOTA extension! :) > > Is anyone using this? I don't have a script which speaks LMTP, so some > example code would be useful. Yes I do with my quotawarn.sh script. Basically it calls 'deliver -l' and sends something like warn_user() { echo "warning mailbox $owner with $pct% quota" cat << EOF | submit MAIL FROM:<$from> RCPT TO:<$owner> IGNOREQUOTA DATA From: $from To: $to Subject: WARNING: Your mailbox has reached $pct% quota. Your server mailbox has reached $pct% of your available quota for storing messages. This will not cause any immediate problem, but if the size of your inbox exceeds your quota, you will not be able to receive any new email until some messages are removed from your server mailbox, either by deleting them or saving them to a local folder. If you need assistance cleaning up your inbox, please contact the Help Desk on tel. $helptel or by email at $helpmail. Thank you. . EOF } Regards, Simon From juergen.wolf at idmt.fraunhofer.de Thu Jun 25 02:36:17 2009 From: juergen.wolf at idmt.fraunhofer.de (Juergen Wolf) Date: Thu, 25 Jun 2009 08:36:17 +0200 Subject: service imap pid nnnn in BUSY state: terminated abnormally In-Reply-To: <70555a475c21dd01ce78177f5fb8514d.squirrel@webmail.bi.corp.invoca.ch> References: <20090624154915.2f9bc7a5@il007.idmt.fraunhofer.de> <70555a475c21dd01ce78177f5fb8514d.squirrel@webmail.bi.corp.invoca.ch> Message-ID: <20090625083617.034731d8@il007.idmt.fraunhofer.de> On Wed, 24 Jun 2009 16:00:08 +0200 "Simon Matter" wrote: > > Hi, > > Did you apply this patch? > http://github.com/brong/cyrus-imapd/commit/ec1bfcf6a1db9c86cbf55b9c25d7eb044dbbe51b#diff-0 > > You seem to be running openpkg rpms but the spec here > http://cvs.openpkg.org/fileview?f=openpkg-src/imapd/imapd.spec doesn't > seem to include that patch. > Hello, thanks for this hint. The patch was indeed missing and after applying it to the openpkg rpms and rebuilding cyrus the lmtp crashes disappeared. Kind regards, J?rgen Wolf -- email: Juergen.Wolf at idmt.fraunhofer.de gilb: Fraunhofer-Institut fuer Digitale Medientechnologie IDMT 98693 Ilmenau, Ehrenbergstr. 31 Tel.: +49 3677 467-234 Fax: +49 3677 467-467 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 1965 bytes Desc: not available Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090625/21f9c00d/attachment.bin From rudy.gevaert at ugent.be Thu Jun 25 03:30:25 2009 From: rudy.gevaert at ugent.be (Rudy Gevaert) Date: Thu, 25 Jun 2009 09:30:25 +0200 Subject: lmtp delivery if over quota In-Reply-To: References: <20090624091305.60574gkuzyr9nv29@langoest.ugent.be> <20090624092138.81697cog6odjgnwi@langoest.ugent.be> Message-ID: <20090625093025.106638dle12yfyq9@langoest.ugent.be> Citeren Andrew Morgan : > Is anyone using this? I don't have a script which speaks LMTP, so > some example code would be useful. I found a php lmtp pear class. I'm going to abuse it :) Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert Rudy.Gevaert at UGent.be tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- From niko at petole.demisel.net Thu Jun 25 03:55:10 2009 From: niko at petole.demisel.net (Nicolas KOWALSKI) Date: Thu, 25 Jun 2009 09:55:10 +0200 Subject: lmtp delivery if over quota In-Reply-To: <20090624091305.60574gkuzyr9nv29@langoest.ugent.be> References: <20090624091305.60574gkuzyr9nv29@langoest.ugent.be> Message-ID: <87tz24px29.fsf@petole.demisel.net> Rudy Gevaert writes: > I was wondering if it is possible to deliver an email to a mailbox > even when the mailbox is over quota. If you are using Exim as MDA, it allows the use of the IGNOREQUOTA extension; I do not remember how to do it. If you are using postfix, you may use the cyrus deliver with the '-q' option, as in the following postfix transport in master.cf: # # cyrus # cyrus unix - n n - 1 pipe user=cyrus argv=/usr/local/libexec/cyrus/deliver -q -r ${sender} ${user} -- Nicolas From D.H.Davis at bath.ac.uk Thu Jun 25 05:10:05 2009 From: D.H.Davis at bath.ac.uk (Dennis Davis) Date: Thu, 25 Jun 2009 10:10:05 +0100 (BST) Subject: lmtp delivery if over quota In-Reply-To: <87tz24px29.fsf@petole.demisel.net> References: <20090624091305.60574gkuzyr9nv29@langoest.ugent.be> <87tz24px29.fsf@petole.demisel.net> Message-ID: On Thu, 25 Jun 2009, Nicolas KOWALSKI wrote: > From: Nicolas KOWALSKI > To: info-cyrus at lists.andrew.cmu.edu > Date: Thu, 25 Jun 2009 09:55:10 +0200 > Subject: Re: lmtp delivery if over quota > X-Spam-Score: 0.0 (/) > > Rudy Gevaert writes: > > > I was wondering if it is possible to deliver an email to a mailbox > > even when the mailbox is over quota. > > If you are using Exim as MDA, it allows the use of the IGNOREQUOTA > extension; I do not remember how to do it. It's the "lmtp_ignore_quota" private option of exim's lmtp transport. Something like the second transport of: begin transports # Transport to deliver mail to the Cyrus IMAP server. We're # going to shovel this down the loopback address using the ltmp # protocol. cyrus_ltmp: driver = smtp protocol = lmtp hosts = LOOPBACK hosts_override = true allow_localhost = true # Similar transport to the above. But quota restrictions should # be ignored. Use under circumstances when you want to force the # message through. cyrus_ltmp_ignore_quota: driver = smtp protocol = lmtp hosts = LOOPBACK lmtp_ignore_quota = true hosts_override = true allow_localhost = true ... should (ie I haven't really tested this) do the job. -- Dennis Davis, BUCS, University of Bath, Bath, BA2 7AY, UK D.H.Davis at bath.ac.uk Phone: +44 1225 386101 From raines at nmr.mgh.harvard.edu Thu Jun 25 17:05:13 2009 From: raines at nmr.mgh.harvard.edu (Paul Raines) Date: Thu, 25 Jun 2009 17:05:13 -0400 (EDT) Subject: deliver fails when -q option used Message-ID: I am moving our cyrus mail server from a CentOS4 to CentOS5 box. This means a cyrus upgrade from 2.2.12 to 2.3.7 We have sendmail use procmail for local mail delivery which in turn delivers to cyrus IMAP via the deliver command. Specifically we have the lines in procmailrc of: ================================================================ # If this fails, it tries without the extension :0w * ? test -n "$EXTENSION" | $DELIVERMAIL -a $USER -m $EXTENSION -q $USER # If this fails, it returns error! :0w | $DELIVERMAIL -a $USER -q $USER :0: /var/mail/lost.$USER ================================================================ where DELIVERMAIL points to a script that is simply the two lines ================================================================ read junkfrom exec /usr/lib/cyrus-imapd/deliver "$@" ================================================================ to get rid of the initial From line that deliver hates. This has worked fine on CentOS4. But on the CentOS5 box delivery always fails with a line like: Jun 25 16:46:31 tmp2 lmtpunix[30842]: sieve runtime error for raines id : Keep: Over quota The mailbox is definitely not over quota and the -q option is supposed to force delivery even if it is. Delivery works if I remove the -q option. Any ideas why I get this error? -- --------------------------------------------------------------- Paul Raines email: raines at nmr.mgh.harvard.edu MGH/MIT/HMS Athinoula A. Martinos Center for Biomedical Imaging 149 (2301) 13th Street Charlestown, MA 02129 USA From Gary at primeexalia.com Fri Jun 26 02:10:23 2009 From: Gary at primeexalia.com (Gary Smith) Date: Thu, 25 Jun 2009 23:10:23 -0700 Subject: multi server migration question/poblem. Message-ID: <5017258D295FBE41917880488689F7B80FABD90F86@VCSSBS.visionarycs.local> We have a client with about 500 accounts spread across 3 server. These were office servers setup some time ago at the three different locations, running various different versions. On server one, their user accounts are sjo_username (sjo being the city prefix). On server two, their user accounts are sfo_username (sfo being the city prefix). On server three, their user accounts are username at domain.com Server three is the destination for servers one and two. All users now have usernames of username at domain.com. We can directly map prefix_username to their new username at domain syntax without any problem We would like to migrate all of the user accounts to the single server. The server will be running cyrus-imapd.x86_64 v 2.3.14-1 (I just compiled it this week) on CentOS 5.3 (waiting to deploy still): New server is currently running: name : Cyrus IMAPD version : v2.3.12p2-Fedora-RPM-2.3.12p2-3.HS 2008/04/24 16:09:27 vendor : Project Cyrus support-url: http://cyrusimap.web.cmu.edu os : Linux os-version : 2.6.18-92.1.18.el5xen environment: Built w/Cyrus SASL 2.1.22 Running w/Cyrus SASL 2.1.22 Built w/Sleepycat Software: Berkeley DB 4.3.29: (January 7, 2007) Running w/Sleepycat Software: Berkeley DB 4.3.29: (January 7, 2007) Built w/OpenSSL 0.9.8b 04 May 2006 Running w/OpenSSL 0.9.8b 04 May 2006 CMU Sieve 2.3 TCP Wrappers NET-SNMP mmap = shared lock = fcntl nonblock = fcntl idle = poll Old servers are running: name : Cyrus IMAPD version : v2.2.12-rPath-Linux-2.2.12 2005/02/14 16:43:51 vendor : Project Cyrus support-url: http://asg.web.cmu.edu/cyrus os : Linux os-version : 2.6.18-92.1.22.el5xen environment: Built w/Cyrus SASL 2.1.22 Running w/Cyrus SASL 2.1.22 Built w/Sleepycat Software: Berkeley DB 4.3.28: (October 23, 2005) Running w/Sleepycat Software: Berkeley DB 4.3.28: (October 23, 2005) Built w/OpenSSL 0.9.7f 22 Mar 2005 Running w/OpenSSL 0.9.7f 22 Mar 2005 CMU Sieve 2.2 DRAC TCP Wrappers mmap = shared lock = fcntl nonblock = fcntl auth = unix idle = poll I can place these machines side by side to migrate the email. Question is, what's the best way to migrate the email while retaining the message flags and siege settings? In the past I have tried replication but I have failed to get it working on the RPath distro and I don't really have time to troubleshoot the build problems on RPath. I can however, setup an entirely new xen instance running CentOS 5.3 server and copy over the mailboxes to the new instance so at least they are on the save version. Any suggestions on how to do this. I'm attempted other ways in the past, but it's time to actually get it done and over with. Gary From awilliam at whitemice.org Fri Jun 26 11:23:18 2009 From: awilliam at whitemice.org (Adam Tauno Williams) Date: Fri, 26 Jun 2009 11:23:18 -0400 Subject: ipurge and delayed expunge Message-ID: <1246029798.5584.4.camel@linux-m3mt> We use delayed expunge with a delay of 120 days. I was just playing with ipurge and it appears to 'physically' delete the messages rather than just marking messages as expunged. Is it possible to have ipurge just mark messages as expunged rather than removing them? Looking at the man page I assume not. $ sudo -u cyrus /usr/lib/cyrus-imapd/ipurge -s -X -f -d 365 user.adam.sent-mail Working on user.adam.sent-mail... total messages 713 total bytes 7683241 Deleted messages 0 Deleted bytes 0 Remaining messages 713 Remaining bytes 7683241 $ ls /var/spool/imap/a/user/adam/SPAM/ | wc -l 1703 $ sudo -u cyrus /usr/lib/cyrus-imapd/ipurge -s -X -f -d 5 user.adam.SPAM Working on user.adam.SPAM... total messages 205 total bytes 6493756 Deleted messages 130 Deleted bytes 4143062 Remaining messages 75 Remaining bytes 2350694 $ ls /var/spool/imap/a/user/adam/SPAM/ | wc -l 1573 From bawood at umich.edu Fri Jun 26 12:35:32 2009 From: bawood at umich.edu (Brian Awood) Date: Fri, 26 Jun 2009 12:35:32 -0400 Subject: ipurge and delayed expunge In-Reply-To: <1246029798.5584.4.camel@linux-m3mt> References: <1246029798.5584.4.camel@linux-m3mt> Message-ID: <08957bd165abc02d12f74573d64bd62c@umich.edu> This issue is fixed in CVS; http://lists.andrew.cmu.edu/pipermail/info-cyrus/2009-March/030753.html -Brian On Friday 26 June 2009 @ 11:23, Adam Tauno Williams wrote: > We use delayed expunge with a delay of 120 days. > > I was just playing with ipurge and it appears to 'physically' > delete the messages rather than just marking messages as expunged. > Is it possible to have ipurge just mark messages as expunged rather > than removing them? > > Looking at the man page I assume not. > > $ sudo -u cyrus /usr/lib/cyrus-imapd/ipurge -s -X -f -d 365 > user.adam.sent-mail > Working on user.adam.sent-mail... > total messages 713 > total bytes 7683241 > Deleted messages 0 > Deleted bytes 0 > Remaining messages 713 > Remaining bytes 7683241 > $ ls /var/spool/imap/a/user/adam/SPAM/ | wc -l > 1703 > > $ sudo -u cyrus /usr/lib/cyrus-imapd/ipurge -s -X -f -d 5 > user.adam.SPAM Working on user.adam.SPAM... > total messages 205 > total bytes 6493756 > Deleted messages 130 > Deleted bytes 4143062 > Remaining messages 75 > Remaining bytes 2350694 > $ ls /var/spool/imap/a/user/adam/SPAM/ | wc -l > 1573 > > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: rcmAttmntNa8yLC Url: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090626/1bea021b/attachment.ksh From awilliam at whitemice.org Fri Jun 26 14:43:57 2009 From: awilliam at whitemice.org (Adam Tauno Williams) Date: Fri, 26 Jun 2009 14:43:57 -0400 Subject: ipurge and delayed expunge In-Reply-To: <08957bd165abc02d12f74573d64bd62c@umich.edu> References: <1246029798.5584.4.camel@linux-m3mt> <08957bd165abc02d12f74573d64bd62c@umich.edu> Message-ID: <1246041837.5584.14.camel@linux-m3mt> On Fri, 2009-06-26 at 12:35 -0400, Brian Awood wrote: > This issue is fixed in CVS; > http://lists.andrew.cmu.edu/pipermail/info-cyrus/2009-March/030753.html Sweet, it appears this patch is included in Simon's awesome RPMs! cyrus-imapd-2.3.14-unexpunge.patch -- OpenGroupware developer: awilliam at whitemice.org OpenGroupare & Cyrus IMAPd documenation @ From awilliam at whitemice.org Fri Jun 26 20:47:59 2009 From: awilliam at whitemice.org (Adam Tauno Williams) Date: Fri, 26 Jun 2009 20:47:59 -0400 Subject: unexpunge Bus Error (signal 7) [Was: Re: ipurge and delayed expunge] In-Reply-To: <1246041837.5584.14.camel@linux-m3mt> References: <1246029798.5584.4.camel@linux-m3mt> <08957bd165abc02d12f74573d64bd62c@umich.edu> <1246041837.5584.14.camel@linux-m3mt> Message-ID: <1246063679.7801.9.camel@linux-m3mt> On Fri, 2009-06-26 at 14:43 -0400, Adam Tauno Williams wrote: > On Fri, 2009-06-26 at 12:35 -0400, Brian Awood wrote: > > This issue is fixed in CVS; > > http://lists.andrew.cmu.edu/pipermail/info-cyrus/2009-March/030753.html > Sweet, it appears this patch is included in Simon's awesome RPMs! > cyrus-imapd-2.3.14-unexpunge.patch > I've updated and ran ipurge. ipurge ran without incident and seems to have performed exactly as expected. But now if I test unexpunge on some of the mailboxes ipurge touched it crashes; other mailboxes seem fine. sudo -u cyrus "/usr/lib/cyrus-imapd/unexpunge -v -l user.adam.sent-mail" ... Bus error A reconstruct works, but then none of the expunged messages appear (list from unexpunge -l is empty). Should a reconstruct loose expunged messages in delayed expunge mode? A backtrace from the core looks like - ... Loaded symbols for /lib/libsepol.so.1 Core was generated by `/usr/lib/cyrus-imapd/unexpunge -v -l user.adam.sent-mail'. Program terminated with signal 7, Bus error. [New process 9137] #0 list_expunged (mailbox=0xbfa32874, msgs=0x8b63628, exists=6, expunge_index_base=0xb6828000
) at unexpunge.c:150 150 cacheitem = CACHE_ITEM_NEXT(cacheitem); /* skip envelope */ (gdb) bt #0 list_expunged (mailbox=0xbfa32874, msgs=0x8b63628, exists=6, expunge_index_base=0xb6828000
) at unexpunge.c:150 #1 0x0804e326 in main (argc=4, argv=0xbfa34c74) at unexpunge.c:632 Environment: Linux 2.6.18-128.1.10.el5 #1 SMP i686 i686 i386 GNU/Linux gcc-4.1.2-44.el5 glibc-2.5-34 cyrus-imapd-2.3.14-8 autoconf-2.59-12 automake-1.9.6-2.1 From bawood at umich.edu Sat Jun 27 10:06:50 2009 From: bawood at umich.edu (Brian Awood) Date: Sat, 27 Jun 2009 10:06:50 -0400 Subject: unexpunge Bus Error (signal 7) [Was: Re: ipurge and delayed expunge] In-Reply-To: <1246063679.7801.9.camel@linux-m3mt> References: <1246029798.5584.4.camel@linux-m3mt> <1246041837.5584.14.camel@linux-m3mt> <1246063679.7801.9.camel@linux-m3mt> Message-ID: <200906271006.52799.bawood@umich.edu> On Friday 26 June 2009 @ 20:47, Adam Tauno Williams wrote: > > A reconstruct works, but then none of the expunged messages appear > (list from unexpunge -l is empty). Should a reconstruct loose > expunged messages in delayed expunge mode? reconstruct will remove the expunged data if you don't specify the -k option, also if it's not able to verify the cyrus.expunge file as valid, it will delete it and put all messages back into the index. I posted a patch to the dev list which addresses this and a couple of other issues in reconstruct. In this case it looks like the cyrus.expunge file was corrupted since the expunge_index_base isn't valid. Possibly due to a previous issue with the expunge file. -Brian From Gary at primeexalia.com Sat Jun 27 13:30:38 2009 From: Gary at primeexalia.com (Gary Smith) Date: Sat, 27 Jun 2009 10:30:38 -0700 Subject: multi server migration question/poblem. References: <5017258D295FBE41917880488689F7B80FABD90F86@VCSSBS.visionarycs.local> <20090626081736.1491382pp738qvwg@langoest.ugent.be> Message-ID: <5017258D295FBE41917880488689F7B80FABD90F9C@VCSSBS.visionarycs.local> > -----Original Message----- > From: Gary Smith > Sent: Thursday, June 25, 2009 11:53 PM > To: 'Rudy Gevaert' > Subject: RE: multi server migration question/poblem. > > > Citeren Gary Smith : > > > > > > > > Any suggestions on how to do this. I'm attempted other ways in the > > > past, but it's time to actually get it done and over with. > > > > You can use imapsync. > I'll give that a test run try. Short of resetting the passwords you map the imap using a privileged account? i.e. can I assign a particular user full access on both sides and then alias it in the connection? It's not too important as I back backup the password database, reset, migrate, and put it back. Just thought that would be easier though. Exactly what I want to do. Do you have a sample string of accessing a mailbox from a different user account with imapcopy? I vaugely remember doing something like authusername at mailboxusername@domain.com/folder or something like that against cyrus some years ago. Any samples would be greatly appreciated. Gary ________________________________________ From nodens2099 at gmail.com Sat Jun 27 14:46:25 2009 From: nodens2099 at gmail.com (=?UTF-8?Q?Cl=C3=A9ment_Hermann_=28nodens=29?=) Date: Sat, 27 Jun 2009 20:46:25 +0200 Subject: multi server migration question/poblem. In-Reply-To: <5017258D295FBE41917880488689F7B80FABD90F9C@VCSSBS.visionarycs.local> References: <5017258D295FBE41917880488689F7B80FABD90F86@VCSSBS.visionarycs.local> <20090626081736.1491382pp738qvwg@langoest.ugent.be> <5017258D295FBE41917880488689F7B80FABD90F9C@VCSSBS.visionarycs.local> Message-ID: <73c61560906271146i3bad7a78y153c9f0fd879891c@mail.gmail.com> 2009/6/27 Gary Smith > > -----Original Message----- > > From: Gary Smith > > Sent: Thursday, June 25, 2009 11:53 PM > > To: 'Rudy Gevaert' > > Subject: RE: multi server migration question/poblem. > > > > > Citeren Gary Smith : > > > > > > > > > > > Any suggestions on how to do this. I'm attempted other ways in the > > > > past, but it's time to actually get it done and over with. > > > > > > You can use imapsync. > > > > > I'll give that a test run try. Short of resetting the passwords you > map the imap using a privileged account? i.e. can I assign a > particular user full access on both sides and then alias it in the > connection? It's not too important as I back backup the password > database, reset, migrate, and put it back. Just thought that would be > easier though. > > Exactly what I want to do. Do you have a sample string of accessing a > mailbox from a different user account with imapcopy? I vaugely remember > doing something like authusername at mailboxusername@domain.com/folder or > something like that against cyrus some years ago. > imapsync can use proxy authentication on source and or destination with "--user1 --authuser1 --authuser2 --user2 " options (authorize as admin, authenticate as user : no need to know/change the password). But more importantly, you'll be able to filter / rewrite any mailbox name using powerful perl regex, not to mention doing incremental transfers and sync mailboxes based on message headers or content, and more. Just grab the tarball, untar it and perldoc imapsync, or even ./imapsync --help. I've used it for several mass user migration (up to 1500 users), IMO the best tool available. However, it may not be the best option if you have a simple cyrus to cyrus migration, without any rewriting, as in this case you could just migrate the files. Cheers, -- Cl?ment Hermann (nodens) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090627/f6821ac0/attachment.html From Gary at primeexalia.com Sat Jun 27 14:54:58 2009 From: Gary at primeexalia.com (Gary Smith) Date: Sat, 27 Jun 2009 11:54:58 -0700 Subject: multi server migration question/poblem. In-Reply-To: <73c61560906271146i3bad7a78y153c9f0fd879891c@mail.gmail.com> References: <5017258D295FBE41917880488689F7B80FABD90F86@VCSSBS.visionarycs.local> <20090626081736.1491382pp738qvwg@langoest.ugent.be> <5017258D295FBE41917880488689F7B80FABD90F9C@VCSSBS.visionarycs.local> <73c61560906271146i3bad7a78y153c9f0fd879891c@mail.gmail.com> Message-ID: <5017258D295FBE41917880488689F7B80FABD90F9D@VCSSBS.visionarycs.local> Clement, Thanks. This is what I?m looking for. We already have a list for source/destination accounts so we should be able to plug it right into this. I just needed a sample so I could do the admin auth for a user account Gary From: Cl?ment Hermann (nodens) [mailto:nodens2099 at gmail.com] Sent: Saturday, June 27, 2009 11:46 AM To: Gary Smith Cc: info-cyrus at lists.andrew.cmu.edu Subject: Re: multi server migration question/poblem. 2009/6/27 Gary Smith > > -----Original Message----- > From: Gary Smith > Sent: Thursday, June 25, 2009 11:53 PM > To: 'Rudy Gevaert' > Subject: RE: multi server migration question/poblem. > > > Citeren Gary Smith >: > > > > > > > > Any suggestions on how to do this. I'm attempted other ways in the > > > past, but it's time to actually get it done and over with. > > > > You can use imapsync. > I'll give that a test run try. Short of resetting the passwords you map the imap using a privileged account? i.e. can I assign a particular user full access on both sides and then alias it in the connection? It's not too important as I back backup the password database, reset, migrate, and put it back. Just thought that would be easier though. Exactly what I want to do. Do you have a sample string of accessing a mailbox from a different user account with imapcopy? I vaugely remember doing something like authusername at mailboxusername@domain.com/folder or something like that against cyrus some years ago. imapsync can use proxy authentication on source and or destination with "--user1 --authuser1 --authuser2 --user2 " options (authorize as admin, authenticate as user : no need to know/change the password). But more importantly, you'll be able to filter / rewrite any mailbox name using powerful perl regex, not to mention doing incremental transfers and sync mailboxes based on message headers or content, and more. Just grab the tarball, untar it and perldoc imapsync, or even ./imapsync --help. I've used it for several mass user migration (up to 1500 users), IMO the best tool available. However, it may not be the best option if you have a simple cyrus to cyrus migration, without any rewriting, as in this case you could just migrate the files. Cheers, -- Cl?ment Hermann (nodens) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090627/6e80e016/attachment.html From rudy.gevaert at ugent.be Tue Jun 30 04:14:15 2009 From: rudy.gevaert at ugent.be (Rudy Gevaert) Date: Tue, 30 Jun 2009 10:14:15 +0200 Subject: catching up with sync logs Message-ID: <20090630101415.46502hyehfje8y8n@langoest.ugent.be> Hello, Last Sunday we had some corruption on the lun of one of the replica's. After recovering it we got it back on line, but we are having some backlog with the sync replication. We have several sync log-* files that need to be processed. While rolling replication is now busy on the normal log file. I'm trying to feed an other sync client the other log files. But it isn't doing much. Even on small log files. E.g.: read(8, "MAILBOX \"ugent.be!user.maarten^b"..., 4096) = 456 time(NULL) = 1246349375 read(8, "", 4096) = 0 write(6, "\27\3\1\0000m\n *\31\215|\303ke\331(\4p\236Q\21Z\313\341"..., 53) = 53 time(NULL) = 1246349375 read(6, and then it waits... and waits. Any tips and tricks on getting this back log fixed would be appreciated! Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert Rudy.Gevaert at UGent.be tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- From brong at fastmail.fm Tue Jun 30 04:51:44 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Tue, 30 Jun 2009 18:51:44 +1000 Subject: catching up with sync logs In-Reply-To: <20090630101415.46502hyehfje8y8n@langoest.ugent.be> References: <20090630101415.46502hyehfje8y8n@langoest.ugent.be> Message-ID: <20090630085144.GA6407@brong.net> On Tue, Jun 30, 2009 at 10:14:15AM +0200, Rudy Gevaert wrote: > Hello, > > Last Sunday we had some corruption on the lun of one of the replica's. > After recovering it we got it back on line, but we are having some > backlog with the sync replication. > > We have several sync log-* files that need to be processed. > > While rolling replication is now busy on the normal log file. I'm > trying to feed an other sync client the other log files. But it isn't > doing much. Even on small log files. E.g.: > > read(8, "MAILBOX \"ugent.be!user.maarten^b"..., 4096) = 456 > time(NULL) = 1246349375 > read(8, "", 4096) = 0 > write(6, "\27\3\1\0000m\n > *\31\215|\303ke\331(\4p\236Q\21Z\313\341"..., 53) = 53 > time(NULL) = 1246349375 > read(6, > > and then it waits... and waits. > > Any tips and tricks on getting this back log fixed would be appreciated! What's happening at the other end? It looks to me like you're either waiting on locks, or the process at the other end has died. (worst case, some corruption isn't being handled correctly by sync_client, and is causing protocol alignment issues, but I suspect locking or death) Bron. From rudy.gevaert at ugent.be Tue Jun 30 05:41:09 2009 From: rudy.gevaert at ugent.be (Rudy Gevaert) Date: Tue, 30 Jun 2009 11:41:09 +0200 Subject: catching up with sync logs In-Reply-To: <20090630085144.GA6407@brong.net> References: <20090630101415.46502hyehfje8y8n@langoest.ugent.be> <20090630085144.GA6407@brong.net> Message-ID: <20090630114109.10306htdey74dbxx@langoest.ugent.be> Hi Bron, Thanks for the quick reply. Citeren Bron Gondwana : > What's happening at the other end? It looks to me like you're either > waiting on locks, or the process at the other end has died. > > (worst case, some corruption isn't being handled correctly by sync_client, > and is causing protocol alignment issues, but I suspect locking or death) (I can also say that the file system check on the replica gave me some lost and found files.) I gave it an other shot and now when I process the 12MB sync log I get: on the cyrus master server: open("/mail/mail3/imap/domain/u/ugent.be/t/user/torsten^dhondt/Ongewenste e-mail/cyrus.index", O_RDWR) = 11 fstat64(11, {st_mode=S_IFREG|0600, st_size=448, ...}) = 0 mmap2(NULL, 16384, PROT_READ, MAP_SHARED, 11, 0) = 0xb7b88000 open("/mail/mail3/imap/domain/u/ugent.be/t/user/torsten^dhondt/Ongewenste e-mail/cyrus.cache", O_RDWR) = 12 fstat64(12, {st_mode=S_IFREG|0600, st_size=8428, ...}) = 0 mmap2(NULL, 24576, PROT_READ, MAP_SHARED, 12, 0) = 0xb7b82000 fstat64(11, {st_mode=S_IFREG|0600, st_size=448, ...}) = 0 fcntl64(3, F_SETLKW, {type=F_RDLCK, whence=SEEK_SET, start=0, len=0}) = 0 fstat64(3, {st_mode=S_IFREG|0600, st_size=4025524, ...}) = 0 stat64("/mail/mail3/var/imap/mailboxes.db", {st_mode=S_IFREG|0600, st_size=4025524, ...}) = 0 fcntl64(3, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0 write(6, "\27\3\1\0p_V\22\0268\256\377\10\360E\257E^\371vV\353\304"..., 117) = 117 time(NULL) = 1246354048 read(6, "\27\3\1\0P", 5) = 5 read(6, "<\374Ec\224\316\3259(\\\341\232\361\272D\275\30=\214\251"..., 80) = 80 time([1246354048]) = 1246354048 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 send(7, "<179>Jun 30 11:27:28 mail3/sync_"..., 160, MSG_NOSIGNAL) = 160 close(9) = 0 munmap(0xb7f4a000, 199) = 0 close(11) = 0 munmap(0xb7b88000, 16384) = 0 close(12) = 0 munmap(0xb7b82000, 24576) = 0 write(6, "\27\3\1\0000\20\241\375A\314R/\230l\362\224e\313\366s\322"..., 53) = 53 time(NULL) = 1246354048 read(6, "\27\3\1\0000", 5) = 5 read(6, "\370\262\322\4\2003\300\213\204\231\322}Dx\324\250\374"..., 48) = 48 time([1246354049]) = 1246354049 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 send(7, "<182>Jun 30 11:27:29 mail3/sync_"..., 71, MSG_NOSIGNAL) = 71 time([1246354049]) = 1246354049 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 send(7, "<179>Jun 30 11:27:29 mail3/sync_"..., 79, MSG_NOSIGNAL) = 79 close(8) = 0 write(6, "\27\3\1\0000%Yw>\301,\4\241GJ\202\261\352G\225\224\304"..., 53) = 53 time(NULL) = 1246354049 read(6, "\27\3\1\0000", 5) = 5 read(6, "\351\313\255\4\tH\24\375u_\21\206\0\3166\312yS\310\216"..., 48) = 48 time(NULL) = 1246354049 select(7, [6], NULL, NULL, {0, 0}) = 0 (Timeout) time(NULL) = 1246354049 shutdown(6, 0 /* receive */) = 0 close(6) = 0 munmap(0xb6345000, 16384) = 0 close(10) = 0 munmap(0xb6477000, 16384) = 0 close(5) = 0 munmap(0xb605b000, 327680) = 0 close(4) = 0 munmap(0xb64c7000, 4038656) = 0 close(3) = 0 munmap(0xb68a1000, 32768) = 0 munmap(0xb7a5d000, 98304) = 0 munmap(0xb68a9000, 18563072) = 0 munmap(0xb7a75000, 663552) = 0 munmap(0xb7f4b000, 16384) = 0 exit_group(-1904809439) = ? Process 16586 detached on the replica I straced the sync server that was handling that sync_client open("/mail/mail3r/imap/sync./8349.cache", O_RDWR|O_CREAT|O_TRUNC, 0666) = 16 fcntl64(18, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0 close(18) = 0 time([1246354049]) = 1246354049 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 send(6, "<182>Jun 30 11:27:29 mail3r/sync"..., 64, MSG_NOSIGNAL) = 64 write(1, "\27\3\1\0000\370\262\322\4\2003\300\213\204\231\322}Dx"..., 53) = 53 time(NULL) = 1246354049 read(0, "\27\3\1\0000", 5) = 5 read(0, "%Yw>\301,\4\241GJ\202\261\352G\225\224\304x=\21+R6#z\254"..., 48) = 48 write(1, "\27\3\1\0000\351\313\255\4\tH\24\375u_\21\206\0\3166\312"..., 53) = 53 close(19) = 0 munmap(0xb7f85000, 201) = 0 close(20) = 0 munmap(0xb6464000, 24576) = 0 close(21) = 0 munmap(0xb629a000, 548864) = 0 close(16) = 0 unlink("/mail/mail3r/imap/sync./8349.cache") = 0 rmdir("") = -1 ENOENT (No such file or directory) close(15) = 0 munmap(0xb6407000, 4096) = 0 unlink("/mail/mail3r/var/imap/proc/8349") = 0 time(NULL) = 1246354049 select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout) time(NULL) = 1246354049 open("/dev/null", O_RDWR) = 15 shutdown(0, 0 /* receive */) = 0 dup2(15, 0) = 0 shutdown(1, 0 /* receive */) = 0 dup2(15, 1) = 1 shutdown(2, 0 /* receive */) = 0 dup2(15, 2) = 2 close(15) = 0 write(3, "\1\0\0\0\235 \0\0", 8) = 8 rt_sigaction(SIGALRM, {0x8072c20, [], SA_ONESHOT}, NULL, 8) = 0 rt_sigaction(SIGHUP, {0x8072c20, [], SA_RESTART|SA_ONESHOT}, NULL, 8) = 0 rt_sigaction(SIGINT, {0x8072c20, [], SA_RESTART|SA_ONESHOT}, NULL, 8) = 0 rt_sigaction(SIGQUIT, {0x8072c20, [], SA_RESTART|SA_ONESHOT}, NULL, 8) = 0 alarm(114) = 0 fcntl64(14, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}) = ? ERESTARTSYS (To be restarted) --- SIGALRM (Alarm clock) @ 0 (0) --- sigreturn() = ? (mask now []) munmap(0xb6320000, 32768) = 0 close(22) = 0 munmap(0xb6475000, 3735552) = 0 close(8) = 0 munmap(0xb6044000, 262144) = 0 close(12) = 0 munmap(0xb643d000, 16384) = 0 close(13) = 0 munmap(0xb639c000, 106496) = 0 close(17) = 0 munmap(0xb6933000, 32768) = 0 munmap(0xb7aef000, 98304) = 0 munmap(0xb693b000, 18563072) = 0 munmap(0xb7b07000, 663552) = 0 munmap(0xb7f86000, 16384) = 0 exit_group(0) = ? Process 8349 detached Maybe this information is not enough, if you tell me what you need I can provide :) -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert Rudy.Gevaert at UGent.be tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- From rudy.gevaert at ugent.be Tue Jun 30 06:02:50 2009 From: rudy.gevaert at ugent.be (Rudy Gevaert) Date: Tue, 30 Jun 2009 12:02:50 +0200 Subject: catching up with sync logs In-Reply-To: <20090630114109.10306htdey74dbxx@langoest.ugent.be> References: <20090630101415.46502hyehfje8y8n@langoest.ugent.be> <20090630085144.GA6407@brong.net> <20090630114109.10306htdey74dbxx@langoest.ugent.be> Message-ID: <20090630120250.430291pu2oqt9496@langoest.ugent.be> About that specific user: on the replica it was on a specific folder the sync was stopping. When I reconstructed his mailbox on the replica I was able to sync_client -u user at domain. Then the user came in sync. Citeren Rudy Gevaert : > Hi Bron, > > Thanks for the quick reply. > > Citeren Bron Gondwana : > >> What's happening at the other end? It looks to me like you're either >> waiting on locks, or the process at the other end has died. >> >> (worst case, some corruption isn't being handled correctly by sync_client, >> and is causing protocol alignment issues, but I suspect locking or death) > > (I can also say that the file system check on the replica gave me some > lost and found files.) > > I gave it an other shot and now when I process the 12MB sync log I get: > > on the cyrus master server: > > > > open("/mail/mail3/imap/domain/u/ugent.be/t/user/torsten^dhondt/Ongewenste > e-mail/cyrus.index", O_RDWR) = 11 > fstat64(11, {st_mode=S_IFREG|0600, st_size=448, ...}) = 0 > mmap2(NULL, 16384, PROT_READ, MAP_SHARED, 11, 0) = 0xb7b88000 > open("/mail/mail3/imap/domain/u/ugent.be/t/user/torsten^dhondt/Ongewenste > e-mail/cyrus.cache", O_RDWR) = 12 > fstat64(12, {st_mode=S_IFREG|0600, st_size=8428, ...}) = 0 > mmap2(NULL, 24576, PROT_READ, MAP_SHARED, 12, 0) = 0xb7b82000 > fstat64(11, {st_mode=S_IFREG|0600, st_size=448, ...}) = 0 > fcntl64(3, F_SETLKW, {type=F_RDLCK, whence=SEEK_SET, start=0, len=0}) = 0 > fstat64(3, {st_mode=S_IFREG|0600, st_size=4025524, ...}) = 0 > stat64("/mail/mail3/var/imap/mailboxes.db", {st_mode=S_IFREG|0600, > st_size=4025524, ...}) = 0 > fcntl64(3, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0 > write(6, > "\27\3\1\0p_V\22\0268\256\377\10\360E\257E^\371vV\353\304"..., 117) = > 117 > time(NULL) = 1246354048 > read(6, "\27\3\1\0P", 5) = 5 > read(6, "<\374Ec\224\316\3259(\\\341\232\361\272D\275\30=\214\251"..., > 80) = 80 > time([1246354048]) = 1246354048 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > send(7, "<179>Jun 30 11:27:28 mail3/sync_"..., 160, MSG_NOSIGNAL) = 160 > close(9) = 0 > munmap(0xb7f4a000, 199) = 0 > close(11) = 0 > munmap(0xb7b88000, 16384) = 0 > close(12) = 0 > munmap(0xb7b82000, 24576) = 0 > write(6, > "\27\3\1\0000\20\241\375A\314R/\230l\362\224e\313\366s\322"..., 53) = 53 > time(NULL) = 1246354048 > read(6, "\27\3\1\0000", 5) = 5 > read(6, "\370\262\322\4\2003\300\213\204\231\322}Dx\324\250\374"..., 48) = 48 > time([1246354049]) = 1246354049 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > send(7, "<182>Jun 30 11:27:29 mail3/sync_"..., 71, MSG_NOSIGNAL) = 71 > time([1246354049]) = 1246354049 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > send(7, "<179>Jun 30 11:27:29 mail3/sync_"..., 79, MSG_NOSIGNAL) = 79 > close(8) = 0 > write(6, > "\27\3\1\0000%Yw>\301,\4\241GJ\202\261\352G\225\224\304"..., 53) = 53 > time(NULL) = 1246354049 > read(6, "\27\3\1\0000", 5) = 5 > read(6, "\351\313\255\4\tH\24\375u_\21\206\0\3166\312yS\310\216"..., 48) = 48 > time(NULL) = 1246354049 > select(7, [6], NULL, NULL, {0, 0}) = 0 (Timeout) > time(NULL) = 1246354049 > shutdown(6, 0 /* receive */) = 0 > close(6) = 0 > munmap(0xb6345000, 16384) = 0 > close(10) = 0 > munmap(0xb6477000, 16384) = 0 > close(5) = 0 > munmap(0xb605b000, 327680) = 0 > close(4) = 0 > munmap(0xb64c7000, 4038656) = 0 > close(3) = 0 > munmap(0xb68a1000, 32768) = 0 > munmap(0xb7a5d000, 98304) = 0 > munmap(0xb68a9000, 18563072) = 0 > munmap(0xb7a75000, 663552) = 0 > munmap(0xb7f4b000, 16384) = 0 > exit_group(-1904809439) = ? > Process 16586 detached > > on the replica I straced the sync server that was handling that sync_client > > open("/mail/mail3r/imap/sync./8349.cache", O_RDWR|O_CREAT|O_TRUNC, 0666) = 16 > fcntl64(18, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0 > close(18) = 0 > time([1246354049]) = 1246354049 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1067, ...}) = 0 > send(6, "<182>Jun 30 11:27:29 mail3r/sync"..., 64, MSG_NOSIGNAL) = 64 > write(1, > "\27\3\1\0000\370\262\322\4\2003\300\213\204\231\322}Dx"..., 53) = 53 > time(NULL) = 1246354049 > read(0, "\27\3\1\0000", 5) = 5 > read(0, "%Yw>\301,\4\241GJ\202\261\352G\225\224\304x=\21+R6#z\254"..., > 48) = 48 > write(1, > "\27\3\1\0000\351\313\255\4\tH\24\375u_\21\206\0\3166\312"..., 53) = 53 > close(19) = 0 > munmap(0xb7f85000, 201) = 0 > close(20) = 0 > munmap(0xb6464000, 24576) = 0 > close(21) = 0 > munmap(0xb629a000, 548864) = 0 > close(16) = 0 > unlink("/mail/mail3r/imap/sync./8349.cache") = 0 > rmdir("") = -1 ENOENT (No such file or > directory) > close(15) = 0 > munmap(0xb6407000, 4096) = 0 > unlink("/mail/mail3r/var/imap/proc/8349") = 0 > time(NULL) = 1246354049 > select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout) > time(NULL) = 1246354049 > open("/dev/null", O_RDWR) = 15 > shutdown(0, 0 /* receive */) = 0 > dup2(15, 0) = 0 > shutdown(1, 0 /* receive */) = 0 > dup2(15, 1) = 1 > shutdown(2, 0 /* receive */) = 0 > dup2(15, 2) = 2 > close(15) = 0 > write(3, "\1\0\0\0\235 \0\0", 8) = 8 > rt_sigaction(SIGALRM, {0x8072c20, [], SA_ONESHOT}, NULL, 8) = 0 > rt_sigaction(SIGHUP, {0x8072c20, [], SA_RESTART|SA_ONESHOT}, NULL, 8) = 0 > rt_sigaction(SIGINT, {0x8072c20, [], SA_RESTART|SA_ONESHOT}, NULL, 8) = 0 > rt_sigaction(SIGQUIT, {0x8072c20, [], SA_RESTART|SA_ONESHOT}, NULL, 8) = 0 > alarm(114) = 0 > fcntl64(14, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}) > = ? ERESTARTSYS (To be restarted) > --- SIGALRM (Alarm clock) @ 0 (0) --- > sigreturn() = ? (mask now []) > munmap(0xb6320000, 32768) = 0 > close(22) = 0 > munmap(0xb6475000, 3735552) = 0 > close(8) = 0 > munmap(0xb6044000, 262144) = 0 > close(12) = 0 > munmap(0xb643d000, 16384) = 0 > close(13) = 0 > munmap(0xb639c000, 106496) = 0 > close(17) = 0 > munmap(0xb6933000, 32768) = 0 > munmap(0xb7aef000, 98304) = 0 > munmap(0xb693b000, 18563072) = 0 > munmap(0xb7b07000, 663552) = 0 > munmap(0xb7f86000, 16384) = 0 > exit_group(0) = ? > Process 8349 detached > > > Maybe this information is not enough, if you tell me what you need I > can provide :) > > -- > -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- > Rudy Gevaert Rudy.Gevaert at UGent.be tel:+32 9 264 4734 > Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. > Groep Systemen Systems group > Universiteit Gent Ghent University > Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be > -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- > > > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert Rudy.Gevaert at UGent.be tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- From ana.ribas at upcnet.es Tue Jun 30 06:58:35 2009 From: ana.ribas at upcnet.es (Ana Ribas Roca) Date: Tue, 30 Jun 2009 12:58:35 +0200 Subject: Registry activity log Message-ID: <20090630125835.1793573h9dalc7vf@vader.upc.es> I'd like knowing if I can obtain more log information, than the habitual in cyrus, about the imap accesses. I need to register all the activity of every mailbox (deleted messages, folders creation ...) Is it possible? How? I've changed the syslog configuration file, adding local6.info and local6.debug, but i haven't seen any difference in the log file. Can someone help me, please? Thank's in advance and sorry for my bad english. -- Anna Ribas Roca Projectes Tecnol?gics UPCnet, Universitat Polit?cnica de Catalunya 08034 BARCELONA From rudy.gevaert at ugent.be Tue Jun 30 08:02:20 2009 From: rudy.gevaert at ugent.be (Rudy Gevaert) Date: Tue, 30 Jun 2009 14:02:20 +0200 Subject: Registry activity log In-Reply-To: <20090630125835.1793573h9dalc7vf@vader.upc.es> References: <20090630125835.1793573h9dalc7vf@vader.upc.es> Message-ID: <20090630140220.14914h4e7ayg4ouk@langoest.ugent.be> Citeren Ana Ribas Roca : > I need to register all the activity of every mailbox (deleted > messages, folders creation ...) > Is it possible? How? http://cyrusimap.web.cmu.edu/twiki/bin/view/Cyrus/CyrusTroubleshooting : It is often useful to get a protocol trace from an IMAP (or other CyrusImap protocol) session, to do this, create a directory for the username in question in /log (e.g. /var/imap/log/rjs3), ensure that this directory is writable by the cyrus user, and then connect using your client application that is having trouble. This will dump the total (unencrypted) protocol log from the session. Note that this is not useful for debugging authentication problems, since the log will only be created after the user successfully authenticates. -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert Rudy.Gevaert at UGent.be tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- From dwhite at olp.net Tue Jun 30 09:29:26 2009 From: dwhite at olp.net (Dan White) Date: Tue, 30 Jun 2009 08:29:26 -0500 Subject: Registry activity log In-Reply-To: <20090630125835.1793573h9dalc7vf@vader.upc.es> References: <20090630125835.1793573h9dalc7vf@vader.upc.es> Message-ID: <4A4A1336.7070508@olp.net> Ana Ribas Roca wrote: > I'd like knowing if I can obtain more log information, than the > habitual in cyrus, about the imap accesses. > I need to register all the activity of every mailbox (deleted > messages, folders creation ...) > Is it possible? How? > > I've changed the syslog configuration file, adding local6.info and > local6.debug, but i haven't seen any difference in the log file. > Can someone help me, please? > > Thank's in advance and sorry for my bad english. > > Ana, Since mailbox/folder administration is usually performed over IMAP, you can enable telemetry logging to capture that information. see 'doc/overview.html' within the cyrus-imapd source for documentation. e.g.: mkdir /var/lib/cyrus/log/cyrus (for the cyrus user) chown cyrus:mail /var/lib/cyrus/log/cyrus mkdir /var/lib/cyrus/log/dwhite (for user dwhite) chown cyrus:mail /var/lib/cyrus/log/dwhite Also, you may want to add this to imapd.conf: sasl_log_level: 7 - Dan From robm at fastmail.fm Tue Jun 30 11:48:08 2009 From: robm at fastmail.fm (Rob Mueller) Date: Wed, 1 Jul 2009 01:48:08 +1000 Subject: Registry activity log References: <20090630125835.1793573h9dalc7vf@vader.upc.es> Message-ID: <040BD38BC5394E52A0F9B00A17BD8DCF@jem> Check out the "auditlog" patches we have here: http://cyrus.brong.fastmail.fm/ It syslogs every message/folder create/delete in a fairly regular format. eg things like: auditlog: create sessionid=<...> mailbox=<...> uniqueid=<...> auditlog: delete sessionid=<...> mailbox=<...> uniqueid=<...> auditlog: append sessionid=<...> mailbox=<...> uniqueid=<...> uid=<...> guid=<...> message-id=<...> auditlog: expunge sessionid=<...> mailbox=<...> uniqueid=<...> uid=<...> auditlog: copy sessionid=<...> srcmailbox=<...> srcuniqueid=<...> srcuid=<...> mailbox=<...> uniqueid=<...> uid=<...> guid=<...> message-id=<...> Rob ----- Original Message ----- From: "Ana Ribas Roca" To: Sent: Tuesday, June 30, 2009 8:58 PM Subject: Registry activity log I'd like knowing if I can obtain more log information, than the habitual in cyrus, about the imap accesses. I need to register all the activity of every mailbox (deleted messages, folders creation ...) Is it possible? How? I've changed the syslog configuration file, adding local6.info and local6.debug, but i haven't seen any difference in the log file. Can someone help me, please? Thank's in advance and sorry for my bad english. -- Anna Ribas Roca Projectes Tecnol?gics UPCnet, Universitat Polit?cnica de Catalunya 08034 BARCELONA ---- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html