From dpc22 at cam.ac.uk  Wed Oct  1 05:03:12 2008
From: dpc22 at cam.ac.uk (David Carter)
Date: Wed, 1 Oct 2008 10:03:12 +0100 (BST)
Subject: More special chars in mailbox names
In-Reply-To: <48E28CFE.2020104@andrew.cmu.edu>
References: <48E28CFE.2020104@andrew.cmu.edu>
Message-ID: <alpine.LSU.2.00.0810010913520.9388@hermes-1.csi.cam.ac.uk>

On Tue, 30 Sep 2008, Ken Murchison wrote:

> I started wondering what other special characters we might want to allow. 
> The original authors were more restrictive with GOODCHARS than the RFC. 
> AFAICT the following characters are permitted by RFC 3501 but aren't included 
> in GOODCHARS
>
> ! # $ & ' ; < > ? [ ^ ` } |

We have used:

#define GOODCHARS
  "\\\" #$%'()*+,-.0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ"[]_`abcdefghijklmnopqrstuvwxyz{|}~"

for a number of years now without any obvious problems. This list was 
taken from someone's suggestion on info-cyrus at the time.

For what it is worth, UW imapd allows '%' and '*' in mailbox names, which 
is why they are on the list above. Those are the two characters I am most 
dubious about. Hardly anyone uses '%'. '*' seems quite popular as a trick 
to push mailboxes up to the top of an alphabetically sorted list.

> ! and ^ have special meaning to Cyrus, so we may not want to add these to 
> GOODCHARS.

I had to remove '!' from the list above when we moved from 2.1.x.

> & and ? might conflict with modified UTF-7 encoding.

& shouldn't be allowed without checking for a valid modified UTF-7 
sequence. What is the problem with '?'.

-- 
David Carter                             Email: David.Carter at ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.

From murch at andrew.cmu.edu  Wed Oct  1 10:13:20 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Wed, 01 Oct 2008 10:13:20 -0400
Subject: More special chars in mailbox names
In-Reply-To: <48E28CFE.2020104@andrew.cmu.edu>
References: <48E28CFE.2020104@andrew.cmu.edu>
Message-ID: <48E38580.9010505@andrew.cmu.edu>

Ken Murchison wrote:
> While looking over bug #3002, 
> https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3002
> 
> I started wondering what other special characters we might want to 
> allow.  The original authors were more restrictive with GOODCHARS than 
> the RFC.  AFAICT the following characters are permitted by RFC 3501 but 
> aren't included in GOODCHARS
> 
> ! # $ & ' ; < > ? [ ^ ` } |
> 
> 
> ! and ^ have special meaning to Cyrus, so we may not want to add these 
> to GOODCHARS.  & and ? might conflict with modified UTF-7 encoding.
> 
> Looking back at the list archives, there have been several requests to 
> allow ' (single quote) and I'm included to do so.
> 
> Does anybody want to make a case for adding more?

I'm leaning towards adding ' # $  to 2.3.13.  Any objections?

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From brong at fastmail.fm  Thu Oct  2 03:41:33 2008
From: brong at fastmail.fm (Bron Gondwana)
Date: Thu, 02 Oct 2008 17:41:33 +1000
Subject: Things for 2.3.13
In-Reply-To: <20080930124326.GA17332@brong.net>
References: <FC1C356C-D735-43BA-805D-D6ECBA276D28@umich.edu>
	<AD374C8EA2744F0BA20490FE5C0AE835@robmhp>
	<37A0266E-15D7-4CB4-856F-5DF6EE23C77D@umich.edu>
	<20080930124326.GA17332@brong.net>
Message-ID: <1222933293.9804.1277105151@webmail.messagingengine.com>

On Tue, 30 Sep 2008 22:43:26 +1000, "Bron Gondwana" <brong at fastmail.fm> said:
> 
> Bunch of whingers the lot of you.
> 
> Here you go...
> 
> It was pretty trivial - most of the time was finding the original
> diff to see how it got enabled:
> 
> https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/sieve/sieve.y.diff?r1=1.36;r2=1.37

Now also known as a blocker so you don't forget to put it in 2.3.13 :)

https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3096

Bron.
-- 
  Bron Gondwana
  brong at fastmail.fm


From murch at andrew.cmu.edu  Thu Oct  2 10:12:34 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Thu, 02 Oct 2008 10:12:34 -0400
Subject: Cyrus 2.3.13 RC1
In-Reply-To: <20080925195904.GA28965@exuma.irbs.com>
References: <48DB78C7.9050903@andrew.cmu.edu>
	<20080925195904.GA28965@exuma.irbs.com>
Message-ID: <48E4D6D2.4080005@andrew.cmu.edu>

This seems like something that should go into 2.3.13.  Wes, Bron, and 
comments?


John Capo wrote:
> Quoting Ken Murchison (murch at andrew.cmu.edu):
>> I just put together a release candidate for Cyrus 2.3.13.  I'd
>> appreciate any independent testing before I release this to the masses.
>>
>> http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc1.tar.gz
>>
>>
>> Noteworthy changes:
>>
>> * Added an experimental "sql" backend for cyrusdb.  Currently MySQL,
>>   PostgreSQL, and SQLite are supported.
>> * Added support for IMAP [CAPABILITY] response code to client-side
>>   of Murder proxies.
>> * Added support for ManageSieve auto-capability response after
>>   STARTTLS and after AUTH with a SASL security layer.
>> * Made MAXWORD and MAXQUOTED sizes configurable via imapd.conf
>> * Rewrote cyrusdb_quotalegacy.c to use readir()
>>   rather than glob.c.  This avoids a potential crash due to
>>   conflicts between glibc and Heimdal implementations of glob().
>> * Added support for fulldirhash to 'ctl_mboxlist -v'
>> * Several skiplist bugfixes.
>>
>> Check doc/changes.html for a complete list of changes.
>>
>> If there are any outstanding issues that you believe still need to be
>> addressed in 2.3.13, please let me know.
> 
> 
> statuscache_db.c assumes an off_t is 32bits.
> 
>     if (p < dend) scdata->index_size = strtoul(p, &p, 10);
> 
> and in the printf that generates the string for the DB.
> 
> Attached is an autoconf patch that Lorenzo Catucci posted to the
> mailing list in May.  The patch applies and re-running autoconf
> does figure out that off_t is 64bits on BSD.  Autoconf fails badly
> in other ways but it always does for me.
> 
> Here is my original post and the details I sent to Rudy Gevaert
> about how the bug bites sometimes.
> 
>> Found a bug handling the off_t index_size value in the statuscache
>> code.  The code assumes off_t is a 32 bits and the message count gets
>>> written as 0.
>> (gdb) p *scdata
>> $22 = {statusitems = 63, index_mtime = 1208893338, index_ino = 1974639,
>> index_size = 6581528, messages = 74789, recent = 174, uidnext = 1547527,
>>   uidvalidity = 1125360596, unseen = 74743, highestmodseq = 0}
>> (gdb) n
>> 247         r = DB->store(statuscachedb, key, keylen, data, datalen, NULL);
>> (gdb) p data
>> $23 = "3 63 1208893338 1974639 6581528 0 74789 174 1547527 1125360596
>> 74743\000
>>
>> The attached patch assumes off_t is 64 bits.  Are there any 32 bit
>> off_t systems left?
>>
> 
> Here's the reason why testing didn't turn up this bug.
> 
> I suspected that this might be something to do with newer compilers
> mucking with arguments to printf and friends and fixing argument
> sizes based on printf paramaters.  This turns out not to be the
> case and I would be astonished if the compiler did not do what it
> was told to do with respect to printf arguments.
> 
> statuscache_db.c:
> 
> 202         /* Sanity check the data */
> 203         if (!scdata->statusitems || !scdata->index_mtime || !scdata->index_ino ||
> 204             !scdata->index_size || !scdata->uidnext || !scdata->uidvalidity) {
> 205             return IMAP_NO_NOSUCHMSG;
> 206         }
> 207
> (gdb) p *scdata
> $11 = {statusitems = 63, index_mtime = 1208964427, index_ino = 48227,
>    index_size = 1856, messages = 0, recent = 20, uidnext = 0,
>    uidvalidity = 216164, unseen = 1166134651, highestmodseq = 17}
> 
> The test above will fail and the status cache data will not be used
> if the bogus data in the status cache is 'just right'.  In the case
> above the uidnext is 0 because the argument that was used to store
> that value was not the right one due to printf argument alignment,
> the 64 bit off_t when a 32 bit value was expected by printf.
> 
> My tests are with folders with many thousands of messages.  When
> printf generates the string to store in the cache, bogus values are
> stored that pass the test above. The bogus status cache data will
> be used resulting in the wrong status.
> 
> * STATUS BK (MESSAGES 75904 RECENT 1620 UIDNEXT 1549555 UIDVALIDITY 1125360596 UNSEEN 75858)
> * STATUS BK (MESSAGES 0 RECENT 75904 UIDNEXT 1620 UIDVALIDITY 1549555 UNSEEN 1125360596)
> 
> John Capo
> 
> 
> 


-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From murch at andrew.cmu.edu  Thu Oct  2 10:20:48 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Thu, 02 Oct 2008 10:20:48 -0400
Subject: Things for 2.3.13
In-Reply-To: <19978FE1-771F-4F18-8A1F-B5AA3C4EB3FA@umich.edu>
References: <200809231617.m8NGH92S005462@unix36.andrew.cmu.edu>	<20080923191515.lhzxulinz4kkos8w@cubmail.cc.columbia.edu>	<48D98F29.6050505@andrew.cmu.edu>	<F6E3B321-2733-43DD-A0B3-21E6D93AA739@umich.edu>	<48D9A20E.8080509@andrew.cmu.edu>	<20080924024310.GA22349@brong.net>
	<19978FE1-771F-4F18-8A1F-B5AA3C4EB3FA@umich.edu>
Message-ID: <48E4D8C0.3090803@andrew.cmu.edu>

#2873 is marked as a blocker, but its from 2006.  Do we know if there 
are still problems with OSX?  I'm checking to see if I can get my hands 
on, or access to, a Mac on campus.

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From wes at umich.edu  Thu Oct  2 11:37:22 2008
From: wes at umich.edu (Wesley Craig)
Date: Thu, 2 Oct 2008 11:37:22 -0400
Subject: Things for 2.3.13
In-Reply-To: <48E4D8C0.3090803@andrew.cmu.edu>
References: <200809231617.m8NGH92S005462@unix36.andrew.cmu.edu>	<20080923191515.lhzxulinz4kkos8w@cubmail.cc.columbia.edu>	<48D98F29.6050505@andrew.cmu.edu>	<F6E3B321-2733-43DD-A0B3-21E6D93AA739@umich.edu>	<48D9A20E.8080509@andrew.cmu.edu>	<20080924024310.GA22349@brong.net>
	<19978FE1-771F-4F18-8A1F-B5AA3C4EB3FA@umich.edu>
	<48E4D8C0.3090803@andrew.cmu.edu>
Message-ID: <F7F70408-9048-45CB-8123-8E55AD4D2C8F@umich.edu>

On 02 Oct 2008, at 10:20, Ken Murchison wrote:
> #2873 is marked as a blocker, but its from 2006.  Do we know if  
> there are still problems with OSX?  I'm checking to see if I can  
> get my hands on, or access to, a Mac on campus.

Goodness knows.  The error seems to be that the locally built cyrus- 
sasl install can't find the locally built openssl install.  The  
reporter doesn't indicate whether the locally built stuff was  
actually there.  Personally, running ./configure from 2.3.13RC1 on my  
10.4.11 laptop gives:

	checking build system type... configure: error: cannot guess build  
type; you must specify one

since the supplied config.guess seems to predate dirt. :)

I'd unblock it, but if you feel the reporter should be given the  
opportunity to clarify the report, more power to you.

:wes

From dimma at higis.ru  Thu Oct  2 11:47:48 2008
From: dimma at higis.ru (Dmitriy Kirhlarov)
Date: Thu, 02 Oct 2008 19:47:48 +0400
Subject: Cyrus 2.3.13 RC1
In-Reply-To: <48E4D6D2.4080005@andrew.cmu.edu>
References: <48DB78C7.9050903@andrew.cmu.edu>	<20080925195904.GA28965@exuma.irbs.com>
	<48E4D6D2.4080005@andrew.cmu.edu>
Message-ID: <48E4ED24.6010005@higis.ru>

Hi, list

Some time ago I report ptloader problem:
http://lists.andrew.cmu.edu/pipermail/info-cyrus/2008-August/029512.html

is it fixed in 2.3.13?

also, in April we discuss about improving log system:
http://lists.andrew.cmu.edu/pipermail/info-cyrus/2008-April/028943.html

is we have this functionality in new release?

WBR
Dmitriy

From lorenzo at sancho.ccd.uniroma2.it  Thu Oct  2 12:23:47 2008
From: lorenzo at sancho.ccd.uniroma2.it (Lorenzo M. Catucci)
Date: Thu, 2 Oct 2008 18:23:47 +0200 (CEST)
Subject: Cyrus 2.3.13 RC1
In-Reply-To: <48E4D6D2.4080005@andrew.cmu.edu>
References: <48DB78C7.9050903@andrew.cmu.edu>
	<20080925195904.GA28965@exuma.irbs.com>
	<48E4D6D2.4080005@andrew.cmu.edu>
Message-ID: <alpine.DEB.1.10.0810021645170.5347@sancho.ccd.uniroma2.it>

On Thu, 2 Oct 2008, Ken Murchison wrote:

KM> This seems like something that should go into 2.3.13.  Wes, Bron, and
KM> comments?
KM> [...]
KM> > 
KM> > statuscache_db.c assumes an off_t is 32bits.
KM> > 
KM> >     if (p < dend) scdata->index_size = strtoul(p, &p, 10);
KM> > 
KM> > and in the printf that generates the string for the DB.
KM> > 
KM>
 
I'd be very happy if the perl patchlet and the update of config.guess/sub 
I wrote about on may, 27 would be applied too; it seems so 80's for cyrus 
not being able to configure out of the box on 64bit x86 systems... as for 
the solaris comment in the dnl lines, I've anabled detection of PIC needs 
as an opt-in... someone sould test (and add, if needed/wanted) new 
versions of Solaris, osX and the like.

As for the patch we are discussing here, I found the reported bug just 
chilling, but I feared the unconditional 64-bit stuff too... The only 
problem is I don't have any old-fashioned pure 32 bit off_t system handy, 
and someone else should check the patch on a real system.

Thank you very much, yours

	lorenzo


+-------------------------+----------------------------------------------+
|   Lorenzo M.  Catucci   | Centro di Calcolo e Documentazione           |
| catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" |
|                         | Via O. Raimondo 18 ** I-00173 ROMA  ** ITALY |
|  Tel. +39 06 7259 2255  | Fax. +39 06 7259 2125                        |
+-------------------------+----------------------------------------------+

From lorenzo at sancho.ccd.uniroma2.it  Thu Oct  2 12:34:21 2008
From: lorenzo at sancho.ccd.uniroma2.it (Lorenzo M. Catucci)
Date: Thu, 2 Oct 2008 18:34:21 +0200 (CEST)
Subject: Cyrus 2.3.13 RC1 [7A2-99B]
Message-ID: <alpine.DEB.1.10.0810021830430.5347@sancho.ccd.uniroma2.it>

I got a bounce from cmu mail-exchangers; resend.

---------- Forwarded message ----------
Date: Thu, 2 Oct 2008 18:23:47 +0200 (CEST)
From: Lorenzo M. Catucci <lorenzo at sancho.ccd.uniroma2.it>
To: Ken Murchison <murch at andrew.cmu.edu>
Cc: jc at irbs.com, cyrus-devel at lists.andrew.cmu.edu
Subject: Re: Cyrus 2.3.13 RC1

On Thu, 2 Oct 2008, Ken Murchison wrote:

KM> This seems like something that should go into 2.3.13.  Wes, Bron, and
KM> comments?
KM> [...]
KM> >
KM> > statuscache_db.c assumes an off_t is 32bits.
KM> >
KM> >     if (p < dend) scdata->index_size = strtoul(p, &p, 10);
KM> >
KM> > and in the printf that generates the string for the DB.
KM> >
KM>

I'd be very happy if the perl patchlet and the update of config.guess/sub
I wrote about on may, 27 would be applied too; it seems so 80's for cyrus
not being able to configure out of the box on 64bit x86 systems... as for
the solaris comment in the dnl lines, I've anabled detection of PIC needs
as an opt-in... someone sould test (and add, if needed/wanted) new
versions of Solaris, osX and the like.

As for the patch we are discussing here, I found the reported bug just
chilling, but I feared the unconditional 64-bit stuff too... The only
problem is I don't have any old-fashioned pure 32 bit off_t system handy,
and someone else should check the patch on a real system.

Thank you very much, yours

 	lorenzo


+-------------------------+----------------------------------------------+
|   Lorenzo M.  Catucci   | Centro di Calcolo e Documentazione           |
| catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" |
|                         | Via O. Raimondo 18 ** I-00173 ROMA  ** ITALY |
|  Tel. +39 06 7259 2255  | Fax. +39 06 7259 2125                        |
+-------------------------+----------------------------------------------+

From lorenzo at sancho.ccd.uniroma2.it  Thu Oct  2 12:40:36 2008
From: lorenzo at sancho.ccd.uniroma2.it (Lorenzo M. Catucci)
Date: Thu, 2 Oct 2008 18:40:36 +0200 (CEST)
Subject: Cyrus 2.3.13 RC1
Message-ID: <alpine.DEB.1.10.0810021839200.5347@sancho.ccd.uniroma2.it>

This is the last attempt. I'm only getting bounces I don't understand!

+-------------------------+----------------------------------------------+
|   Lorenzo M.  Catucci   | Centro di Calcolo e Documentazione           |
| catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" |
|                         | Via O. Raimondo 18 ** I-00173 ROMA  ** ITALY |
|  Tel. +39 06 7259 2255  | Fax. +39 06 7259 2125                        |
+-------------------------+----------------------------------------------+

---------- Forwarded message ----------
Subject: Re: Cyrus 2.3.13 RC1

On Thu, 2 Oct 2008, Ken Murchison wrote:

KM> This seems like something that should go into 2.3.13.  Wes, Bron, and
KM> comments?
KM> [...]
KM> >
KM> > statuscache_db.c assumes an off_t is 32bits.
KM> >
KM> >     if (p < dend) scdata->index_size = strtoul(p, &p, 10);
KM> >
KM> > and in the printf that generates the string for the DB.
KM> >
KM>

I'd be very happy if the perl patchlet and the update of config.guess/sub
I wrote about on may, 27 would be applied too; it seems so 80's for cyrus
not being able to configure out of the box on 64bit x86 systems... as for
the solaris comment in the dnl lines, I've anabled detection of PIC needs
as an opt-in... someone sould test (and add, if needed/wanted) new
versions of Solaris, osX and the like.

As for the patch we are discussing here, I found the reported bug just
chilling, but I feared the unconditional 64-bit stuff too... The only
problem is I don't have any old-fashioned pure 32 bit off_t system handy,
and someone else should check the patch on a real system.

Thank you very much, yours

 	lorenzo


+-------------------------+----------------------------------------------+
|   Lorenzo M.  Catucci   | Centro di Calcolo e Documentazione           |
| catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" |
|                         | Via O. Raimondo 18 ** I-00173 ROMA  ** ITALY |
|  Tel. +39 06 7259 2255  | Fax. +39 06 7259 2125                        |
+-------------------------+----------------------------------------------+

From lorenzo at sancho.ccd.uniroma2.it  Thu Oct  2 12:52:12 2008
From: lorenzo at sancho.ccd.uniroma2.it (Lorenzo M. Catucci)
Date: Thu, 2 Oct 2008 18:52:12 +0200 (CEST)
Subject: Cyrus 2.3.13 RC1
In-Reply-To: <alpine.DEB.1.10.0810021839200.5347@sancho.ccd.uniroma2.it>
References: <alpine.DEB.1.10.0810021839200.5347@sancho.ccd.uniroma2.it>
Message-ID: <alpine.DEB.1.10.0810021846040.6794@sancho.ccd.uniroma2.it>

Sorry for spamming the list... I've just seen from mailman the message did 
pass through all of the times...

The undelivery resposnes I got came from [...] bracknellroofing.com
with Microsoft SMTPSVC(6.0.3790.3959); Thu, 2 Oct 2008 17:41:00 +0100

and [...] by mail.exasoft.info (Postfix) with ESMTP id 92EA947416E

Hope there is some way to black-list those domains, or have the subscribed 
users there talk to their postmasters.

Have a nice forthweek, yours

	lorenzo


On Thu, 2 Oct 2008, Lorenzo M. Catucci wrote:

LMC> This is the last attempt. I'm only getting bounces I don't understand!
LMC>

+-------------------------+----------------------------------------------+
|   Lorenzo M.  Catucci   | Centro di Calcolo e Documentazione           |
| catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" |
|                         | Via O. Raimondo 18 ** I-00173 ROMA  ** ITALY |
|  Tel. +39 06 7259 2255  | Fax. +39 06 7259 2125                        |
+-------------------------+----------------------------------------------+

From pnfisher at berkeley.edu  Thu Oct  2 13:35:50 2008
From: pnfisher at berkeley.edu (Paul Fisher)
Date: Thu, 02 Oct 2008 10:35:50 -0700
Subject: More special chars in mailbox names
In-Reply-To: <48E28CFE.2020104@andrew.cmu.edu>
References: <48E28CFE.2020104@andrew.cmu.edu>
Message-ID: <48E50676.5000507@berkeley.edu>

Ken Murchison wrote:
>
> Has anybody already added some of these in production without any 
> adverse effects?

We've been using #define GOODCHARS "
#$%'()*+,-.0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~"
at Berkeley since 2006.  No adverse effects, and the change was needed
as part of the migration from CGPro to Cyrus.

Paul

From wes at umich.edu  Thu Oct  2 17:35:43 2008
From: wes at umich.edu (Wesley Craig)
Date: Thu, 2 Oct 2008 17:35:43 -0400
Subject: Cyrus 2.3.13 RC1
In-Reply-To: <20080925195904.GA28965@exuma.irbs.com>
References: <48DB78C7.9050903@andrew.cmu.edu>
	<20080925195904.GA28965@exuma.irbs.com>
Message-ID: <62C29548-7E05-4EB5-8849-CD7D5936B474@umich.edu>

On 25 Sep 2008, at 15:59, John Capo wrote:
> statuscache_db.c assumes an off_t is 32bits.
>
>     if (p < dend) scdata->index_size = strtoul(p, &p, 10);
>
> and in the printf that generates the string for the DB.
>
> Attached is an autoconf patch that Lorenzo Catucci posted to the
> mailing list in May.

Was this input into bugzilla in May?  It's pretty easy to lose track  
of valuable patches in the wash of the mailing list.  Doing a brief  
search, there are a number of pending 64-bit related items in Bugzilla.

:wes

From wes at umich.edu  Thu Oct  2 17:45:11 2008
From: wes at umich.edu (Wesley Craig)
Date: Thu, 2 Oct 2008 17:45:11 -0400
Subject: Cyrus 2.3.13 RC1
In-Reply-To: <48E4ED24.6010005@higis.ru>
References: <48DB78C7.9050903@andrew.cmu.edu>	<20080925195904.GA28965@exuma.irbs.com>
	<48E4D6D2.4080005@andrew.cmu.edu> <48E4ED24.6010005@higis.ru>
Message-ID: <94B05A6E-4B16-41A4-B728-F67C740F0A66@umich.edu>

On 02 Oct 2008, at 11:47, Dmitriy Kirhlarov wrote:
> Some time ago I report ptloader problem:
> http://lists.andrew.cmu.edu/pipermail/info-cyrus/2008-August/ 
> 029512.html
> is it fixed in 2.3.13?

I'm sure not.  Did you post a fix?  Also, the report doesn't seem to  
have ever made it into bugzilla.

> also, in April we discuss about improving log system:
> http://lists.andrew.cmu.edu/pipermail/info-cyrus/2008-April/ 
> 028943.html
> is we have this functionality in new release?

That's not going to make it into 2.3.13, as we're down to "blocker"  
bugs at this point.  Perhaps add it as a feature request to bugzilla.

:wes

From wes at umich.edu  Thu Oct  2 18:00:48 2008
From: wes at umich.edu (Wesley Craig)
Date: Thu, 2 Oct 2008 18:00:48 -0400
Subject: Cyrus 2.3.13 RC1
In-Reply-To: <alpine.DEB.1.10.0810021645170.5347@sancho.ccd.uniroma2.it>
References: <48DB78C7.9050903@andrew.cmu.edu>
	<20080925195904.GA28965@exuma.irbs.com>
	<48E4D6D2.4080005@andrew.cmu.edu>
	<alpine.DEB.1.10.0810021645170.5347@sancho.ccd.uniroma2.it>
Message-ID: <7B160101-4FBC-49AF-B4A6-00845AB636D5@umich.edu>

On 02 Oct 2008, at 12:23, Lorenzo M. Catucci wrote:
> I'd be very happy if the perl patchlet and the update of  
> config.guess/sub
> I wrote about on may, 27 would be applied too;

Is there some reason we wouldn't move to the latest config.guess/ 
sub?  Also, can you submit a bugzilla about the perl dlopen thing?

> As for the patch we are discussing here, I found the reported bug just
> chilling, but I feared the unconditional 64-bit stuff too... The only
> problem is I don't have any old-fashioned pure 32 bit off_t system  
> handy,
> and someone else should check the patch on a real system.

Perhaps we need to have a 64-bit strategy discussion.  There doesn't  
seem to be a lot of cyrus code related to size sensitivity.  I  
personally hate the syntax:

#if SIZEOF_OFF_T == 8
     if (p < dend) scdata->index_size = strtoull(p, &p, 10);
#else
      if (p < dend) scdata->index_size = strtoul(p, &p, 10);
#endif

and:

      datalen = snprintf(data, sizeof(data),
#if SIZEOF_OFF_T == 8
		       "%u %u %ld %lu %llu %lu %u %lu %lu %u " MODSEQ_FMT,
#else
  		       "%u %u %ld %lu %lu %lu %u %lu %lu %u " MODSEQ_FMT,
#endif
  		       STATUSCACHE_VERSION, scdata->statusitems,

(Don't take it personally, I can totally see why you'd write it that  
way -- cyrus offers no hints on doing it any other way.)  I would  
typically have a header which isolated all the size sensitivity and  
write the above blocks as:

     if (p < dend) scdata->index_size = strtoofft(p, &p, 10);

and:

      datalen = snprintf(data, sizeof(data),
		       "%u %u %ld %lu %" PRIofft "d %lu %u %lu %lu %u " MODSEQ_FMT,
  		       STATUSCACHE_VERSION, scdata->statusitems,

We have a patch in bugzilla that adds a 'z' modifier specifically to  
support size_t.  But what about uid_t, off_t, pid_t.

:wes

From dimma at higis.ru  Fri Oct  3 02:09:22 2008
From: dimma at higis.ru (Dmitriy Kirhlarov)
Date: Fri, 03 Oct 2008 10:09:22 +0400
Subject: Cyrus 2.3.13 RC1
In-Reply-To: <94B05A6E-4B16-41A4-B728-F67C740F0A66@umich.edu>
References: <48DB78C7.9050903@andrew.cmu.edu>
	<20080925195904.GA28965@exuma.irbs.com>
	<48E4D6D2.4080005@andrew.cmu.edu> <48E4ED24.6010005@higis.ru>
	<94B05A6E-4B16-41A4-B728-F67C740F0A66@umich.edu>
Message-ID: <op.uifpzbrsnagr1v@dmitriy-kirhlarovs-macbook-pro-15.local>

On Fri, 03 Oct 2008 01:45:11 +0400, Wesley Craig <wes at umich.edu> wrote:

> On 02 Oct 2008, at 11:47, Dmitriy Kirhlarov wrote:
>> Some time ago I report ptloader problem:
>> http://lists.andrew.cmu.edu/pipermail/info-cyrus/2008-August/029512.html
>> is it fixed in 2.3.13?
>
> I'm sure not.  Did you post a fix?  Also, the report doesn't seem to  
> have ever made it into bugzilla.

No. I haven't fix.
As workaround, I building cyrus with defined "LDAP_DEPRECATED" option.

About bugzilla. I tryed to register twice, but doesn't get confirmation  
e-mail.
It's very strange. Will try again..

>> also, in April we discuss about improving log system:
>> http://lists.andrew.cmu.edu/pipermail/info-cyrus/2008-April/028943.html
>> is we have this functionality in new release?
>
> That's not going to make it into 2.3.13, as we're down to "blocker" bugs  
> at this point.  Perhaps add it as a feature request to bugzilla.

ok

WBR.
Dmitriy

From lorenzo at sancho.ccd.uniroma2.it  Fri Oct  3 07:19:29 2008
From: lorenzo at sancho.ccd.uniroma2.it (Lorenzo M. Catucci)
Date: Fri, 3 Oct 2008 13:19:29 +0200 (CEST)
Subject: Cyrus 2.3.13 RC1
In-Reply-To: <7B160101-4FBC-49AF-B4A6-00845AB636D5@umich.edu>
References: <48DB78C7.9050903@andrew.cmu.edu>
	<20080925195904.GA28965@exuma.irbs.com>
	<48E4D6D2.4080005@andrew.cmu.edu>
	<alpine.DEB.1.10.0810021645170.5347@sancho.ccd.uniroma2.it>
	<7B160101-4FBC-49AF-B4A6-00845AB636D5@umich.edu>
Message-ID: <alpine.DEB.1.10.0810030921480.14167@sancho.ccd.uniroma2.it>

On Thu, 2 Oct 2008, Wesley Craig wrote:

WC> On 02 Oct 2008, at 12:23, Lorenzo M. Catucci wrote:
WC> > I'd be very happy if the perl patchlet and the update of config.guess/sub
WC> > I wrote about on may, 27 would be applied too;
WC> 
WC> Is there some reason we wouldn't move to the latest config.guess/sub?  Also,
WC> can you submit a bugzilla about the perl dlopen thing?
WC> 

As for config.{guess,sub}, I'd go for the latest too, and for trying not 
to get too outdated in the future...

As for bugzilla, the bug-tracker should be better advertised both on 
the website and in released documentation; I think some summary in the 
changelogs like "fixes xxx bugzilla bugs and introduces yyy bugzilla 
enhancements" would be nice to show...

WC> 
WC> Perhaps we need to have a 64-bit strategy discussion.  There doesn't seem to
WC> be a lot of cyrus code related to size sensitivity.  I personally hate the
WC> syntax:
WC> 

[ ... ]

WC> 
WC> (Don't take it personally, I can totally see why you'd write it that way --
WC> cyrus offers no hints on doing it any other way.)  I would typically have a
WC> header which isolated all the size sensitivity and write the above blocks
WC>
 
[ ... ]

No offense taken, I mostly agree with you, even though I don't really like 
string concatenation magic for formats. Still, I think a step-wise 
strategy would be better, maybe putting in some kind of "silent" 
grep-able comments to help in the following steps.

WC> 
WC> We have a patch in bugzilla that adds a 'z' modifier specifically to support
WC> size_t.  But what about uid_t, off_t, pid_t.
WC> 

add gid_t here!

Thank you very much, yours

	lorenzo

+-------------------------+----------------------------------------------+
|   Lorenzo M.  Catucci   | Centro di Calcolo e Documentazione           |
| catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" |
|                         | Via O. Raimondo 18 ** I-00173 ROMA  ** ITALY |
|  Tel. +39 06 7259 2255  | Fax. +39 06 7259 2125                        |
+-------------------------+----------------------------------------------+

From hdembkowski at alcatel-lucent.com  Fri Oct  3 09:34:54 2008
From: hdembkowski at alcatel-lucent.com (DEMBKOWSKI, Henryk (Henryk))
Date: Fri, 3 Oct 2008 15:34:54 +0200
Subject: How do drop databases?
Message-ID: <B576524D0905194499F90736CAA2758A01E94DD2@DEEXC1U03.de.lucent.com>

Hi,

Is it possible to clear/remove Cyrus databases (mailboxes.db and
annotations.db)?
I can remove file but maybe there is command which sends request to
Cyrus?

Kind Regards,
Henryk


From murch at andrew.cmu.edu  Fri Oct  3 10:42:19 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Fri, 03 Oct 2008 10:42:19 -0400
Subject: How do drop databases?
In-Reply-To: <B576524D0905194499F90736CAA2758A01E94DD2@DEEXC1U03.de.lucent.com>
References: <B576524D0905194499F90736CAA2758A01E94DD2@DEEXC1U03.de.lucent.com>
Message-ID: <48E62F4B.8090202@andrew.cmu.edu>

No special command to do so.  If any of the databases are using BDB, 
then you should also clear out the contents of the db/ subdirectory.


DEMBKOWSKI, Henryk (Henryk) wrote:
> Hi,
> 
> Is it possible to clear/remove Cyrus databases (mailboxes.db and
> annotations.db)?
> I can remove file but maybe there is command which sends request to
> Cyrus?
> 
> Kind Regards,
> Henryk
> 
> 


-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From hdembkowski at alcatel-lucent.com  Mon Oct  6 09:45:34 2008
From: hdembkowski at alcatel-lucent.com (DEMBKOWSKI, Henryk (Henryk))
Date: Mon, 6 Oct 2008 15:45:34 +0200
Subject: How do drop databases?
In-Reply-To: <48E62F4B.8090202@andrew.cmu.edu>
References: <B576524D0905194499F90736CAA2758A01E94DD2@DEEXC1U03.de.lucent.com>
	<48E62F4B.8090202@andrew.cmu.edu>
Message-ID: <B576524D0905194499F90736CAA2758A01E951C5@DEEXC1U03.de.lucent.com>

Ken,

Thanks for info. 

We are using skiplist for all databases. The problem is that we are
using annotations.db
to store some data. When we just remove this database process
sync_server is reinitialized
because there is no annotations.db files. I guess that Cyrus should be
reinitialized and then
empty database will be created. However we can't do this since it is
service impacted. 

I thought that maybe there is command which could tell Cyrus to drop its
database.

Kind Regards,
Henryk



-----Original Message-----
From: Ken Murchison [mailto:murch at andrew.cmu.edu] 
Sent: Friday, October 03, 2008 4:42 PM
To: DEMBKOWSKI, Henryk (Henryk)
Cc: cyrus-devel at lists.andrew.cmu.edu
Subject: Re: How do drop databases?

No special command to do so.  If any of the databases are using BDB,
then you should also clear out the contents of the db/ subdirectory.


DEMBKOWSKI, Henryk (Henryk) wrote:
> Hi,
> 
> Is it possible to clear/remove Cyrus databases (mailboxes.db and
> annotations.db)?
> I can remove file but maybe there is command which sends request to
> Cyrus?
> 
> Kind Regards,
> Henryk
> 
> 


-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From murch at andrew.cmu.edu  Mon Oct  6 11:33:18 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Mon, 06 Oct 2008 11:33:18 -0400
Subject: Cyrus 2.3.13 RC2
Message-ID: <48EA2FBE.2000009@andrew.cmu.edu>

I just put together a second release candidate for Cyrus 2.3.13.  I'd
appreciate any independent testing before I release this to the masses.

http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc2.tar.gz
http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc2.tar.gz.sig


Noteworthy changes:

* Added an experimental "sql" backend for cyrusdb.  Currently MySQL,
   PostgreSQL, and SQLite are supported.
* Added support for IMAP [CAPABILITY] response code to client-side
   of Murder proxies.
* Added support for ManageSieve auto-capability response after
   STARTTLS and after AUTH with a SASL security layer.
* Made MAXWORD and MAXQUOTED sizes configurable via imapd.conf
* Rewrote cyrusdb_quotalegacy.c to use readir()
   rather than glob.c.  This avoids a potential crash due to
   conflicts between glibc and Heimdal implementations of glob().
* Added support for fulldirhash to 'ctl_mboxlist -v'
* Several skiplist transaction bugfixes.
* cyr_expire no longer has a default of 0 (zero) for -X and -D.
   These options must be used explicitly in order to have the desired
   effect.
* Added sieve_utf8fileinto option.

Check doc/changes.html for a complete list of changes.

If there are any outstanding issues that you believe still need to be
addressed in 2.3.13, please let me know.

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University






From wes at umich.edu  Mon Oct  6 15:41:11 2008
From: wes at umich.edu (Wesley Craig)
Date: Mon, 6 Oct 2008 15:41:11 -0400
Subject: [Fwd: Re: Cyrus 2.3.13 RC1]
In-Reply-To: <48DB8BA6.20703@andrew.cmu.edu>
References: <48DB8BA6.20703@andrew.cmu.edu>
Message-ID: <2252539C-614D-40C6-A9DF-81255D4BAF1A@umich.edu>

On 25 Sep 2008, at 09:01, Ken Murchison wrote:
> Anyone want to comment on these requests?  I had planned on getting  
> to these patches, but then the push to get out 2.3.13 began.

> On 25 Sep 2008, at 08:52, Michael Menge wrote:
>
>> https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2991
>> This bug seams easy to fix and users of x86_64 systems don't
>> need to copy an newer version of config.sub to cyrus


Fixed, I think.

>> https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3024
>> I don't know how big the impact of this bug is,
>> but as we use x86_64 I would like to have this fixed.

I don't care for the approach, but I think we can take this report as  
a partial audit.

>> https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3057
>> As indicated in my mail from Sep 11 2008 I would like
>> this patch in 2.3.13

I'd apply this after 2.3.13 is released.

:wes

From murch at andrew.cmu.edu  Tue Oct  7 06:26:18 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Tue, 07 Oct 2008 06:26:18 -0400
Subject: Cyrus 2.3.13 RC2
In-Reply-To: <2e630656ac2399f84971202c829c3874.squirrel@webmail.bi.corp.invoca.ch>
References: <48EA2FBE.2000009@andrew.cmu.edu>
	<2e630656ac2399f84971202c829c3874.squirrel@webmail.bi.corp.invoca.ch>
Message-ID: <48EB394A.40702@andrew.cmu.edu>

Simon Matter wrote:
>> I just put together a second release candidate for Cyrus 2.3.13.  I'd
>> appreciate any independent testing before I release this to the masses.
>>
>> http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc2.tar.gz
>> http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc2.tar.gz.sig
>>
>>
>> Noteworthy changes:
>>
>> * Added an experimental "sql" backend for cyrusdb.  Currently MySQL,
>>    PostgreSQL, and SQLite are supported.
>> * Added support for IMAP [CAPABILITY] response code to client-side
>>    of Murder proxies.
>> * Added support for ManageSieve auto-capability response after
>>    STARTTLS and after AUTH with a SASL security layer.
>> * Made MAXWORD and MAXQUOTED sizes configurable via imapd.conf
>> * Rewrote cyrusdb_quotalegacy.c to use readir()
>>    rather than glob.c.  This avoids a potential crash due to
>>    conflicts between glibc and Heimdal implementations of glob().
>> * Added support for fulldirhash to 'ctl_mboxlist -v'
>> * Several skiplist transaction bugfixes.
>> * cyr_expire no longer has a default of 0 (zero) for -X and -D.
>>    These options must be used explicitly in order to have the desired
>>    effect.
>> * Added sieve_utf8fileinto option.
>>
>> Check doc/changes.html for a complete list of changes.
>>
>> If there are any outstanding issues that you believe still need to be
>> addressed in 2.3.13, please let me know.
> 
> I did some test builds on different systems and found that postgresql
> support doesn't work with postgresql 7.1.x and 7.2.x as shown in the error
> below. I understand that these are old versions but if there is an easy
> workaround for the problem it would still be nice.

Do you happen to have a workaround?


> One question to the new sieve_utf8fileinto options, is the default that it
> behaves like old cyrus versions?

Yes.


-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From ktm at rice.edu  Tue Oct  7 08:48:36 2008
From: ktm at rice.edu (Kenneth Marshall)
Date: Tue, 7 Oct 2008 07:48:36 -0500
Subject: Cyrus 2.3.13 RC2
In-Reply-To: <2e630656ac2399f84971202c829c3874.squirrel@webmail.bi.corp.invoca.ch>
References: <48EA2FBE.2000009@andrew.cmu.edu>
	<2e630656ac2399f84971202c829c3874.squirrel@webmail.bi.corp.invoca.ch>
Message-ID: <20081007124836.GG547@it.is.rice.edu>

On Tue, Oct 07, 2008 at 12:18:01PM +0200, Simon Matter wrote:
> > I just put together a second release candidate for Cyrus 2.3.13.  I'd
> > appreciate any independent testing before I release this to the masses.
> >
> > http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc2.tar.gz
> > http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc2.tar.gz.sig
> >
> >
> > Noteworthy changes:
> >
> > * Added an experimental "sql" backend for cyrusdb.  Currently MySQL,
> >    PostgreSQL, and SQLite are supported.
> > * Added support for IMAP [CAPABILITY] response code to client-side
> >    of Murder proxies.
> > * Added support for ManageSieve auto-capability response after
> >    STARTTLS and after AUTH with a SASL security layer.
> > * Made MAXWORD and MAXQUOTED sizes configurable via imapd.conf
> > * Rewrote cyrusdb_quotalegacy.c to use readir()
> >    rather than glob.c.  This avoids a potential crash due to
> >    conflicts between glibc and Heimdal implementations of glob().
> > * Added support for fulldirhash to 'ctl_mboxlist -v'
> > * Several skiplist transaction bugfixes.
> > * cyr_expire no longer has a default of 0 (zero) for -X and -D.
> >    These options must be used explicitly in order to have the desired
> >    effect.
> > * Added sieve_utf8fileinto option.
> >
> > Check doc/changes.html for a complete list of changes.
> >
> > If there are any outstanding issues that you believe still need to be
> > addressed in 2.3.13, please let me know.
> 
> I did some test builds on different systems and found that postgresql
> support doesn't work with postgresql 7.1.x and 7.2.x as shown in the error
> below. I understand that these are old versions but if there is an easy
> workaround for the problem it would still be nice.
> 
> One question to the new sieve_utf8fileinto options, is the default that it
> behaves like old cyrus versions?
> 
> Thanks,
> Simon
> 

There have been 5 major releases of PostgreSQL since 7.2 was released
and 7.2 is EOL in the next few months. I think it is completely reasonable
to not support version 7.1/7.2 in a new system considering that 7.1 is
EOL and 7.2 will be shortly.

Cheers,
Ken

From murch at andrew.cmu.edu  Tue Oct  7 08:52:30 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Tue, 07 Oct 2008 08:52:30 -0400
Subject: Cyrus 2.3.13 RC2
In-Reply-To: <20081007124836.GG547@it.is.rice.edu>
References: <48EA2FBE.2000009@andrew.cmu.edu>
	<2e630656ac2399f84971202c829c3874.squirrel@webmail.bi.corp.invoca.ch>
	<20081007124836.GG547@it.is.rice.edu>
Message-ID: <48EB5B8E.6090009@andrew.cmu.edu>

Kenneth Marshall wrote:
> On Tue, Oct 07, 2008 at 12:18:01PM +0200, Simon Matter wrote:
>>> I just put together a second release candidate for Cyrus 2.3.13.  I'd
>>> appreciate any independent testing before I release this to the masses.
>>>
>>> http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc2.tar.gz
>>> http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc2.tar.gz.sig
>>>
>>>
>>> Noteworthy changes:
>>>
>>> * Added an experimental "sql" backend for cyrusdb.  Currently MySQL,
>>>    PostgreSQL, and SQLite are supported.
>>> * Added support for IMAP [CAPABILITY] response code to client-side
>>>    of Murder proxies.
>>> * Added support for ManageSieve auto-capability response after
>>>    STARTTLS and after AUTH with a SASL security layer.
>>> * Made MAXWORD and MAXQUOTED sizes configurable via imapd.conf
>>> * Rewrote cyrusdb_quotalegacy.c to use readir()
>>>    rather than glob.c.  This avoids a potential crash due to
>>>    conflicts between glibc and Heimdal implementations of glob().
>>> * Added support for fulldirhash to 'ctl_mboxlist -v'
>>> * Several skiplist transaction bugfixes.
>>> * cyr_expire no longer has a default of 0 (zero) for -X and -D.
>>>    These options must be used explicitly in order to have the desired
>>>    effect.
>>> * Added sieve_utf8fileinto option.
>>>
>>> Check doc/changes.html for a complete list of changes.
>>>
>>> If there are any outstanding issues that you believe still need to be
>>> addressed in 2.3.13, please let me know.
>> I did some test builds on different systems and found that postgresql
>> support doesn't work with postgresql 7.1.x and 7.2.x as shown in the error
>> below. I understand that these are old versions but if there is an easy
>> workaround for the problem it would still be nice.
>>
>> One question to the new sieve_utf8fileinto options, is the default that it
>> behaves like old cyrus versions?
>>
>> Thanks,
>> Simon
>>
> 
> There have been 5 major releases of PostgreSQL since 7.2 was released
> and 7.2 is EOL in the next few months. I think it is completely reasonable
> to not support version 7.1/7.2 in a new system considering that 7.1 is
> EOL and 7.2 will be shortly.

I wasn't aware of the release history, thanks for that.

Given this, I agree that support for 7.1/7.2 isn't necessary, especially 
since the cyrusdb_sql.c code is experimental and not built by default. 
If somebody really wants/needs 7.1/7.2 for Cyrus, they can send a patch.

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From brong at fastmail.fm  Wed Oct  8 00:36:06 2008
From: brong at fastmail.fm (Bron Gondwana)
Date: Wed, 8 Oct 2008 15:36:06 +1100
Subject: Cyrus 2.3.13 RC2
In-Reply-To: <48EA2FBE.2000009@andrew.cmu.edu>
References: <48EA2FBE.2000009@andrew.cmu.edu>
Message-ID: <20081008043606.GA25036@brong.net>

On Mon, Oct 06, 2008 at 11:33:18AM -0400, Ken Murchison wrote:
> I just put together a second release candidate for Cyrus 2.3.13.  I'd
> appreciate any independent testing before I release this to the masses.

Sorry about the delay in testing - we've had a few exciting issues
here that had to be fixed first.

> If there are any outstanding issues that you believe still need to be
> addressed in 2.3.13, please let me know.

No, it's looking good.  I just removed the patches that have gone into
CVS from my build tree and it built fine.  Running on our test server
now with no problems.  All the patches that have gone upstream have
been running happily on our production machines for a bit too.

I think now's a good time to release a 2.3.13.

Bron.



From brong at fastmail.fm  Wed Oct  8 06:19:49 2008
From: brong at fastmail.fm (Bron Gondwana)
Date: Wed, 08 Oct 2008 21:19:49 +1100
Subject: Cyrus 2.3.13 RC2
In-Reply-To: <89AB2FFA-9C15-4DB5-8E68-8FDAC9CDD284@uk.fujitsu.com>
References: <48EA2FBE.2000009@andrew.cmu.edu>
	<20081008043606.GA25036@brong.net>
	<89AB2FFA-9C15-4DB5-8E68-8FDAC9CDD284@uk.fujitsu.com>
Message-ID: <1223461189.4822.1278157073@webmail.messagingengine.com>


On Wed, 8 Oct 2008 09:24:49 +0100, "Ian G Batten" <ian.batten at uk.fujitsu.com> said:
> What's the testing status of the SQL backend for cyrusdb?   I'll  
> switch batten.eu.org over to it, but that only has a dozen or so  
> users; ftel.co.uk's 1000+ users might be a little tenser.  I'm keen to  
> switch as the ability to replicate cyrusdb as well as replicating the  
> entire mailsystem is attractive.

You are aware that cyrus replication replicates DB records for all the
important things as well, aren't you?

Bron.
-- 
  Bron Gondwana
  brong at fastmail.fm


From murch at andrew.cmu.edu  Wed Oct  8 06:25:45 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Wed, 08 Oct 2008 06:25:45 -0400
Subject: Cyrus 2.3.13 RC2
In-Reply-To: <89AB2FFA-9C15-4DB5-8E68-8FDAC9CDD284@uk.fujitsu.com>
References: <48EA2FBE.2000009@andrew.cmu.edu>
	<20081008043606.GA25036@brong.net>
	<89AB2FFA-9C15-4DB5-8E68-8FDAC9CDD284@uk.fujitsu.com>
Message-ID: <48EC8AA9.7030500@andrew.cmu.edu>

Ian G Batten wrote:
> 
> On 08 Oct 08, at 0536, Bron Gondwana wrote:
> 
>> On Mon, Oct 06, 2008 at 11:33:18AM -0400, Ken Murchison wrote:
>>> I just put together a second release candidate for Cyrus 2.3.13.  I'd
>>> appreciate any independent testing before I release this to the masses.
>>
>> Sorry about the delay in testing - we've had a few exciting issues
>> here that had to be fixed first.
>>
>>> If there are any outstanding issues that you believe still need to be
>>> addressed in 2.3.13, please let me know.
>>
>> No, it's looking good.  I just removed the patches that have gone into
>> CVS from my build tree and it built fine.  Running on our test server
>> now with no problems.  All the patches that have gone upstream have
>> been running happily on our production machines for a bit too.
>>
>> I think now's a good time to release a 2.3.13.
> 
> What's the testing status of the SQL backend for cyrusdb?   I'll switch 
> batten.eu.org over to it, but that only has a dozen or so users; 
> ftel.co.uk's 1000+ users might be a little tenser.  I'm keen to switch 
> as the ability to replicate cyrusdb as well as replicating the entire 
> mailsystem is attractive.

Minimal.  My boss asked if we could try to get away from BDB and move to 
something like SQlite.  I dusted off an old SQL patch that I wrote while 
working for my old employer and ported it to the 2.3 code.  We set it up 
on a test machine and threw a couple of low volume users on it and the 
campus didn't burn down.  We haven't done any kind of load testing or 
performance testing yet.  That's why its listed as experimental -- use 
at your own risk.


-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From dpc22 at cam.ac.uk  Wed Oct  8 07:06:02 2008
From: dpc22 at cam.ac.uk (David Carter)
Date: Wed, 8 Oct 2008 12:06:02 +0100 (BST)
Subject: THREAD curiosity
Message-ID: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>

If I import the attached Unix mailbox (culled from some real messages from 
an ITEF list) into Cyrus, then the index_thread_compare() call started by:

   . THREAD REFERENCES US-ASCII ALL

segfaults with a NULL pointer deference:

   Program received signal SIGSEGV, Segmentation fault.
   [Switching to Thread 0xb79916d0 (LWP 30378)]
   0x0807b0fe in index_thread_compare (t1=0x817ff60, t2=0x817ffc0,
     call_data=0xbfcc0e28) at index.c:4417
   4417      md2 = t2->msgdata ? t2->msgdata : t2->child->msgdata;

This is because t2->msgdata and t2->child are both NULL.

I infer that the large DKIM-Signature: header blocks are causing a buffer 
overrun somewhere although I can't see where right now. This is true of 
both 2.3.11 (which I am currently running) and a vanilla 2.3.13rc2.

-- 
David Carter                             Email: David.Carter at ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dkim.gz
Type: application/x-gzip
Size: 1388 bytes
Desc: 
Url : http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20081008/9ac10705/attachment.bin 

From murch at andrew.cmu.edu  Wed Oct  8 08:28:09 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Wed, 08 Oct 2008 08:28:09 -0400
Subject: Cyrus 2.3.13 RC2
In-Reply-To: <731A6FB1-C8D2-4B5C-845F-AF80E9EC4982@uk.fujitsu.com>
References: <48EA2FBE.2000009@andrew.cmu.edu>
	<20081008043606.GA25036@brong.net>
	<89AB2FFA-9C15-4DB5-8E68-8FDAC9CDD284@uk.fujitsu.com>
	<1223461189.4822.1278157073@webmail.messagingengine.com>
	<731A6FB1-C8D2-4B5C-845F-AF80E9EC4982@uk.fujitsu.com>
Message-ID: <48ECA759.7030009@andrew.cmu.edu>

Ian G Batten wrote:
> 
> On 08 Oct 08, at 1119, Bron Gondwana wrote:
> 
>>
>> On Wed, 8 Oct 2008 09:24:49 +0100, "Ian G Batten" 
>> <ian.batten at uk.fujitsu.com> said:
>>> What's the testing status of the SQL backend for cyrusdb?   I'll
>>> switch batten.eu.org over to it, but that only has a dozen or so
>>> users; ftel.co.uk's 1000+ users might be a little tenser.  I'm keen to
>>> switch as the ability to replicate cyrusdb as well as replicating the
>>> entire mailsystem is attractive.
>>
>> You are aware that cyrus replication replicates DB records for all the
>> important things as well, aren't you?
> 
> Yes, of course.  It's just that having, many years ago, experienced the 
> loss of a cyrusdb, being able to keep up-to-date copies of it which I 
> can use without the nuclear option of failing over to my off-site 
> replica is a good thing.  So I will shortly have my whole Cyrus instance 
> (~60K mailboxes, ~1000 users, ~4TB of mail) replicated via GigE to a 
> remote site.  But if my local instance went south just because Cyrus DB 
> had gone, being able to simply switch cyrusdb to a MySQL/PostgresQL 
> replica while keeping mail service on the master is preferable to doing 
> a full off-site failover.

We were only looking at SQL to replace BDB (deliver.db, 
tls_sessions.db), because we still think that skiplist is superior from 
mailboxes.db and seen.db.  If you do some heavy testing with the BDB 
code we'd be interested in your results.


-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From brong at fastmail.fm  Wed Oct  8 08:45:42 2008
From: brong at fastmail.fm (Bron Gondwana)
Date: Wed, 8 Oct 2008 23:45:42 +1100
Subject: THREAD curiosity
In-Reply-To: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>
References: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>
Message-ID: <20081008124542.GA18939@brong.net>

On Wed, Oct 08, 2008 at 12:06:02PM +0100, David Carter wrote:
> If I import the attached Unix mailbox (culled from some real messages 
> from an ITEF list) into Cyrus, then the index_thread_compare() call 
> started by:
>
>   . THREAD REFERENCES US-ASCII ALL
>
> segfaults with a NULL pointer deference:
>
>   Program received signal SIGSEGV, Segmentation fault.
>   [Switching to Thread 0xb79916d0 (LWP 30378)]
>   0x0807b0fe in index_thread_compare (t1=0x817ff60, t2=0x817ffc0,
>     call_data=0xbfcc0e28) at index.c:4417
>   4417      md2 = t2->msgdata ? t2->msgdata : t2->child->msgdata;
>
> This is because t2->msgdata and t2->child are both NULL.
>
> I infer that the large DKIM-Signature: header blocks are causing a buffer 
> overrun somewhere although I can't see where right now. This is true of  
> both 2.3.11 (which I am currently running) and a vanilla 2.3.13rc2.

Interesting.  Putting on my "don't let anything derail the release
plans" hat, it's not a regression :)

I don't like things that can segfault Cyrus though... makes me
uncomfortable!  Any chance you can bisect the list until you find
the message which causes the crash?

Bron.

From dpc22 at cam.ac.uk  Wed Oct  8 08:49:43 2008
From: dpc22 at cam.ac.uk (David Carter)
Date: Wed, 8 Oct 2008 13:49:43 +0100 (BST)
Subject: THREAD curiosity
In-Reply-To: <20081008124542.GA18939@brong.net>
References: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>
	<20081008124542.GA18939@brong.net>
Message-ID: <alpine.LSU.2.00.0810081348200.25676@hermes-1.csi.cam.ac.uk>

On Wed, 8 Oct 2008, Bron Gondwana wrote:

> Interesting.  Putting on my "don't let anything derail the release
> plans" hat, it's not a regression :)

I agree.

> I don't like things that can segfault Cyrus though... makes me 
> uncomfortable!  Any chance you can bisect the list until you find the 
> message which causes the crash?

It needs all three messages: this was the minimal test case. If I cut a 
few lines out of one of the DKIM-Signature: lines it starts to work again.

-- 
David Carter                             Email: David.Carter at ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.

From dpc22 at cam.ac.uk  Wed Oct  8 09:11:57 2008
From: dpc22 at cam.ac.uk (David Carter)
Date: Wed, 8 Oct 2008 14:11:57 +0100 (BST)
Subject: THREAD curiosity
In-Reply-To: <alpine.LSU.2.00.0810081348200.25676@hermes-1.csi.cam.ac.uk>
References: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>
	<20081008124542.GA18939@brong.net>
	<alpine.LSU.2.00.0810081348200.25676@hermes-1.csi.cam.ac.uk>
Message-ID: <alpine.LSU.2.00.0810081407100.25676@hermes-1.csi.cam.ac.uk>

On Wed, 8 Oct 2008, David Carter wrote:

> It needs all three messages: this was the minimal test case. If I cut a few 
> lines out of one of the DKIM-Signature: lines it starts to work again.

Think that I've found it:

index.c: index_get_ids()

     /* grab the References header */
     if ((refstr = stristr(headers, "references:"))) {

Isn't a sensible thing to do given the following input:

>From dpc22 at magenta.csi.cam.ac.uk Tue Oct  7 16:23:52 2008 +0100
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
 	d=qualcomm.com; i=hardie at qualcomm.com; q=dns/txt;
 	s=qcdkim; t=1223306812; x=1254842812;
 	h=mime-version:message-id:in-reply-to:references:date:to:
 	from:subject:cc:content-type:x-ironport-av;
 	z=MIME-Version:=201.0|Message-ID:=20<p06240602c50fdb702e5d
 	@[10.227.68.106]>|In-Reply-To:=20<20081006101429.DF8E23A6
 	89D at core3.amsl.com>|References:=20<20081006101429.DF8E23A
 	689D at core3.amsl.com>|Date:=20Mon,=206=20Oct=202008=2008:2
 	6:46=20-0700|To:=20IAB=20Chair=20<iab-chair at ietf.org>,=20
 	<ietf at ietf.org>|From:=20Ted=20Hardie=20<hardie at qualcomm.c
 	om>|Subject:=20Re:=20Call=20for=20Comments:=20Principles
 	=20of=20Internet=20Host=20Configuration|CC:=20"iab at iab.or
 	g"=20<iab at iab.org>|Content-Type:=20text/plain=3B=20charse
 	t=3D"us-ascii"|X-IronPort-AV:=20E=3DMcAfee=3Bi=3D"5300,27
 	77,5398"=3B=20a=3D"10116819";
 	bh=1QWtA80vJrN81vPOqPHundKPVJ/ygOFCJOCyqNmmpjY=;
 	b=Gzi2jfH6WJXXTk0PCI8EAiAM7cGAixb4RaDe9uIkFLz3wtBHqTlfyM9J
 	a5VH3w2b7OjTY6IbsJ11cpC9yJK0A3WsAb6I4gp65WZKeZnfWwu6WvPGR
 	RjPUPcxjqoseCOPettCxdE9eOlAjTJtQXMo0PQGj5QEtMkrP8cqi/WnxK w=;
Message-ID: <p06240602c50fdb702e5d@[10.227.68.106]>
In-Reply-To: <20081006101429.DF8E23A689D at core3.amsl.com>
References: <20081006101429.DF8E23A689D at core3.amsl.com>

The IETF people are probably living on the bleeding edge a bit, but we 
should expect more of these as time goes on.

-- 
David Carter                             Email: David.Carter at ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.

From wes at umich.edu  Wed Oct  8 09:32:32 2008
From: wes at umich.edu (Wesley Craig)
Date: Wed, 8 Oct 2008 09:32:32 -0400
Subject: THREAD curiosity
In-Reply-To: <alpine.LSU.2.00.0810081407100.25676@hermes-1.csi.cam.ac.uk>
References: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>
	<20081008124542.GA18939@brong.net>
	<alpine.LSU.2.00.0810081348200.25676@hermes-1.csi.cam.ac.uk>
	<alpine.LSU.2.00.0810081407100.25676@hermes-1.csi.cam.ac.uk>
Message-ID: <EF972BE2-F6EB-4CA7-8084-356F4253AD86@umich.edu>

Sounds like a blocker to me.

:wes

On 08 Oct 2008, at 09:11, David Carter wrote:
> index.c: index_get_ids()
>
>     /* grab the References header */
>     if ((refstr = stristr(headers, "references:"))) {
>
> Isn't a sensible thing to do given the following input:

From murch at andrew.cmu.edu  Wed Oct  8 18:16:15 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Wed, 08 Oct 2008 18:16:15 -0400
Subject: THREAD curiosity
In-Reply-To: <alpine.LSU.2.00.0810081407100.25676@hermes-1.csi.cam.ac.uk>
References: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>	<20081008124542.GA18939@brong.net>	<alpine.LSU.2.00.0810081348200.25676@hermes-1.csi.cam.ac.uk>
	<alpine.LSU.2.00.0810081407100.25676@hermes-1.csi.cam.ac.uk>
Message-ID: <48ED312F.1000203@andrew.cmu.edu>

David Carter wrote:
> On Wed, 8 Oct 2008, David Carter wrote:
> 
>> It needs all three messages: this was the minimal test case. If I cut 
>> a few lines out of one of the DKIM-Signature: lines it starts to work 
>> again.
> 
> Think that I've found it:
> 
> index.c: index_get_ids()
> 
>     /* grab the References header */
>     if ((refstr = stristr(headers, "references:"))) {
> 
> Isn't a sensible thing to do given the following input:
> 
>> From dpc22 at magenta.csi.cam.ac.uk Tue Oct  7 16:23:52 2008 +0100
> DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
>     d=qualcomm.com; i=hardie at qualcomm.com; q=dns/txt;
>     s=qcdkim; t=1223306812; x=1254842812;
>     h=mime-version:message-id:in-reply-to:references:date:to:
>     from:subject:cc:content-type:x-ironport-av;
>     z=MIME-Version:=201.0|Message-ID:=20<p06240602c50fdb702e5d
>     @[10.227.68.106]>|In-Reply-To:=20<20081006101429.DF8E23A6
>     89D at core3.amsl.com>|References:=20<20081006101429.DF8E23A
>     689D at core3.amsl.com>|Date:=20Mon,=206=20Oct=202008=2008:2
>     6:46=20-0700|To:=20IAB=20Chair=20<iab-chair at ietf.org>,=20
>     <ietf at ietf.org>|From:=20Ted=20Hardie=20<hardie at qualcomm.c
>     om>|Subject:=20Re:=20Call=20for=20Comments:=20Principles
>     =20of=20Internet=20Host=20Configuration|CC:=20"iab at iab.or
>     g"=20<iab at iab.org>|Content-Type:=20text/plain=3B=20charse
>     t=3D"us-ascii"|X-IronPort-AV:=20E=3DMcAfee=3Bi=3D"5300,27
>     77,5398"=3B=20a=3D"10116819";
>     bh=1QWtA80vJrN81vPOqPHundKPVJ/ygOFCJOCyqNmmpjY=;
>     b=Gzi2jfH6WJXXTk0PCI8EAiAM7cGAixb4RaDe9uIkFLz3wtBHqTlfyM9J
>     a5VH3w2b7OjTY6IbsJ11cpC9yJK0A3WsAb6I4gp65WZKeZnfWwu6WvPGR
>     RjPUPcxjqoseCOPettCxdE9eOlAjTJtQXMo0PQGj5QEtMkrP8cqi/WnxK w=;
> Message-ID: <p06240602c50fdb702e5d@[10.227.68.106]>
> In-Reply-To: <20081006101429.DF8E23A689D at core3.amsl.com>
> References: <20081006101429.DF8E23A689D at core3.amsl.com>

Changing it to the following fixes it as long as References isn't the 
first header field:

if ((refstr = stristr(headers, "\r\nreferences:"))) {


If we're worried about it being the first header field, there is an easy 
test for that.  Comments?

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From murch at andrew.cmu.edu  Wed Oct  8 19:30:03 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Wed, 08 Oct 2008 19:30:03 -0400
Subject: Cyrus 2.3.13 RC1
In-Reply-To: <20080925195904.GA28965@exuma.irbs.com>
References: <48DB78C7.9050903@andrew.cmu.edu>
	<20080925195904.GA28965@exuma.irbs.com>
Message-ID: <48ED427B.5020304@andrew.cmu.edu>

John Capo wrote:
> Quoting Ken Murchison (murch at andrew.cmu.edu):
>> I just put together a release candidate for Cyrus 2.3.13.  I'd
>> appreciate any independent testing before I release this to the masses.
>>
>> http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc1.tar.gz
>>
>>
>> Noteworthy changes:
>>
>> * Added an experimental "sql" backend for cyrusdb.  Currently MySQL,
>>   PostgreSQL, and SQLite are supported.
>> * Added support for IMAP [CAPABILITY] response code to client-side
>>   of Murder proxies.
>> * Added support for ManageSieve auto-capability response after
>>   STARTTLS and after AUTH with a SASL security layer.
>> * Made MAXWORD and MAXQUOTED sizes configurable via imapd.conf
>> * Rewrote cyrusdb_quotalegacy.c to use readir()
>>   rather than glob.c.  This avoids a potential crash due to
>>   conflicts between glibc and Heimdal implementations of glob().
>> * Added support for fulldirhash to 'ctl_mboxlist -v'
>> * Several skiplist bugfixes.
>>
>> Check doc/changes.html for a complete list of changes.
>>
>> If there are any outstanding issues that you believe still need to be
>> addressed in 2.3.13, please let me know.
> 
> 
> statuscache_db.c assumes an off_t is 32bits.
> 
>     if (p < dend) scdata->index_size = strtoul(p, &p, 10);
> 
> and in the printf that generates the string for the DB.


Fixed in CVS using a OFF_T_FMT macro and a strtoofft() macro.  Will be 
in 2.3.13rc3

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From wes at umich.edu  Wed Oct  8 19:57:06 2008
From: wes at umich.edu (Wesley Craig)
Date: Wed, 8 Oct 2008 19:57:06 -0400
Subject: Cyrus 2.3.13 RC1
In-Reply-To: <alpine.DEB.1.10.0810030921480.14167@sancho.ccd.uniroma2.it>
References: <48DB78C7.9050903@andrew.cmu.edu>
	<20080925195904.GA28965@exuma.irbs.com>
	<48E4D6D2.4080005@andrew.cmu.edu>
	<alpine.DEB.1.10.0810021645170.5347@sancho.ccd.uniroma2.it>
	<7B160101-4FBC-49AF-B4A6-00845AB636D5@umich.edu>
	<alpine.DEB.1.10.0810030921480.14167@sancho.ccd.uniroma2.it>
Message-ID: <5EFFD01D-5432-4AD5-8D6B-DCB48797816F@umich.edu>

On 03 Oct 2008, at 07:19, Lorenzo M. Catucci wrote:
> As for bugzilla, the bug-tracker should be better advertised both on
> the website and in released documentation;

I agree.  I'd really like it if the main cyrus page was just a  
framework to get to bugzilla, the wiki, the mailing lists, and the  
download pages.  The static pages are not really serving the cyrus  
community very well, probably they never have.  But if users read &  
contributed to the bugzilla, the wiki, and the mailing list (I feel  
like I'm missing something else, but I can't think of it), then we'd  
be doing much better.

> I think some summary in the
> changelogs like "fixes xxx bugzilla bugs and introduces yyy bugzilla
> enhancements" would be nice to show...

As a separate topic, I agree.  The converse is also true: close  
bugzilla entries should refer to CVS diffs.  Personally, my CVS  
commits read:

	what was changed
	who was involved in the change
	bugzilla number

And my closed bugzilla entries refer back to the CVS-web diff URLs.   
Perhaps we could agree on a methodology, here...

:wes

From dpc22 at cam.ac.uk  Thu Oct  9 03:59:28 2008
From: dpc22 at cam.ac.uk (David Carter)
Date: Thu, 9 Oct 2008 08:59:28 +0100 (BST)
Subject: THREAD curiosity
In-Reply-To: <48ED312F.1000203@andrew.cmu.edu>
References: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>
	<20081008124542.GA18939@brong.net>
	<alpine.LSU.2.00.0810081348200.25676@hermes-1.csi.cam.ac.uk>
	<alpine.LSU.2.00.0810081407100.25676@hermes-1.csi.cam.ac.uk>
	<48ED312F.1000203@andrew.cmu.edu>
Message-ID: <alpine.LSU.2.00.0810090846190.5441@hermes-1.csi.cam.ac.uk>

On Wed, 8 Oct 2008, Ken Murchison wrote:

> Changing it to the following fixes it as long as References isn't the first 
> header field:
>
> if ((refstr = stristr(headers, "\r\nreferences:"))) {
>
> If we're worried about it being the first header field, there is an easy test 
> for that.  Comments?

That would certainly be better than the current situation.

Is there a particular reason that we can't use index_pruneheader()? It 
does require a bit of scratch space. However we could do something like 
index_readheader().

-- 
David Carter                             Email: David.Carter at ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.

From murch at andrew.cmu.edu  Thu Oct  9 06:54:54 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Thu, 09 Oct 2008 06:54:54 -0400
Subject: THREAD curiosity
In-Reply-To: <alpine.LSU.2.00.0810090846190.5441@hermes-1.csi.cam.ac.uk>
References: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>	<20081008124542.GA18939@brong.net>	<alpine.LSU.2.00.0810081348200.25676@hermes-1.csi.cam.ac.uk>	<alpine.LSU.2.00.0810081407100.25676@hermes-1.csi.cam.ac.uk>	<48ED312F.1000203@andrew.cmu.edu>
	<alpine.LSU.2.00.0810090846190.5441@hermes-1.csi.cam.ac.uk>
Message-ID: <48EDE2FE.7030701@andrew.cmu.edu>

David Carter wrote:
> On Wed, 8 Oct 2008, Ken Murchison wrote:
> 
>> Changing it to the following fixes it as long as References isn't the 
>> first header field:
>>
>> if ((refstr = stristr(headers, "\r\nreferences:"))) {
>>
>> If we're worried about it being the first header field, there is an 
>> easy test for that.  Comments?
> 
> That would certainly be better than the current situation.
> 
> Is there a particular reason that we can't use index_pruneheader()? It 
> does require a bit of scratch space.

Mainly because its destructive, but it also seemed like overkill.  We're 
working directly with the cache entry, so changing its contents isn't 
cool.  I supposed we could malloc() a working copy.

> However we could do something like index_readheader().

Do mean in the sense that we work with the actual message, or just carve 
out some static scratch space?

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From murch at andrew.cmu.edu  Thu Oct  9 07:40:06 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Thu, 09 Oct 2008 07:40:06 -0400
Subject: THREAD curiosity
In-Reply-To: <alpine.LSU.2.00.0810090846190.5441@hermes-1.csi.cam.ac.uk>
References: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>
	<20081008124542.GA18939@brong.net>
	<alpine.LSU.2.00.0810081348200.25676@hermes-1.csi.cam.ac.uk>
	<alpine.LSU.2.00.0810081407100.25676@hermes-1.csi.cam.ac.uk>
	<48ED312F.1000203@andrew.cmu.edu>
	<alpine.LSU.2.00.0810090846190.5441@hermes-1.csi.cam.ac.uk>
Message-ID: <48EDED96.6020506@andrew.cmu.edu>

David Carter wrote:
> On Wed, 8 Oct 2008, Ken Murchison wrote:
> 
>> Changing it to the following fixes it as long as References isn't the 
>> first header field:
>>
>> if ((refstr = stristr(headers, "\r\nreferences:"))) {
>>
>> If we're worried about it being the first header field, there is an 
>> easy test for that.  Comments?
> 
> That would certainly be better than the current situation.
> 
> Is there a particular reason that we can't use index_pruneheader()? It 
> does require a bit of scratch space. However we could do something like 
> index_readheader().
> 

How does this look?


--- index.c	30 Sep 2008 13:06:06 -0400	1.245
+++ index.c	09 Oct 2008 07:37:19 -0400	
@@ -173,7 +173,7 @@
  static char *index_extract_subject(const char *subj, size_t len, int 
*is_refwd);
  static char *_index_extract_subject(char *s, int *is_refwd);
  static void index_get_ids(MsgData *msgdata,
-			  char *envtokens[], const char *headers);
+			  char *envtokens[], const char *headers, unsigned size);
  static MsgData *index_msgdata_load(unsigned *msgno_list, int n,
  				   struct sortcrit *sortcrit);

@@ -3724,7 +3724,8 @@
   		cur->nannot++;
   		break;
  	    case LOAD_IDS:
-		index_get_ids(cur, envtokens, headers + CACHE_ITEM_SIZE_SKIP);
+		index_get_ids(cur, envtokens, headers + CACHE_ITEM_SIZE_SKIP,
+			      CACHE_ITEM_LEN(headers));
  		break;
  	    }
  	}
@@ -4101,27 +4102,42 @@
  }

  /* Get message-id, and references/in-reply-to */
-#define REFGROWSIZE 10
+#define REFGROWSIZE 20

-void index_get_ids(MsgData *msgdata, char *envtokens[], const char 
*headers)
+void index_get_ids(MsgData *msgdata, char *envtokens[], const char 
*headers,
+		   unsigned size)
  {
+    static char *buf;
+    static unsigned bufsize;
+    static struct strlist refhdr;
      char *refstr, *ref, *in_reply_to;
      int refsize = REFGROWSIZE;
-    char buf[100];
+
+    if (bufsize < size+2) {
+	bufsize = size+100;
+	buf = xrealloc(buf, bufsize);
+    }

      /* get msgid */
      msgdata->msgid = find_msgid(envtokens[ENV_MSGID], NULL);
       /* if we don't have one, create one */
      if (!msgdata->msgid) {
-	snprintf(buf, sizeof(buf), "<Empty-ID: %u>", msgdata->msgno);
+	snprintf(buf, bufsize, "<Empty-ID: %u>", msgdata->msgno);
  	msgdata->msgid = xstrdup(buf);
      }

+    /* Copy headers to the buffer */
+    memcpy(buf, headers, size);
+    buf[size] = '\0';
+
      /* grab the References header */
-    if ((refstr = stristr(headers, "references:"))) {
+    refhdr.s = "references";
+    index_pruneheader(buf, &refhdr, 0);
+    if (*buf) {
  	/* allocate some space for refs */
  	msgdata->ref = (char **) xmalloc(refsize * sizeof(char *));
  	/* find references */
+	refstr = buf;
  	while ((ref = find_msgid(refstr, &refstr)) != NULL) {
  	    /* reallocate space for this msgid if necessary */
  	    if (msgdata->nref == refsize) {


-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From dpc22 at cam.ac.uk  Thu Oct  9 07:48:59 2008
From: dpc22 at cam.ac.uk (David Carter)
Date: Thu, 9 Oct 2008 12:48:59 +0100 (BST)
Subject: THREAD curiosity
In-Reply-To: <48EDED96.6020506@andrew.cmu.edu>
References: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>
	<20081008124542.GA18939@brong.net>
	<alpine.LSU.2.00.0810081348200.25676@hermes-1.csi.cam.ac.uk>
	<alpine.LSU.2.00.0810081407100.25676@hermes-1.csi.cam.ac.uk>
	<48ED312F.1000203@andrew.cmu.edu>
	<alpine.LSU.2.00.0810090846190.5441@hermes-1.csi.cam.ac.uk>
	<48EDED96.6020506@andrew.cmu.edu>
Message-ID: <alpine.LSU.2.00.0810091242060.5441@hermes-1.csi.cam.ac.uk>

On Thu, 9 Oct 2008, Ken Murchison wrote:

> How does this look?

That is what I had in mind.

In particular we are no longer running stristr() on a mmap()ed file and 
praying we find a '\0' before we run out of file. While we got away with 
this because of the following CACHE_ITEMs, this has bitten me in the past.

-- 
David Carter                             Email: David.Carter at ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.

From murch at andrew.cmu.edu  Thu Oct  9 08:05:31 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Thu, 09 Oct 2008 08:05:31 -0400
Subject: THREAD curiosity
In-Reply-To: <alpine.LSU.2.00.0810091242060.5441@hermes-1.csi.cam.ac.uk>
References: <alpine.LSU.2.00.0810081153340.21030@hermes-1.csi.cam.ac.uk>
	<20081008124542.GA18939@brong.net>
	<alpine.LSU.2.00.0810081348200.25676@hermes-1.csi.cam.ac.uk>
	<alpine.LSU.2.00.0810081407100.25676@hermes-1.csi.cam.ac.uk>
	<48ED312F.1000203@andrew.cmu.edu>
	<alpine.LSU.2.00.0810090846190.5441@hermes-1.csi.cam.ac.uk>
	<48EDED96.6020506@andrew.cmu.edu>
	<alpine.LSU.2.00.0810091242060.5441@hermes-1.csi.cam.ac.uk>
Message-ID: <48EDF38B.5060406@andrew.cmu.edu>

David Carter wrote:
> On Thu, 9 Oct 2008, Ken Murchison wrote:
> 
>> How does this look?
> 
> That is what I had in mind.
> 
> In particular we are no longer running stristr() on a mmap()ed file and 
> praying we find a '\0' before we run out of file. While we got away with 
> this because of the following CACHE_ITEMs, this has bitten me in the past.

I need to do something similar for index_overview() and then I'll commit 
the fix.


-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From murch at andrew.cmu.edu  Thu Oct  9 10:57:33 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Thu, 09 Oct 2008 10:57:33 -0400
Subject: Cyrus 2.3.13 RC3
Message-ID: <48EE1BDD.8010504@andrew.cmu.edu>

I just put together a third and hopefully FINAL release candidate for 
Cyrus 2.3.13.  I'd appreciate any independent testing before I release 
this to the masses.

http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc3.tar.gz
http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.13rc3.tar.gz.sig


Noteworthy changes:

* Added an experimental "sql" backend for cyrusdb.  Currently MySQL,
   PostgreSQL, and SQLite are supported.
* Added support for IMAP [CAPABILITY] response code to client-side
   of Murder proxies.
* Added support for ManageSieve auto-capability response after
   STARTTLS and after AUTH with a SASL security layer.
* Made MAXWORD and MAXQUOTED sizes configurable via imapd.conf
* Rewrote cyrusdb_quotalegacy.c to use readir()
   rather than glob.c.  This avoids a potential crash due to
   conflicts between glibc and Heimdal implementations of glob().
* Added support for fulldirhash to 'ctl_mboxlist -v'
* Several skiplist transaction bugfixes.
* cyr_expire no longer has a default of 0 (zero) for -X and -D.
   These options must be used explicitly in order to have the desired
   effect.
* Added sieve_utf8fileinto option.
* Added sieve_sasl_send_unsolicited_capability and
   sieve_sasl_expect_unsolicited_capability options.
* Several 32/64-bit compatibility fixes.


Check doc/changes.html and doc/install-upgrade.html for a complete list 
of changes.

If there are any outstanding critical issues that you believe still need 
to be addressed in 2.3.13, please let me know.  This code has been in 
feature freeze for a while, so no new requests please.

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University







From brong at fastmail.fm  Thu Oct  9 20:28:23 2008
From: brong at fastmail.fm (Bron Gondwana)
Date: Fri, 10 Oct 2008 11:28:23 +1100
Subject: Cyrus 2.3.13 RC3
In-Reply-To: <48EE1BDD.8010504@andrew.cmu.edu>
References: <48EE1BDD.8010504@andrew.cmu.edu>
Message-ID: <20081010002823.GA8870@brong.net>

On Thu, Oct 09, 2008 at 10:57:33AM -0400, Ken Murchison wrote:
> I just put together a third and hopefully FINAL release candidate for  
> Cyrus 2.3.13.  I'd appreciate any independent testing before I release  
> this to the masses.

Looks good.  The syslog deleted change broke the fastrename patch, so I
had to re-build it, but otherwise applied fine and is working fine on
our testbed.

Bron.

From jc at irbs.com  Fri Oct 10 15:33:43 2008
From: jc at irbs.com (John Capo)
Date: Fri, 10 Oct 2008 15:33:43 -0400
Subject: Cyrus 2.3.13 RC3
In-Reply-To: <48EE1BDD.8010504@andrew.cmu.edu>
References: <48EE1BDD.8010504@andrew.cmu.edu>
Message-ID: <20081010193343.GA14236@exuma.irbs.com>

Quoting Ken Murchison (murch at andrew.cmu.edu):
> I just put together a third and hopefully FINAL release candidate for 
> Cyrus 2.3.13.  I'd appreciate any independent testing before I release 
> this to the masses.
> 

Up and running on a FreeBSD 6.3+ test server.  All of the 64bit
issues I have had trouble with are fixed.




From bawood at umich.edu  Wed Oct 15 13:53:24 2008
From: bawood at umich.edu (Brian Awood)
Date: Wed, 15 Oct 2008 13:53:24 -0400
Subject: ctl_mboxlist with unified mupdate config
Message-ID: <200810151353.24512.bawood@umich.edu>

I've been looking into issues that ctl_mboxlist has with the unified 
mupdate config.  One thing we've noticed after upgrading our proxy 
servers to 2.3 running in unified mode is that running "ctl_mboxlist -m" 
was quite slow.  We do have over 1M mailboxes, but it can take ~6-7min to 
run which seemed rather long even for that many mailboxes.  

The problem stems from when ctl_mboxlist gets it's list of mailboxes that 
the mupdate master thinks it should have.  It uses the config_servername 
as it's prefix arg to the LIST command.  Since the proxy machine doesn't 
host any mailboxes, the mupdate master doesn't return anything.  This 
ends up causing ctl_mboxlist to do a individual FIND for each mailbox in 
it's local copy of the db.  It seems like the correct behavior for 
ctl_mboxlist in a unified mupdate config would be for it to do LIST 
without args so it gets back all of the mailboxes that the mupdate master 
has.  I tested this change and it brought the run time down to ~30sec.

This has some side effects though, since ctl_mboxlist generally assumes 
that mailboxes listed in the local database are hosted locally and 
mailboxes from mupdate that aren't listed locally should be removed from 
mupdate.  So for this change to be made, additional checks would need to 
be added to determine if the mailbox is local or remote and then "do the 
right thing".  It seems like this could generally be described as the 
local database is only authoritative if the mailbox is local, otherwise 
the mupdate master is authoritative.  

Are there any thoughts on what the "right thing" is or how ctl_mboxlist 
should behave in a unified config in general?  I'll open a report in 
bugzilla with a patch, but I'd like to get an idea of what other people 
think or if anyone else is even using/interested in unified mupdate. 

---
Brian Awood
University of Michigan
ITCS

From murch at andrew.cmu.edu  Wed Oct 15 13:55:37 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Wed, 15 Oct 2008 13:55:37 -0400
Subject: Upcoming Cyrus 2.3.13 Release
Message-ID: <48F62E99.4050906@andrew.cmu.edu>

Folks,

I'm planning on releasing 2.3.13 late Friday afternoon EDT unless 
someone reports a bug that we would consider a blocker.  So, if there is 
anyone planning on doing testing on RC3, please do it soon.

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University


From wes at umich.edu  Wed Oct 15 14:18:51 2008
From: wes at umich.edu (Wesley Craig)
Date: Wed, 15 Oct 2008 14:18:51 -0400
Subject: ctl_mboxlist with unified mupdate config
In-Reply-To: <200810151353.24512.bawood@umich.edu>
References: <200810151353.24512.bawood@umich.edu>
Message-ID: <0F241781-F4FC-485E-8BF2-3ED4433336C5@umich.edu>

On 15 Oct 2008, at 13:53, Brian Awood wrote:
> It seems like this could generally be described as the
> local database is only authoritative if the mailbox is local,  
> otherwise
> the mupdate master is authoritative.

That's more or less what I'd like to see happen.  I'd like  
ctl_mboxlist (regardless of the unified setting) to be able to fully  
populate mailboxes.db on a frontend.  I'd rather not have to bring up  
all of cyrus to get a reasonable mailbox list on a machines.

:wes

From brong at fastmail.fm  Wed Oct 15 19:53:00 2008
From: brong at fastmail.fm (Bron Gondwana)
Date: Thu, 16 Oct 2008 10:53:00 +1100
Subject: Upcoming Cyrus 2.3.13 Release
In-Reply-To: <48F65971.8030008@simutronics.com>
References: <48F62E99.4050906@andrew.cmu.edu>
	<48F65971.8030008@simutronics.com>
Message-ID: <20081015235300.GC8495@brong.net>

On Wed, Oct 15, 2008 at 03:58:25PM -0500, James Miller wrote:
> Good God man, have you never heard of the axiom NOTING NEW ON FIRDAY?  
> You're tempting the gods and fates.

You could just disable your "automatically download and install the
latest release from cmu without testing it" script for the weekend.

Ho hum.

Bron.

From murch at andrew.cmu.edu  Mon Oct 20 09:02:38 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Mon, 20 Oct 2008 09:02:38 -0400
Subject: Cyrus IMAPd 2.3.13 Released
Message-ID: <48FC816E.1090002@andrew.cmu.edu>

I am pleased to announce the release of Cyrus IMAPd 2.3.13.  This
release should be considered production quality.


Noteworthy changes:

* Added an experimental "sql" backend for cyrusdb.  Currently MySQL,
   PostgreSQL, and SQLite are supported.
* Added support for IMAP [CAPABILITY] response code to client-side
   of Murder proxies.
* Added support for ManageSieve auto-capability response after
   STARTTLS and after AUTH with a SASL security layer.
* Made MAXWORD and MAXQUOTED sizes configurable via imapd.conf
* Rewrote cyrusdb_quotalegacy.c to use readir()
   rather than glob.c.  This avoids a potential crash due to
   conflicts between glibc and Heimdal implementations of glob().
* Added support for fulldirhash to 'ctl_mboxlist -v'
* Several skiplist transaction bugfixes.
* cyr_expire no longer has a default of 0 (zero) for -X and -D.
   These options must be used explicitly in order to have the desired
   effect.
* Added sieve_utf8fileinto option.
* Added sieve_sasl_send_unsolicited_capability and
   sieve_sasl_expect_unsolicited_capability options.
* Several 32/64-bit compatibility fixes.


For full details, please see doc/changes.html and
doc/install-upgrade.html which are included in the distribution.

URLs for this release:
ftp://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.13.tar.gz
or
http://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.13.tar.gz


Questions and comments can be directed to
info-cyrus at lists.andrew.cmu.edu (public list), or cyrus-bugs at andrew.cmu.edu.

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University













From hdembkowski at alcatel-lucent.com  Mon Oct 20 11:02:38 2008
From: hdembkowski at alcatel-lucent.com (DEMBKOWSKI, Henryk (Henryk))
Date: Mon, 20 Oct 2008 17:02:38 +0200
Subject: Problem with stage
Message-ID: <B576524D0905194499F90736CAA2758A01F17F45@DEEXC1U03.de.lucent.com>

All,

We saw problem with Cyrus. After problems with disks, file's system has
been "damaged".
And from some reasons instead of directories we had files with the same
name.
For example: 

Oct  2 02:21:13 imap[7264]: couldn't create stage directory:
/mbox/vm0/stage./: File exists
Oct  2 02:21:13 imap[7264]: IOERROR: creating message file
/mbox/vm0/stage./7264-1222928473-1: File exists

Due to this error IMAP process doesn't work. Is it OK that it works in
such way? Maybe we should remove file
and create directory? What do you think?

Kind Regards,
Henryk Dembkowski

From brong at fastmail.fm  Mon Oct 20 17:48:59 2008
From: brong at fastmail.fm (Bron Gondwana)
Date: Tue, 21 Oct 2008 08:48:59 +1100
Subject: Problem with stage
In-Reply-To: <B576524D0905194499F90736CAA2758A01F17F45@DEEXC1U03.de.lucent.com>
References: <B576524D0905194499F90736CAA2758A01F17F45@DEEXC1U03.de.lucent.com>
Message-ID: <1224539339.3782.1280319483@webmail.messagingengine.com>


On Mon, 20 Oct 2008 17:02:38 +0200, "DEMBKOWSKI, Henryk (Henryk)" <hdembkowski at alcatel-lucent.com> said:
> All,
> 
> We saw problem with Cyrus. After problems with disks, file's system has
> been "damaged".
> And from some reasons instead of directories we had files with the same
> name.
> For example: 
> 
> Oct  2 02:21:13 imap[7264]: couldn't create stage directory:
> /mbox/vm0/stage./: File exists
> Oct  2 02:21:13 imap[7264]: IOERROR: creating message file
> /mbox/vm0/stage./7264-1222928473-1: File exists
> 
> Due to this error IMAP process doesn't work. Is it OK that it works in
> such way? Maybe we should remove file
> and create directory? What do you think?

While cyrus is stopped, it's quite safe to remove the stage. directory (or file!)

Bron.
-- 
  Bron Gondwana
  brong at fastmail.fm


From brong at fastmail.fm  Fri Oct 24 19:01:35 2008
From: brong at fastmail.fm (Bron Gondwana)
Date: Sat, 25 Oct 2008 10:01:35 +1100
Subject: Cyrus IMAPd 2.3.13 Released
In-Reply-To: <48FC816E.1090002@andrew.cmu.edu>
References: <48FC816E.1090002@andrew.cmu.edu>
Message-ID: <20081024230135.GA12776@brong.net>

On Mon, Oct 20, 2008 at 09:02:38AM -0400, Ken Murchison wrote:
> I am pleased to announce the release of Cyrus IMAPd 2.3.13.  This
> release should be considered production quality.
> 
> 
> Noteworthy changes:
> 
> * Added an experimental "sql" backend for cyrusdb.  Currently MySQL,
>    PostgreSQL, and SQLite are supported.
> * Added support for IMAP [CAPABILITY] response code to client-side
>    of Murder proxies.
> * Added support for ManageSieve auto-capability response after
>    STARTTLS and after AUTH with a SASL security layer.
> * Made MAXWORD and MAXQUOTED sizes configurable via imapd.conf
> * Rewrote cyrusdb_quotalegacy.c to use readir()
>    rather than glob.c.  This avoids a potential crash due to
>    conflicts between glibc and Heimdal implementations of glob().
> * Added support for fulldirhash to 'ctl_mboxlist -v'
> * Several skiplist transaction bugfixes.
> * cyr_expire no longer has a default of 0 (zero) for -X and -D.
>    These options must be used explicitly in order to have the desired
>    effect.
> * Added sieve_utf8fileinto option.
> * Added sieve_sasl_send_unsolicited_capability and
>    sieve_sasl_expect_unsolicited_capability options.
> * Several 32/64-bit compatibility fixes.

  * Changed the list of valid characters in mailbox names, which is
	  interesting because it means that when you upgrade a master and
		not the associated replica, and then someone creates a mailbox
		with those characters and replication breaks.  Bah.

Bron ( at least the fix was easy! Just upgrade the replica )

From maxim.gorbachyov at gmail.com  Tue Oct 28 09:37:22 2008
From: maxim.gorbachyov at gmail.com (Maxim Gorbachyov)
Date: Tue, 28 Oct 2008 16:37:22 +0300
Subject: patch: fix sql plugin crash
Message-ID: <291f35090810280637i6e0d7336ybdfb3f56a76896ba@mail.gmail.com>

Hello.
There is a bug in the sql plugin of the cyrus-sasl library.
First, description. Here is relevant piece from my config:

pwcheck_method: auxprop
auxprop_plugin: sql
sql_engine: sqlite
sql_database: my.db
sql_select: SELECT pwd FROM users WHERE login = '%u'
sql_verbose: yes

I get SEGFAULTs when user with unknown login attempts to authorise.
I've found that sql_auxprop_lookup() uses 'value' and (most important)
'value_len' without previous initialization. Sqlite backend does not
change value_len if nothing was read from db. So we get
utils->prop_set() accessing random memory.

Proposed patch (see attachment) initializes 'value' and 'value_len'
with zeroes. It works for me.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sql-plugin-fix-crash-on-unknown-user
Type: application/octet-stream
Size: 577 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20081028/a5505d02/attachment.obj 

From selsky at columbia.edu  Tue Oct 28 09:58:28 2008
From: selsky at columbia.edu (Matt Selsky)
Date: Tue, 28 Oct 2008 09:58:28 -0400
Subject: patch: fix sql plugin crash
In-Reply-To: <291f35090810280637i6e0d7336ybdfb3f56a76896ba@mail.gmail.com>
References: <291f35090810280637i6e0d7336ybdfb3f56a76896ba@mail.gmail.com>
Message-ID: <20081028135828.GF23722@columbia.edu>

On Tue, Oct 28, 2008 at 04:37:22PM +0300, Maxim Gorbachyov wrote:
> Hello.
> There is a bug in the sql plugin of the cyrus-sasl library.
> First, description. Here is relevant piece from my config:
> 
> pwcheck_method: auxprop
> auxprop_plugin: sql
> sql_engine: sqlite
> sql_database: my.db
> sql_select: SELECT pwd FROM users WHERE login = '%u'
> sql_verbose: yes
> 
> I get SEGFAULTs when user with unknown login attempts to authorise.
> I've found that sql_auxprop_lookup() uses 'value' and (most important)
> 'value_len' without previous initialization. Sqlite backend does not
> change value_len if nothing was read from db. So we get
> utils->prop_set() accessing random memory.
> 
> Proposed patch (see attachment) initializes 'value' and 'value_len'
> with zeroes. It works for me.

Please add this to bugzilla so we don't lose track of it.


-- 
Matt

From maxim.gorbachyov at gmail.com  Tue Oct 28 10:51:40 2008
From: maxim.gorbachyov at gmail.com (Maxim Gorbachyov)
Date: Tue, 28 Oct 2008 17:51:40 +0300
Subject: patch: add sqlite3 backend support to the cyrus-sasl sql plugin
Message-ID: <291f35090810280751w7f05df0eo520a3a849d0a0090@mail.gmail.com>

Hello.

Proposed patch (see attachment) adds sqlite3 backend support to the cyrus-sasl
sql plugin. The patch does not interfere with present sqlite(v2) code. Build
infrastructure is updated (tested on Debian Sid and on Ubuntu Hardy), but this
update does not address win32 systems.

It seems like this patch is a solution for:
https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2754

I've added the patch to bugzilla:
https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3112
-------------- next part --------------
A non-text attachment was scrubbed...
Name: add-sqlite3-support
Type: application/octet-stream
Size: 7036 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20081028/343eb6e6/attachment.obj 

From alexey.melnikov at isode.com  Tue Oct 28 15:32:24 2008
From: alexey.melnikov at isode.com (Alexey Melnikov)
Date: Tue, 28 Oct 2008 19:32:24 +0000
Subject: patch: fix sql plugin crash
In-Reply-To: <291f35090810280637i6e0d7336ybdfb3f56a76896ba@mail.gmail.com>
References: <291f35090810280637i6e0d7336ybdfb3f56a76896ba@mail.gmail.com>
Message-ID: <490768C8.1030306@isode.com>

Maxim Gorbachyov wrote:

>Hello.
>There is a bug in the sql plugin of the cyrus-sasl library.
>First, description. Here is relevant piece from my config:
>
>pwcheck_method: auxprop
>auxprop_plugin: sql
>sql_engine: sqlite
>sql_database: my.db
>sql_select: SELECT pwd FROM users WHERE login = '%u'
>sql_verbose: yes
>
>I get SEGFAULTs when user with unknown login attempts to authorise.
>I've found that sql_auxprop_lookup() uses 'value' and (most important)
>'value_len' without previous initialization. Sqlite backend does not
>change value_len if nothing was read from db. So we get
>utils->prop_set() accessing random memory.
>
>Proposed patch (see attachment) initializes 'value' and 'value_len'
>with zeroes. It works for me.
>  
>
This looks reasonable.

However, I am wondering if the following line:

+    memset(value, 0, sizeof(value));

can be replaced with:

+    value[0] = '\0';

Can you test?

Also, please add this to bugzilla, if you haven't yet.



From maxim.gorbachyov at gmail.com  Wed Oct 29 04:59:58 2008
From: maxim.gorbachyov at gmail.com (Maxim Gorbachyov)
Date: Wed, 29 Oct 2008 11:59:58 +0300
Subject: patch: fix sql plugin crash
In-Reply-To: <490768C8.1030306@isode.com>
References: <291f35090810280637i6e0d7336ybdfb3f56a76896ba@mail.gmail.com>
	<490768C8.1030306@isode.com>
Message-ID: <291f35090810290159x2d3e213q5223bc8aafb912de@mail.gmail.com>

On Tue, Oct 28, 2008 at 10:32 PM, Alexey Melnikov
<alexey.melnikov at isode.com> wrote:

> This looks reasonable.
>
> However, I am wondering if the following line:
>
> +    memset(value, 0, sizeof(value));
>
> can be replaced with:
>
> +    value[0] = '\0';
>
> Can you test?

Tested. It works.

> Also, please add this to bugzilla, if you haven't yet.

That's done, but with initial version of the patch attached:
http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3111