From murch at andrew.cmu.edu Wed Sep 9 09:47:14 2009 From: murch at andrew.cmu.edu (Ken Murchison) Date: Wed, 09 Sep 2009 09:47:14 -0400 Subject: Cyrus IMAPd 2.2.13p1 & 2.3.15 Released Message-ID: <4AA7B1E2.9010507@andrew.cmu.edu> I'd like to announce the releases of Cyrus IMAPd 2.2.13p1 and 2.3.15. These releases should both be considered production quality. These releases are being made at this time to fix the potential buffer overflow vulnerability described in CERT VU#336053: http://www.kb.cert.org/vuls/id/336053 The 2.2.13p1 release is no different from 2.2.13 other than the buffer overflow fix. The 2.3.15 release contains several other non-critical bugfixes and feature enhancements. For full details, please see doc/changes.html and doc/install-upgrade.html which are included in the distribution. I'd personally like to thank Bron Gondwana of Fastmail.fm for finding and fixing the buffer overflow, as well as his numerous other contributions to the 2.3.15 release. URLs for these releases: ftp://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.2.13p1.tar.gz ftp://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.15.tar.gz or http://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.2.13p1.tar.gz http://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.15.tar.gz Questions and comments can be directed to info-cyrus at lists.andrew.cmu.edu (public list), or cyrus-bugs at andrew.cmu.edu. -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University From thomas.jarosch at intra2net.com Wed Sep 9 10:03:19 2009 From: thomas.jarosch at intra2net.com (Thomas Jarosch) Date: Wed, 9 Sep 2009 16:03:19 +0200 Subject: Cyrus IMAPd 2.2.13p1 & 2.3.15 Released In-Reply-To: <4AA7B1E2.9010507@andrew.cmu.edu> References: <4AA7B1E2.9010507@andrew.cmu.edu> Message-ID: <200909091603.19312.thomas.jarosch@intra2net.com> On Wednesday, 9. September 2009 15:47:14 Ken Murchison wrote: > I'd like to announce the releases of Cyrus IMAPd 2.2.13p1 and 2.3.15. > These releases should both be considered production quality. These > releases are being made at this time to fix the potential buffer > overflow vulnerability described in CERT VU#336053: > http://www.kb.cert.org/vuls/id/336053 Thanks for the new release! Regarding the buffer overflow: The cert website currently outputs a "Lotus Notes exception". Is the overflow theoretically exploitable via a malicious email or does a user need to upload a malicious sieve script? Cheers, Thomas From duncan.gibb at siriusit.co.uk Wed Sep 9 12:35:55 2009 From: duncan.gibb at siriusit.co.uk (Duncan Gibb) Date: Wed, 09 Sep 2009 17:35:55 +0100 Subject: Cyrus IMAPd 2.2.13p1 & 2.3.15 Released In-Reply-To: <200909091603.19312.thomas.jarosch@intra2net.com> References: <4AA7B1E2.9010507@andrew.cmu.edu> <200909091603.19312.thomas.jarosch@intra2net.com> Message-ID: <4AA7D96B.701@siriusit.co.uk> Thomas Jarosch wrote: KM> I'd like to announce the releases of Cyrus IMAPd 2.2.13p1 and 2.3.15. KM> These releases should both be considered production quality. These KM> releases are being made at this time to fix the potential buffer KM> overflow vulnerability described in CERT VU#336053: KM> http://www.kb.cert.org/vuls/id/336053 TJ> Regarding the buffer overflow: The cert website currently outputs a TJ> "Lotus Notes exception". Is the overflow theoretically exploitable TJ> via a malicious email or does a user need to upload a malicious TJ> sieve script? Hmmm... Still down... The user has to upload a malicious sieve script. The DSA reads It was discovered that the SIEVE component of cyrus-imapd, a highly scalable enterprise mail system, is vulnerable to a buffer overflow when processing SIEVE scripts. Due to incorrect use of the sizeof() operator an attacker is able to pass a negative length to snprintf() calls resulting in large positive values due to integer conversion. This causes a buffer overflow which can be used to elevate privileges to the cyrus system user. An attacker who is able to install SIEVE scripts executed by the server is therefore able to read and modify arbitrary email messages on the system. Bron's fix is at https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/sieve/script.c.diff?r1=1.67;r2=1.68 Cheers Duncan -- Duncan Gibb - Technical Director Sirius Corporation plc - control through freedom http://www.siriusit.co.uk/ || t: +44 870 608 0063 Debian Cyrus Team - https://alioth.debian.org/projects/pkg-cyrus-imapd/ From dave64 at andrew.cmu.edu Wed Sep 9 12:43:43 2009 From: dave64 at andrew.cmu.edu (Dave McMurtrie) Date: Wed, 09 Sep 2009 12:43:43 -0400 Subject: Cyrus IMAPd 2.2.13p1 & 2.3.15 Released In-Reply-To: <4AA7D96B.701@siriusit.co.uk> References: <4AA7B1E2.9010507@andrew.cmu.edu> <200909091603.19312.thomas.jarosch@intra2net.com> <4AA7D96B.701@siriusit.co.uk> Message-ID: <4AA7DB3F.2010105@andrew.cmu.edu> Duncan Gibb wrote: > Thomas Jarosch wrote: > > KM> I'd like to announce the releases of Cyrus IMAPd 2.2.13p1 and 2.3.15. > KM> These releases should both be considered production quality. These > KM> releases are being made at this time to fix the potential buffer > KM> overflow vulnerability described in CERT VU#336053: > KM> http://www.kb.cert.org/vuls/id/336053 > > TJ> Regarding the buffer overflow: The cert website currently outputs a > TJ> "Lotus Notes exception". Is the overflow theoretically exploitable > TJ> via a malicious email or does a user need to upload a malicious > TJ> sieve script? > > Hmmm... Still down... Apologies for the CERT vulnerability link not existing. We had planned, along with CERT, to make a coordinated announcement about this tomorrow in order to give the major Cyrus vendors a chance to get their distributions patched. Unfortunately, Debian put out their DSA over the weekend so we didn't want to wait until tomorrow to put out our announcement. CERT provided that URL for us, but since they haven't yet formally released this vulnerability the URL isn't active yet. Thanks, Dave -- Dave McMurtrie, SPE Email Systems Team Leader Carnegie Mellon University, Computing Services From brong at fastmail.fm Thu Sep 10 00:41:33 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Thu, 10 Sep 2009 14:41:33 +1000 Subject: Cyrus IMAPd 2.2.13p1 & 2.3.15 Released In-Reply-To: <4AA7DB3F.2010105@andrew.cmu.edu> References: <4AA7B1E2.9010507@andrew.cmu.edu> <200909091603.19312.thomas.jarosch@intra2net.com> <4AA7D96B.701@siriusit.co.uk> <4AA7DB3F.2010105@andrew.cmu.edu> Message-ID: <20090910044133.GA11606@brong.net> On Wed, Sep 09, 2009 at 12:43:43PM -0400, Dave McMurtrie wrote: > Duncan Gibb wrote: > >Thomas Jarosch wrote: > > > >KM> I'd like to announce the releases of Cyrus IMAPd 2.2.13p1 and 2.3.15. > >KM> These releases should both be considered production quality. These > >KM> releases are being made at this time to fix the potential buffer > >KM> overflow vulnerability described in CERT VU#336053: > >KM> http://www.kb.cert.org/vuls/id/336053 > > > >TJ> Regarding the buffer overflow: The cert website currently outputs a > >TJ> "Lotus Notes exception". Is the overflow theoretically exploitable > >TJ> via a malicious email or does a user need to upload a malicious > >TJ> sieve script? > > > >Hmmm... Still down... > > Apologies for the CERT vulnerability link not existing. > > We had planned, along with CERT, to make a coordinated announcement > about this tomorrow in order to give the major Cyrus vendors a > chance to get their distributions patched. > > Unfortunately, Debian put out their DSA over the weekend so we > didn't want to wait until tomorrow to put out our announcement. > CERT provided that URL for us, but since they haven't yet formally > released this vulnerability the URL isn't active yet. Which I'm afraid was my fault for saying "it's already been committed to CVS, so it's out there" to them. Sorry about that. *sigh*. Bron. From brong at fastmail.fm Thu Sep 10 09:53:32 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Thu, 10 Sep 2009 23:53:32 +1000 Subject: Cyrus Fastmail Patches Message-ID: <1252590817-12728-1-git-send-email-brong@fastmail.fm> This is an email in the style of the Linux Kernel Mailing Lists patch series. In a threaded mail reader, all the patches will be replies to this message, meaning you can easily reply to any of them and it will all thread neatly together. Along with targetted CCs (basically, you add the interested users for each patch as "Cc:" lines in the commit message and it automatically adds them as a CC just to the patch they care about) it's an interesting way to do development. I'm thinking of pushing all these changes to CVS soon, but I'd appreciate any comments. [PATCH 1/5] Rewrite zlib inflate handling This is the one Simon tested. It's running on our testbed, and will be on production shortly. [PATCH 2/5] Add "-x" option to cyr_expire to disable expunge I've had this for a while - though we're not using it yet. The idea is to allow trimming the duplicate DB more frequently without having to pay the IO hit of statting and reading all the cyrus.expunge files. [PATCH 3/5] Fix missing closedir() - bug #3159 [PATCH 4/5] cyrus-imapd-2.3.15-oldgcc.patch from Simon Matter Two little patches from the release announcement for 2.3.15 - sorry about the rush meaning we couldn't include these for this release! [PATCH 5/5] Track idling state and tell idled if we lose connection I just wrote this! Untested, but it will be pretty soon, because I'm tired of random processes being shot with a signal they aren't expecting. Bron. From brong at fastmail.fm Thu Sep 10 09:53:35 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Thu, 10 Sep 2009 23:53:35 +1000 Subject: [PATCH 3/5] Fix missing closedir() - bug #3159 In-Reply-To: <1252590817-12728-1-git-send-email-brong@fastmail.fm> References: <1252590817-12728-1-git-send-email-brong@fastmail.fm> Message-ID: <1252590817-12728-4-git-send-email-brong@fastmail.fm> Fixes bug #3159 - from Simon Matter --- timsieved/actions.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/timsieved/actions.c b/timsieved/actions.c index 45960f2..29a844f 100644 --- a/timsieved/actions.c +++ b/timsieved/actions.c @@ -301,6 +301,8 @@ static int countscripts(char *name) } } + closedir(dp); + return number; } @@ -555,6 +557,8 @@ int listscripts(struct protstream *conn) } } + closedir(dp); + prot_printf(conn,"OK\r\n"); return TIMSIEVE_OK; -- 1.6.2.2.446.gfbdc0 From brong at fastmail.fm Thu Sep 10 09:53:36 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Thu, 10 Sep 2009 23:53:36 +1000 Subject: [PATCH 4/5] cyrus-imapd-2.3.15-oldgcc.patch from Simon Matter In-Reply-To: <1252590817-12728-1-git-send-email-brong@fastmail.fm> References: <1252590817-12728-1-git-send-email-brong@fastmail.fm> Message-ID: <1252590817-12728-5-git-send-email-brong@fastmail.fm> Make Cyrus compile with older GCC --- imap/mbexamine.c | 6 +++--- imap/mupdate.c | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/imap/mbexamine.c b/imap/mbexamine.c index d36897f..c248ba7 100644 --- a/imap/mbexamine.c +++ b/imap/mbexamine.c @@ -218,6 +218,7 @@ int do_examine(char *name, struct mailbox mailbox; struct index_record record; cacherecord crec; + int j; signals_poll(); @@ -321,7 +322,6 @@ int do_examine(char *name, for(i=1; i<=mailbox.exists; i++) { mailbox_read_index_record(&mailbox, i, &record); - int j; if(wantvalue) { if(!wantuid) { @@ -416,6 +416,8 @@ int do_quota(char *name, struct mailbox mailbox; struct index_record record; uquota_t total = 0; + char fnamebuf[MAILBOX_FNAME_LEN]; + struct stat sbuf; signals_poll(); @@ -450,8 +452,6 @@ int do_quota(char *name, for(i=1; i<=mailbox.exists; i++) { mailbox_read_index_record(&mailbox, i, &record); - char fnamebuf[MAILBOX_FNAME_LEN]; - struct stat sbuf; strlcpy(fnamebuf, mailbox.path, sizeof(fnamebuf)); strlcat(fnamebuf, "/", sizeof(fnamebuf)); diff --git a/imap/mupdate.c b/imap/mupdate.c index 4e22c9b..7dc7bbd 100644 --- a/imap/mupdate.c +++ b/imap/mupdate.c @@ -1163,6 +1163,7 @@ static void *thread_main(void *rock __attribute__((unused))) int connflag; int new_fd; int ret = 0; + struct conn *ni; /* Lock Worker Count Mutex */ pthread_mutex_lock(&worker_count_mutex); /* LOCK */ @@ -1299,7 +1300,6 @@ static void *thread_main(void *rock __attribute__((unused))) /* Free all connections on idle_connlist. Note that * any connection not currently on the idle_connlist will * instead be freed when they drop out of their docmd() below */ - struct conn *ni; pthread_mutex_lock(&idle_connlist_mutex); /* LOCK */ for(C=idle_connlist; C; C = ni) { -- 1.6.2.2.446.gfbdc0 From brong at fastmail.fm Thu Sep 10 09:53:34 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Thu, 10 Sep 2009 23:53:34 +1000 Subject: [PATCH 2/5] Add "-x" option to cyr_expire to disable expunge In-Reply-To: <1252590817-12728-1-git-send-email-brong@fastmail.fm> References: <1252590817-12728-1-git-send-email-brong@fastmail.fm> Message-ID: <1252590817-12728-3-git-send-email-brong@fastmail.fm> Delayed expunge is great and all, but it causes a LOT of IO because you need to stat every mailbox's meta files, and you need to read all of every cyrus.expunge file. That's not something you might want to do every time you want to clean out the duplicate delivery database. Change cyr_expire to add a "-x" option. Ideally I'd like to just have not specifying a "-X" mean don't do it, but I think it might be too late for that. It would make disks fill up with junk at too many sites. --- imap/cyr_expire.c | 74 +++++++++++++++++++++++++++++----------------------- 1 files changed, 41 insertions(+), 33 deletions(-) diff --git a/imap/cyr_expire.c b/imap/cyr_expire.c index e3db589..3bc9f30 100644 --- a/imap/cyr_expire.c +++ b/imap/cyr_expire.c @@ -349,7 +349,7 @@ int delete(char *name, int main(int argc, char *argv[]) { extern char *optarg; - int opt, r = 0, expire_days = 0, expunge_days = -1, delete_days = -1; + int opt, r = 0, expire_days = 0, expunge_days = -1, delete_days = -1, do_expunge = 1; char *alt_config = NULL; char *find_prefix = NULL; char buf[100]; @@ -366,7 +366,7 @@ int main(int argc, char *argv[]) memset(&erock, 0, sizeof(erock)); memset(&drock, 0, sizeof(drock)); - while ((opt = getopt(argc, argv, "C:D:E:X:p:va")) != EOF) { + while ((opt = getopt(argc, argv, "C:D:E:X:p:vax")) != EOF) { switch (opt) { case 'C': /* alt config file */ alt_config = optarg; @@ -387,6 +387,11 @@ int main(int argc, char *argv[]) expunge_days = atoi(optarg); break; + case 'x': + if (!do_expunge) usage(); + do_expunge = 0; + break; + case 'p': find_prefix = optarg; break; @@ -427,44 +432,47 @@ int main(int argc, char *argv[]) exit(1); } - /* xxx better way to determine a size for this table? */ - construct_hash_table(&expire_table, 10000, 1); - - /* expire messages from mailboxes, - * build a hash table of mailboxes in which we expired messages, - * and perform a cleanup of expunged messages - */ - erock.table = &expire_table; - erock.expunge_mode = config_getenum(IMAPOPT_EXPUNGE_MODE); - if (expunge_days == -1) { - erock.expunge_mark = 0; - } else { - erock.expunge_mark = time(0) - (expunge_days * 60 * 60 * 24); + if (do_expunge) { + /* xxx better way to determine a size for this table? */ + construct_hash_table(&expire_table, 10000, 1); + + /* expire messages from mailboxes, + * build a hash table of mailboxes in which we expired messages, + * and perform a cleanup of expunged messages + */ + erock.table = &expire_table; + erock.expunge_mode = config_getenum(IMAPOPT_EXPUNGE_MODE); + if (expunge_days == -1) { + erock.expunge_mark = 0; + } else { + erock.expunge_mark = time(0) - (expunge_days * 60 * 60 * 24); + + if (erock.verbose && + erock.expunge_mode != IMAP_ENUM_EXPUNGE_MODE_IMMEDIATE) { + fprintf(stderr, + "Expunging deleted messages in mailboxes older than %d days\n", + expunge_days); + } + } - if (erock.verbose && - erock.expunge_mode != IMAP_ENUM_EXPUNGE_MODE_IMMEDIATE) { - fprintf(stderr, - "Expunging deleted messages in mailboxes older than %d days\n", - expunge_days); + if (find_prefix) { + strlcpy(buf, find_prefix, sizeof(buf)); + } else { + strlcpy(buf, "*", sizeof(buf)); } - } - if (find_prefix) { - strlcpy(buf, find_prefix, sizeof(buf)); - } else { - strlcpy(buf, "*", sizeof(buf)); - } - mboxlist_findall(NULL, buf, 1, 0, 0, &expire, &erock); + mboxlist_findall(NULL, buf, 1, 0, 0, &expire, &erock); - syslog(LOG_NOTICE, "Expunged %lu out of %lu messages from %lu mailboxes", - erock.deleted, erock.messages, erock.mailboxes); - if (erock.verbose) { - fprintf(stderr, "\nExpunged %lu out of %lu messages from %lu mailboxes\n", - erock.deleted, erock.messages, erock.mailboxes); + syslog(LOG_NOTICE, "Expunged %lu out of %lu messages from %lu mailboxes", + erock.deleted, erock.messages, erock.mailboxes); + if (erock.verbose) { + fprintf(stderr, "\nExpunged %lu out of %lu messages from %lu mailboxes\n", + erock.deleted, erock.messages, erock.mailboxes); + } } if ((delete_days != -1) && mboxlist_delayed_delete_isenabled() && - (deletedprefix = config_getstring(IMAPOPT_DELETEDPREFIX))) { + (deletedprefix = config_getstring(IMAPOPT_DELETEDPREFIX))) { struct delete_node *node; int count = 0; -- 1.6.2.2.446.gfbdc0 From brong at fastmail.fm Thu Sep 10 09:53:33 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Thu, 10 Sep 2009 23:53:33 +1000 Subject: [PATCH 1/5] Rewrite zlib inflate handling In-Reply-To: <1252590817-12728-1-git-send-email-brong@fastmail.fm> References: <1252590817-12728-1-git-send-email-brong@fastmail.fm> Message-ID: <1252590817-12728-2-git-send-email-brong@fastmail.fm> Use a single fixed-size buffer, and return to "inflate" again for further data rather than resizing the output buffer. Also don't use deflateBound in zlib because it doesn't exist in older versions of the library, and we can just resize the buffer in increments anyway. --- lib/prot.c | 163 ++++++++++++++++++++++++++++++++---------------------------- lib/prot.h | 1 - 2 files changed, 87 insertions(+), 77 deletions(-) diff --git a/lib/prot.c b/lib/prot.c index 6eb285d..1ae690d 100644 --- a/lib/prot.c +++ b/lib/prot.c @@ -93,7 +93,6 @@ int write; newstream = (struct protstream *) xzmalloc(sizeof(struct protstream)); newstream->buf = (unsigned char *) xmalloc(sizeof(char) * (PROT_BUFSIZE)); - newstream->buf_size = PROT_BUFSIZE; newstream->ptr = newstream->buf; newstream->maxplain = PROT_BUFSIZE; newstream->fd = fd; @@ -103,6 +102,11 @@ int write; if(write) newstream->cnt = PROT_BUFSIZE; +#ifdef HAVE_ZLIB + newstream->zbuf = 0; + newstream->zbuf_size = 0; +#endif + return newstream; } @@ -515,6 +519,34 @@ int prot_fill(struct protstream *s) if (s->eof || s->error) return EOF; do { +#ifdef HAVE_ZLIB + /* check if there's anything in the zlib buffer already */ + if (s->zstrm && s->zstrm->avail_in) { + int zr; + if (!s->zbuf) { + s->zbuf = (unsigned char *)xmalloc(sizeof(unsigned char) * (PROT_BUFSIZE)); + s->zbuf_size = PROT_BUFSIZE; + } + s->zstrm->next_out = s->zbuf; + s->zstrm->avail_out = s->zbuf_size; + zr = inflate(s->zstrm, Z_SYNC_FLUSH); + if (zr != Z_OK) { + /* Error decompressing */ + syslog(LOG_ERR, "zlib error: %d %s", zr, s->zstrm->msg); + s->error = xstrdup("Error decompressing data"); + return EOF; + } + + if (s->zstrm->avail_out < PROT_BUFSIZE) { + /* inflated some data */ + s->ptr = s->zbuf; + s->cnt = PROT_BUFSIZE - s->zstrm->avail_out; + syslog(LOG_DEBUG, " => decompressed to %d bytes", s->cnt); + break; + } + } +#endif + /* wait until get input */ haveinput = 0; @@ -633,74 +665,41 @@ int prot_fill(struct protstream *s) } #ifdef HAVE_ZLIB - if (s->zstrm && s->cnt > 0) { - /* Decompress the data */ - int zr = Z_OK; + if (s->zstrm) { + /* transfer the stream to the input of the z_stream and loop */ s->zstrm->next_in = s->ptr; s->zstrm->avail_in = s->cnt; - s->zstrm->next_out = s->zbuf; - s->zstrm->avail_out = s->zbuf_size; - - syslog(LOG_DEBUG, "inflate(%d bytes)", s->cnt); + s->cnt = 0; + } +#endif /* HAVE_ZLIB */ + } while (!s->cnt); - do { - if (!s->zstrm->avail_out) { - /* Need more space to decompress */ - syslog(LOG_DEBUG, - "growing decompress buffer from %d to %d bytes", - s->zbuf_size, s->zbuf_size + PROT_BUFSIZE); - - s->zbuf = (unsigned char *) - xrealloc(s->zbuf, s->zbuf_size + PROT_BUFSIZE); - s->zstrm->next_out = s->zbuf + s->zbuf_size; - s->zstrm->avail_out = PROT_BUFSIZE; - s->zbuf_size += PROT_BUFSIZE; - } + if (s->logfd != -1) { + time_t newtime; + char timebuf[20]; - zr = inflate(s->zstrm, Z_SYNC_FLUSH); - } while (zr == Z_OK && !s->zstrm->avail_out); + time(&newtime); + snprintf(timebuf, sizeof(timebuf), "<%ld<", newtime); + write(s->logfd, timebuf, strlen(timebuf)); - if (zr != Z_OK || s->zstrm->avail_in) { - /* Error decompressing */ - s->error = xstrdup("Error decompressing data"); - return EOF; + left = s->cnt; + ptr = s->ptr; + do { + n = write(s->logfd, ptr, left); + if (n == -1 && (errno != EINTR)) { + break; } - s->ptr = s->zbuf; - s->cnt = s->zbuf_size - s->zstrm->avail_out; - - syslog(LOG_DEBUG, " => decompressed to %d bytes", s->cnt); - } -#endif /* HAVE_ZLIB */ - - if (s->cnt > 0) { - if (s->logfd != -1) { - time_t newtime; - char timebuf[20]; - - time(&newtime); - snprintf(timebuf, sizeof(timebuf), "<%ld<", newtime); - write(s->logfd, timebuf, strlen(timebuf)); - - left = s->cnt; - ptr = s->ptr; - do { - n = write(s->logfd, ptr, left); - if (n == -1 && errno != EINTR) { - break; - } - if (n > 0) { - ptr += n; - left -= n; - } - } while (left); + if (n > 0) { + ptr += n; + left -= n; } + } while (left); + } - s->cnt--; /* we return the first char */ - return *s->ptr++; - } - } while (1); + s->cnt--; /* we return the first char */ + return *s->ptr++; } /* @@ -751,20 +750,12 @@ static int prot_flush_encode(struct protstream *s, #ifdef HAVE_ZLIB if (s->zstrm) { /* Compress the data */ - unsigned long def_size = deflateBound(s->zstrm, left); int zflush = s->boundary ? Z_FULL_FLUSH : Z_SYNC_FLUSH; int zr = Z_OK; - if (def_size > s->zbuf_size) { - /* Make sure buffer is large enough to hold compressed data. - * Oversize the buffer, so we (hopefully) eliminate - * multiple small incremental reallocations. - */ - syslog(LOG_DEBUG, "growing compress buffer from %u to %lu bytes", - s->zbuf_size, def_size + PROT_BUFSIZE); - - s->zbuf_size = def_size + PROT_BUFSIZE; - s->zbuf = (unsigned char *) xrealloc(s->zbuf, s->zbuf_size); + if (!s->zbuf) { + s->zbuf = (unsigned char *)xmalloc(sizeof(unsigned char) * (PROT_BUFSIZE)); + s->zbuf_size = PROT_BUFSIZE; } s->zstrm->next_in = ptr; @@ -779,14 +770,34 @@ static int prot_flush_encode(struct protstream *s, if (s->boundary) { /* Set (new) compression level */ zr = deflateParams(s->zstrm, s->zlevel, Z_DEFAULT_STRATEGY); + if (zr != Z_OK) { + s->error = xstrdup("Error setting compression level"); + return EOF; + } } - if (zr == Z_OK) zr = deflate(s->zstrm, zflush); - if (zr != Z_OK || s->zstrm->avail_in) { - /* Error compressing */ - s->error = xstrdup("Error compressing data"); - return EOF; - } + do { + if (!s->zstrm->avail_out) { + syslog(LOG_DEBUG, "growing compress buffer from %u to %u bytes", + s->zbuf_size, s->zbuf_size + PROT_BUFSIZE); + + s->zbuf_size += PROT_BUFSIZE; + s->zbuf = (unsigned char *) xrealloc(s->zbuf, s->zbuf_size); + s->zstrm->avail_out = PROT_BUFSIZE; + } + zr = deflate(s->zstrm, zflush); + if (!(zr == Z_OK || zr == Z_STREAM_END || zr == Z_BUF_ERROR)) { + /* something went wrong */ + s->error = xstrdup("Error compressing stream"); + return EOF; + } + } while (!s->zstrm->avail_out); + /* http://www.zlib.net/manual.html says: + * If deflate returns with avail_out == 0, this function must be + * called again with the same value of the flush parameter and + * more output space (updated avail_out), until the flush is + * complete (deflate returns with non-zero avail_out). + */ ptr = s->zbuf; left = s->zbuf_size - s->zstrm->avail_out; diff --git a/lib/prot.h b/lib/prot.h index 3b8ad75..9df0976 100644 --- a/lib/prot.h +++ b/lib/prot.h @@ -73,7 +73,6 @@ typedef void prot_readcallback_t(struct protstream *s, void *rock); struct protstream { /* The Buffer */ unsigned char *buf; - unsigned buf_size; unsigned char *ptr; /* The end of data in the buffer */ unsigned cnt; /* Space Remaining in buffer */ -- 1.6.2.2.446.gfbdc0 From brong at fastmail.fm Thu Sep 10 09:53:37 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Thu, 10 Sep 2009 23:53:37 +1000 Subject: [PATCH 5/5] Track idling state and tell idled if we lose connection In-Reply-To: <1252590817-12728-1-git-send-email-brong@fastmail.fm> References: <1252590817-12728-1-git-send-email-brong@fastmail.fm> Message-ID: <1252590817-12728-6-git-send-email-brong@fastmail.fm> We get a bunch of "process killed with signal 10" errors after restarting one of our imap proxies, and it's because idled goes a little crazy throwing SIGUSR1 and SIGUSR2 at processids which have long since gone away. It's like a loaded gun that randomly shoots whatever process was unlucky enough to get that PID. Now, the whole design sucks really - but at least the worst of it will be fixed by telling idled that we've idle_done() when we shut_down(), no matter how we got there. --- imap/imapd.c | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-) diff --git a/imap/imapd.c b/imap/imapd.c index 8a32461..e4f7c63 100644 --- a/imap/imapd.c +++ b/imap/imapd.c @@ -208,6 +208,9 @@ static const int max_monthdays[] = { 31, 31, 30, 31, 30, 31 }; +/* track if we're idling */ +static int idling = 0; + void motd_file(int fd); void shut_down(int code); void fatal(const char *s, int code); @@ -870,6 +873,9 @@ void shut_down(int code) } if (backend_cached) free(backend_cached); + if (idling) + idle_done(imapd_mailbox ? imapd_mailbox->name : NULL); + if (imapd_mailbox) { index_closemailbox(imapd_mailbox); mailbox_close(imapd_mailbox); @@ -2549,11 +2555,15 @@ void cmd_idle(char *tag) /* Start doing mailbox updates */ if (imapd_mailbox) index_check(imapd_mailbox, 0, 1); idle_start(imapd_mailbox ? imapd_mailbox->name : NULL); + /* use this flag so if getc causes a shutdown due to + * connection abort we tell idled about it */ + idling = 1; /* Get continuation data */ c = getword(imapd_in, &arg); /* Stop updates and do any necessary cleanup */ + idling = 0; idle_done(imapd_mailbox ? imapd_mailbox->name : NULL); } else { /* Remote mailbox */ -- 1.6.2.2.446.gfbdc0 From thomas.jarosch at intra2net.com Tue Sep 15 07:59:07 2009 From: thomas.jarosch at intra2net.com (Thomas Jarosch) Date: Tue, 15 Sep 2009 13:59:07 +0200 Subject: Cyrus IMAPd 2.2.13p1 & 2.3.15 Released In-Reply-To: <4AA7DB3F.2010105@andrew.cmu.edu> References: <4AA7B1E2.9010507@andrew.cmu.edu> <4AA7D96B.701@siriusit.co.uk> <4AA7DB3F.2010105@andrew.cmu.edu> Message-ID: <200909151359.07615.thomas.jarosch@intra2net.com> On Wednesday, 9. September 2009 18:43:43 Dave McMurtrie wrote: > > TJ> Regarding the buffer overflow: The cert website currently outputs a > > TJ> "Lotus Notes exception". Is the overflow theoretically exploitable > > TJ> via a malicious email or does a user need to upload a malicious > > TJ> sieve script? > > > > Hmmm... Still down... > > Apologies for the CERT vulnerability link not existing. > > We had planned, along with CERT, to make a coordinated announcement > about this tomorrow in order to give the major Cyrus vendors a chance to > get their distributions patched. > > Unfortunately, Debian put out their DSA over the weekend so we didn't > want to wait until tomorrow to put out our announcement. CERT provided > that URL for us, but since they haven't yet formally released this > vulnerability the URL isn't active yet. Thanks for clearing this up! I'm very happy this is not triggerable via a malicious email :) Thomas From hmh at debian.org Sat Sep 19 12:42:20 2009 From: hmh at debian.org (Henrique de Moraes Holschuh) Date: Sat, 19 Sep 2009 13:42:20 -0300 Subject: Cyrus IMAPd 2.2.13p1 & 2.3.15 Released In-Reply-To: <20090910044133.GA11606@brong.net> References: <4AA7B1E2.9010507@andrew.cmu.edu> <200909091603.19312.thomas.jarosch@intra2net.com> <4AA7D96B.701@siriusit.co.uk> <4AA7DB3F.2010105@andrew.cmu.edu> <20090910044133.GA11606@brong.net> Message-ID: <20090919164220.GA25470@khazad-dum.debian.net> On Thu, 10 Sep 2009, Bron Gondwana wrote: > On Wed, Sep 09, 2009 at 12:43:43PM -0400, Dave McMurtrie wrote: > > Duncan Gibb wrote: > > >Thomas Jarosch wrote: > > >KM> I'd like to announce the releases of Cyrus IMAPd 2.2.13p1 and 2.3.15. > > >KM> These releases should both be considered production quality. These > > >KM> releases are being made at this time to fix the potential buffer > > >KM> overflow vulnerability described in CERT VU#336053: > > >KM> http://www.kb.cert.org/vuls/id/336053 > > > > > >TJ> Regarding the buffer overflow: The cert website currently outputs a > > >TJ> "Lotus Notes exception". Is the overflow theoretically exploitable > > >TJ> via a malicious email or does a user need to upload a malicious > > >TJ> sieve script? > > > > > >Hmmm... Still down... > > > > Apologies for the CERT vulnerability link not existing. > > > > We had planned, along with CERT, to make a coordinated announcement > > about this tomorrow in order to give the major Cyrus vendors a > > chance to get their distributions patched. > > > > Unfortunately, Debian put out their DSA over the weekend so we > > didn't want to wait until tomorrow to put out our announcement. > > CERT provided that URL for us, but since they haven't yet formally > > released this vulnerability the URL isn't active yet. > > Which I'm afraid was my fault for saying "it's already been > committed to CVS, so it's out there" to them. Sorry about > that. *sigh*. The problem is not that you told us we could release, you *were* correct in doing so: the problem was already as good as published to the whole world by the public cvs commit. The problem is that CERT, for whatever reason, tried to embargo something that was already semi-public, and to make the matters worse, the correct people were not told about it in a timely manner. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh From dave64 at andrew.cmu.edu Sat Sep 19 13:39:43 2009 From: dave64 at andrew.cmu.edu (Dave McMurtrie) Date: Sat, 19 Sep 2009 13:39:43 -0400 Subject: Cyrus IMAPd 2.2.13p1 & 2.3.15 Released In-Reply-To: <20090919164220.GA25470@khazad-dum.debian.net> References: <4AA7B1E2.9010507@andrew.cmu.edu> <200909091603.19312.thomas.jarosch@intra2net.com> <4AA7D96B.701@siriusit.co.uk> <4AA7DB3F.2010105@andrew.cmu.edu> <20090910044133.GA11606@brong.net> <20090919164220.GA25470@khazad-dum.debian.net> Message-ID: <4AB5175F.204@andrew.cmu.edu> Henrique de Moraes Holschuh wrote: >> Which I'm afraid was my fault for saying "it's already been >> committed to CVS, so it's out there" to them. Sorry about >> that. *sigh*. > I already spoke with Bron off-list at great length about this. There's really no need for any apology on his part. We greatly appreciate the work that Bron puts into Cyrus imapd. > The problem is not that you told us we could release, you *were* correct in > doing so: the problem was already as good as published to the whole world by > the public cvs commit. > > The problem is that CERT, for whatever reason, tried to embargo something > that was already semi-public, and to make the matters worse, the correct > people were not told about it in a timely manner. Fair enough. The next time an issue like this comes up, I think the first thing we can do better on our side is to not commit the fix to CVS immediately. When you say that CERT did not contact the correct people, can you be more specific? Feel free to respond off-list if you feel that's necessary. I have no problem getting back in touch with CERT to provide updated contact information for them. As far as this not having been handled in a timely manner, I don't think that's a fair criticism. This bug has existed in the code since Oct 22, 2003. Bron reported it to us on 09/02/2009. I reported it to CERT on the evening of 09/02/2009. CERT responded back to me on 09/03/2009 with a plan to contact Cyrus vendors immediately and make an announcement on 09/10/2009 -- a total of 3 business days for any US vendors since it was a holiday weekend. If you have any additional suggestions on how to better handle security issues in the future, please let me know. Thanks, Dave From hmh at debian.org Sat Sep 19 15:12:02 2009 From: hmh at debian.org (Henrique de Moraes Holschuh) Date: Sat, 19 Sep 2009 16:12:02 -0300 Subject: Cyrus IMAPd 2.2.13p1 & 2.3.15 Released In-Reply-To: <4AB5175F.204@andrew.cmu.edu> References: <4AA7B1E2.9010507@andrew.cmu.edu> <200909091603.19312.thomas.jarosch@intra2net.com> <4AA7D96B.701@siriusit.co.uk> <4AA7DB3F.2010105@andrew.cmu.edu> <20090910044133.GA11606@brong.net> <20090919164220.GA25470@khazad-dum.debian.net> <4AB5175F.204@andrew.cmu.edu> Message-ID: <20090919191202.GE28777@khazad-dum.debian.net> On Sat, 19 Sep 2009, Dave McMurtrie wrote: > Henrique de Moraes Holschuh wrote: > >>Which I'm afraid was my fault for saying "it's already been > >>committed to CVS, so it's out there" to them. Sorry about > >>that. *sigh*. > > I already spoke with Bron off-list at great length about this. > There's really no need for any apology on his part. We greatly > appreciate the work that Bron puts into Cyrus imapd. Indeed we do! Bron, thank you very much for the work you've been doing on Cyrus IMAP. We are also *very* thankful to CMU for Cyrus IMAP and all the work done on it over the years. > When you say that CERT did not contact the correct people, can you > be more specific? Feel free to respond off-list if you feel that's > necessary. I have no problem getting back in touch with CERT to > provide updated contact information for them. Well, when you warn all downstream maintainers about an issue, you have to keep them all in the loop. This was not done for whatever reason. Unless told differently, CERT will only contact the security contact points for a given distro or package, which is likely not going to include all of us... > As far as this not having been handled in a timely manner, I don't > think that's a fair criticism. This bug has existed in the code I was complaining about the embargo request from CERT coming too late, that's all. And explaining why Debian had releases out before the embargo lift date: the Debian maintainers were never told about the embargo until it was too late. > If you have any additional suggestions on how to better handle > security issues in the future, please let me know. 1. If it is going to be embargoed, tell everyone upfront and don't commit to public cvs. This will avoid leaks before the embargo is lifted. 2. If it is semi-public for whatever reason, make it clear to CERT, and don't embargo. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh From brong at fastmail.fm Sat Sep 19 21:04:21 2009 From: brong at fastmail.fm (Bron Gondwana) Date: Sun, 20 Sep 2009 11:04:21 +1000 Subject: Cyrus IMAPd 2.2.13p1 & 2.3.15 Released In-Reply-To: <20090919191202.GE28777@khazad-dum.debian.net> References: <4AA7B1E2.9010507@andrew.cmu.edu> <200909091603.19312.thomas.jarosch@intra2net.com> <4AA7D96B.701@siriusit.co.uk> <4AA7DB3F.2010105@andrew.cmu.edu> <20090910044133.GA11606@brong.net> <20090919164220.GA25470@khazad-dum.debian.net> <4AB5175F.204@andrew.cmu.edu> <20090919191202.GE28777@khazad-dum.debian.net> Message-ID: <20090920010421.GA3754@brong.net> On Sat, Sep 19, 2009 at 04:12:02PM -0300, Henrique de Moraes Holschuh wrote: > On Sat, 19 Sep 2009, Dave McMurtrie wrote: > > Henrique de Moraes Holschuh wrote: > > >>Which I'm afraid was my fault for saying "it's already been > > >>committed to CVS, so it's out there" to them. Sorry about > > >>that. *sigh*. > > > > I already spoke with Bron off-list at great length about this. > > There's really no need for any apology on his part. We greatly > > appreciate the work that Bron puts into Cyrus imapd. > > Indeed we do! Bron, thank you very much for the work you've been doing on > Cyrus IMAP. Aww, shucks. > We are also *very* thankful to CMU for Cyrus IMAP and all the work done on > it over the years. Yeah, it's a nice system (mostly, it does have its warts!) We certainly appreciate it here at FastMail. > > When you say that CERT did not contact the correct people, can you > > be more specific? Feel free to respond off-list if you feel that's > > necessary. I have no problem getting back in touch with CERT to > > provide updated contact information for them. > > Well, when you warn all downstream maintainers about an issue, you have to > keep them all in the loop. This was not done for whatever reason. Unless > told differently, CERT will only contact the security contact points for a > given distro or package, which is likely not going to include all of us... I should have fed that information back to you guys too, since I sent the initial heads-up to the people I knew were maintaining packages. I didn't realise that CERT contacts wouldn't be forwarded to you. > > As far as this not having been handled in a timely manner, I don't > > think that's a fair criticism. This bug has existed in the code > > I was complaining about the embargo request from CERT coming too late, > that's all. And explaining why Debian had releases out before the embargo > lift date: the Debian maintainers were never told about the embargo until > it was too late. We'll have to be more careful about this one next time. > > If you have any additional suggestions on how to better handle > > security issues in the future, please let me know. > > 1. If it is going to be embargoed, tell everyone upfront and don't commit to > public cvs. This will avoid leaks before the embargo is lifted. This is somewhat interesting actually - I made my initial commit to my github tree before I realised the security implications. That's a whole lot less public than CVS, but it's still out there. My workflow always goes via the public space, even to get commits onto our testbed. For backups! If we did move to using git then it would be good to have a "private branch" that didn't get exported to the public repository or notified to anybody until we cherry-picked the commits over to mainline! Bron. > 2. If it is semi-public for whatever reason, make it clear to CERT, and > don't embargo. > > -- > "One disk to rule them all, One disk to find them. One disk to bring > them all and in the darkness grind them. In the Land of Redmond > where the shadows lie." -- The Silicon Valley Tarot > Henrique Holschuh From dave64 at andrew.cmu.edu Mon Sep 21 06:58:00 2009 From: dave64 at andrew.cmu.edu (Dave McMurtrie) Date: Mon, 21 Sep 2009 06:58:00 -0400 Subject: Cyrus IMAPd 2.2.13p1 & 2.3.15 Released In-Reply-To: <20090919191202.GE28777@khazad-dum.debian.net> References: <4AA7B1E2.9010507@andrew.cmu.edu> <200909091603.19312.thomas.jarosch@intra2net.com> <4AA7D96B.701@siriusit.co.uk> <4AA7DB3F.2010105@andrew.cmu.edu> <20090910044133.GA11606@brong.net> <20090919164220.GA25470@khazad-dum.debian.net> <4AB5175F.204@andrew.cmu.edu> <20090919191202.GE28777@khazad-dum.debian.net> Message-ID: <4AB75C38.7060905@andrew.cmu.edu> Henrique de Moraes Holschuh wrote: > 1. If it is going to be embargoed, tell everyone upfront and don't commit to > public cvs. This will avoid leaks before the embargo is lifted. > > 2. If it is semi-public for whatever reason, make it clear to CERT, and > don't embargo. Duly noted. We'll keep these things in mind and try to do better next time. Thank you, Dave -- Dave McMurtrie, SPE Email Systems Team Leader Carnegie Mellon University, Computing Services From bawood at umich.edu Mon Sep 21 12:30:09 2009 From: bawood at umich.edu (Brian Awood) Date: Mon, 21 Sep 2009 12:30:09 -0400 Subject: 2.3.15 cvs tag Message-ID: <200909211230.09634.bawood@umich.edu> Hello, I'm tracking the cvs repository and managing our local patches with git, when I went to rebase our patches on 2.3.15 I noticed there wasn't anything in cvs tagged as that version. Just wondering if that's intentional or if it was overlooked? Thanks, Brian From murch at andrew.cmu.edu Mon Sep 21 13:03:38 2009 From: murch at andrew.cmu.edu (Ken Murchison) Date: Mon, 21 Sep 2009 13:03:38 -0400 Subject: 2.3.15 cvs tag In-Reply-To: <200909211230.09634.bawood@umich.edu> References: <200909211230.09634.bawood@umich.edu> Message-ID: <4AB7B1EA.3010807@andrew.cmu.edu> It was an oversight. I'll get it tagged soon. Brian Awood wrote: > Hello, > > I'm tracking the cvs repository and managing our local patches with > git, when I went to rebase our patches on 2.3.15 I noticed there > wasn't anything in cvs tagged as that version. Just wondering if > that's intentional or if it was overlooked? > > Thanks, > Brian > -- Kenneth Murchison Systems Programmer Carnegie Mellon University