use valgrind / Re: SIGSEGV in cyrus-imapd 3.0.7 mupdate

[Michael Menge]

Hi Ellie

thanks for your replies,

Quoting ellie timoney <ellie at fastmail.com>:

> From what I'm seeing here, it looks like mupdate calls  
> tls_init_serverengine() for each new STARTTLS session, and then  
> calls tls_shutdown_serverengine()  when that session ends.
>
> The thing is though, the TLS state that these functions manage is  
> something like a singleton, it should only exist once for each  
> process.  And in fact, tls_init_serverengine() enforces this (which  
> is why you don't see the dh_params being allocated for each  
> connection: it only lets itself run once).  This works fine for all  
> the single-threaded service programs, but mupdate is a threaded  
> application and can have multiple connected client sessions in the  
> same process.  The comments at the top of imap/tls.c are instructive:
>
>> * DESCRIPTION
>> *       This module is the interface between Cyrus Imapd and the  
>> OpenSSL library.
>> *       As of now only one filedescriptor can be handled, so only one
>> *       TLS channel can be open at a time.
>
> I can't see anything in the code that makes me suspect this comment  
> is out of date.  It doesn't look the slightest bit thread-safe.
>
> At the moment, it looks to me like the STARTTLS support was added to  
> mupdate with no consideration of its thread-safety, and I'm left to  
> assume that people aren't really using STARTTLS for their mupdate  
> connections, otherwise this probably would've been tripped over long  
> ago.  (Or, maybe it was, and someone said "don't do that", and then  
> it all got lost to history?  It would be interesting to hear from  
> the front lines.)

I have been running mupdate with STARTTLS for a few years with 2.3.x  
and 2.4.x.
It didn't crash as predictable as 3.0, but we had some instances where the
mupdate master crashed and stopped working and we had to restart the service
(on an other host). At that time we where unable to debug it, as it would
run without crash for weeks or months. I suspect now that the  
non-thread-safe usage
of openssl could have been the cause for this.

Using STARTTLS for mupdate was the obvious solution after the default  
for "allowplaintext"
(Allow the use of cleartext passwords on the wire) was changed to 0.

>
> Anyway, it looks to me like the STARTTLS support in mupdate is just  
> fundamentally broken at the moment, and my recommendation is to not  
> use it.  If your IMAP servers need to connect to an mupdate server  
> that's not within their trusted network, I guess you'd need to set  
> up a VPN for it or something along those lines (but I'm no network  
> specialist).
>

could you add a warning in the relevant murder/installation guides and  
manuals?

> I don't honestly see this being fixed any time soon -- it would  
> require either:
>
>  * a big rewrite of imap/tls.c to make it thread-safe
>  * a big rewrite of mupdate to make it single-threaded like the rest of Cyrus

out of curiosity, why is mupdate multi-threaded in the first place?

>  * a big rewrite of mupdate to make it do its own TLS handling  
> (rather than using imap/tls.c)
>

This sounds like it is the same work as the first item, but it is
only used for mupdate.

> All of these are serious tasks with serious testing requirements,  
> especially considering the need to interact with OpenSSL correctly.   
> Even if someone does produce patches for master, they won't make it  
> back to the 3.0 series.
>

so is there a chance for a patch in 3.1?

> Sorry to be the bearer of annoying news!

I will see how I can work around this.

The cyrus project has much improved since fastmail dedicated
some developers to it, but I think it is time that fastmail switches
to a murder setup, so that this part of the code gets the same love
and testing as the other parts of cyrus ;-)

Thanks to all helping debugging this problem. I will gladly help
with testing any fixes for this bug.

Kind regards,

    Michael

--------------------------------------------------------------------------------
M.Menge                                Tel.: (49) 7071/29-70316
Universität Tübingen                   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung          mail:  
michael.menge at zdv.uni-tuebingen.de
Wächterstraße 76
72074 Tübingen