upgrading with murder/aggregator

Fri May 28 10:53:59 EDT 2010

Hi Ragnar!

> Could this be mitigated by letting the proxys have a lifetime of only
> one session?

Possibly. I am sure if any proxyd terminates after having done its work
(which means either after a logout or a given idle period) this would
prevent re-use and possibly avoid the problem in a quite save way.

Just I am not aware of any parameter which could make this happen.

Which puts you back with your hands in the code.

I know the code a bit in the meanwhile; so I can imagine I would be able
to provide a patch, or better call it a hack. But I wonder if the effort
to do it properly, i.e. make sure some in memory structures for SASL get
properly reinitialized before the proxyd thread gets reused, would be so
much more. And that would have a bigger chance of being accepted into
the codebase.

Unfortunately, the developers are a bit quite on this problem so far.

Anyone?

Regards,
Torsten

Ragnar Sundblad schrieb:
> Hello Torsten!
> 
> On 28 maj 2010, at 15.51, Torsten Schlabach wrote:
> 
>> Hi Ragnar!
>>
>> We are currently doing exactly that. Interesting enough, starting off
>> the same software version 2.1.18. IMO this is a very nice and very smart
>> way to migrate, especially if the total number and size of your
>> mailboxes doesn't make it easy to just "qickly" copy everything over to
>> a new box. Also I guess you're aware that some of the internals changed
>> between Cyrus 2.1 and 2.2, so you cannot do a plugin replacement. Cyrus
>> 2.2 or 2.3 will just not read your stored mailboxes from 2.1 without
>> some (irreversible) upgrades of the data.
>>
>> Maybe it would be worth to create a page on the Cyrus wiki for just that
>> subject. I think you and us are not the first people to work on exactly
>> this. And there are some gotchas. Unfortunately, once people mastered
>> the task, they seldom come back to do some documentation and share their
>> experience.
>>
>> One gotcha I can save you:
>>
>> If you have made your setup, i.e. after you will have an aggregator and
>> what's your one and only mailserver today will be just one of multiple
>> backends, you will possibly want to move mailboxes off the backend which
>> is running 2.1.18 to some 2.3.x backend.
>>
>> You will find your Sieve scripts to be not working anymore.
>>
>> This is *not* because they are not migrated when you move a mailbox. But
>> it's because on newer versions of the sieve daemon, you need to compile
>> them to bytecode and this isn't done on the fly by the mailbox rename
>> (move) process.
> 
> Thanks - good to know!
> 
>> The other problem you may or you may not have:
>>
>> https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3213
>>
>> Unfortunately this bug is not getting a lot of attention; some previous
>> questions about this on this mailing list neither. So we digged a bit
>> deeper into the issue and I would be more then interested to hear if you
>> run into that problem as well or not.
>>
>> Basically, AFAIK, the communication between a Murder frontend and a
>> backend is IMAP. It's not secret proprietary Cyrus internal protocol.
>> The gurus here may correct me, but I thin theoretically, a backend
>> doesn't even have to be a Cyrus IMAPd. (Would be fun to try and run for
>> example an MS Exchange server as a backend in a Murder cluster.) The
>> only thing I *think* you'd need to use an arbitrary IMAP server as a
>> backend would be to make sure you find a way to tell the mupdate master
>> that mailbox XYZ is on a server a with credentials b and that the
>> backend server supports proxy authentication. But I am just speculating
>> here. But on that background, the version number of your backends really
>> shouldn't make any difference.
> 
> Unghh.
> 
> Could this be mitigated by letting the proxys have a lifetime of only
> one session? Sure there will be a lot more forking, but if CPU cycles
> can fix any problems with the migration I'd be happy to spend the little
> extra money.
> Maybe some more preforked proxies could improve the user experience,
> though I doubt it would be really noticeable. When the 2.1 servers are
> removed, one could possiby start using the proxies for more sessions again.
> 
>> Regarding the above mentioned ticket, we have in the meanwhile made some
>> more tests and added a new, fresh 2.1.18 backend to our Murder. And
>> voila; we can *not* reproduce the problem. So it may be something in the
>> config ouf our old 2.1.18 machine which is causing this behaviour we
>> see, i.e. occasional "Server(s) unavailable to complete operation" error
>> messages which seem to be a race condition at first.
> 
> Sigh. :-)
> 
>> I'll keep you posted, but at the same time,
> 
> Please do! I'll do the same!
> 
>> I'd also be interested in
>> your findings. But me personally, I can only encourage you to go for
>> what you have in mind. Unless you have a very tight deadline and very
>> little data to migrate.
> 
> We don't have a very tight deadline: the sooner the better, but
> correctly and tested are preconditions. I am not even sure we will
> start with the test rig before the summer vacations, hopefully
> we will.
> 
> Regards,
> 
> /ragge
> 
>> Regards,
>> Torsten
>>
>>
>>
>> Ragnar Sundblad schrieb:
>>> Is murder/aggregator backwards compatible with older imap
>>> servers?
>>>
>>> We want to migrate form a single 2.1.18 cyrus imap server to
>>> multiple 2.3.x servers. An idea is to insert a common
>>> murder/aggregator in front of both the 2.1.18 server and the
>>> new servers, and seamlessly migrate users. Would this be
>>> doable?
>>>
>>> Thanks!
>>>
>>> /ragge
>>>
>>> ----
>>> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
>>> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
>>> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>> ----
>> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
>> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
>> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html