Cyrus process model...

[David Lang]

as someone attempting to get apache 2 running (reliably) in a high volume
environment I can say the idea is interesting, but I definantly wouldn't
rush into useing it. if you have some time and want to get a start on
something that may (or may not) be worth doing in the long run you can
start on it, but don't stop maintaining the current version, the apache
core code may not be the right thing in the long run.

David Lang


 On Wed, 26 Feb 2003, Rob Mueller wrote:

> Date: Wed, 26 Feb 2003 16:45:00 +1100
> From: Rob Mueller <robm at fastmail.fm>
> To: Lawrence Greenfield <leg+ at andrew.cmu.edu>,
>      Rob Siemborski <rjs3 at andrew.cmu.edu>
> Cc: Ken Murchison <ken at oceana.com>,
>      info-cyrus <info-cyrus at lists.andrew.cmu.edu>
> Subject: Cyrus process model...
>
> [ Continued from an off mailing list conversation about killing cyrus lmtpd
> processes when they go haywire, and cyrus process accounting ]
>
> > > Surely this is a relatively well solved problem? Just about every unix
> > > system uses this master/forked child approach? How does apache do it?
> > > Net::Server::PreFork? I can't imagine that there aren't cookbook
> solutions
> > > to this issue since it's what unix has been doing for 30 years? Or is
> there
> > > something I'm missing here?
>
> > There are many different possibilities. Most other systems limit the
> > number of clients instead of forking new processes on demand without a
> > set limit. Apache also doesn't have differentiated children or
> > substantial shared state. (All children are members of the same
> > service or you don't particularly care how many additional unused children
> > you have...)
>
> I was under the impression that Apache 2 was planning on making it's
> forking/threading model much more generic, and supporting a general
> 'services' model, including a library to abstract the underlying OS? Hmmm,
> looking into that, it appears that it's mostly done already.
>
> http://apr.apache.org/
> http://apr.apache.org/apr2_0intro/apr2_0intro.htm
>
> And more:
>
> Contains following functionality
> -Reading and writing of files
> -Character set conversion
> -Network communications using sockets
> -Time management used for Internet type conversions
> -String management like C++ including natural order management
> -UNIX Password management routines
> -Table management routines
> -UUID Internet generation
> -Filename canonicalization
> -Random data generation
> -Global lock management
> -Threads and process management
> -Dynamic library loading routines
> -Memory mapped and shared memory
>
> -----
>
> http://www.arctic.org/~dean/apache/2.0/process-model.html
>
> I think the above is general enough to implement the interesting process
> models, and to implement optimizations that are available only in some of
> the multi-threaded models. Note that nothing above is specific to HTTP, and
> I believe that we should strive to keep the abstraction so that the same
> libraries can be used to implement other types of servers (i.e. FTP,
> streaming video/audio, corba).
>
> -----
>
> Would the cyrus team think it worthwhile to consider refactoring to use the
> new Apache 2 APR modules? I know off hand that it would be a lot of work,
> but it could be a gradual re-factoring process, and the idea of actually
> reusing code between projects would be *really* nice.
>
> Joel Spolsky is a big proponent of refactoring over time to improve software
> and you can read some of his thoughts here.
>
> http://www.joelonsoftware.com/articles/fog0000000069.html
> http://www.joelonsoftware.com/news/fog0000000328.html
>
> Ooops, I'm feeling a rant come along...
>
> *** RANT MODE ***
>
> I know this is a little off topic, but the source for cyrus is really
> showing it's age a bit. I know that happens with all software, you start
> with certain assumptions, and the more you go on, the more the original
> assumptions get blown away, so you hack this in here, and there, and then
> every now and then, you go on a big cleanup spree! The problem I feel is
> that the cleanup hasn't been big enough or often enough.
>
> Also, over time programming habits change. Many old C idioms are pretty much
> dead. Most of the C string handling methods are now annoying, or downright
> dangerous. There are several dozen replacement libraries, including the APR
> one above, and good ones like
> http://www.annexia.org/freeware/c2lib/index.msp. This library also
> implements automatically resizing arrays and memory pools, a common way to
> avoid all subtle leaks introduced by malloc() and the like, and to avoid the
> buffer overflows of stack buffers.
>
> I'm sure I could go on and on, and I'd eventually get back to the fact that
> the biggest problem is that the original C lanuage and library is pretty
> horrible in hindsight. But people in general still only dare to use the
> original library and idioms, and loath to include extra dependencies in
> their products, often instead rewriting the same new set of libraries from
> scratch! *sigh* Ok, better stop now...
>
> *** RANT MODE OFF ***
>
> Sorry, don't mean for any of this to come across as too negative. Actually,
> I'm hoping to inspire people enough to look into some of these alternatives,
> and see if there can at least be some pushes into what I feel would be a
> better direction for the coding of cyrus to go. Of course, it's only IMHO...
>
> Rob
>