Performance and cheap storage

[Bron Gondwana]

On Sat, 5 Aug 2006 16:02:44 -0700 (PDT), "David Lang" <dlang at digitalinsight.com> said:
> On Sat, 5 Aug 2006, Bron Gondwana wrote:
> 
> > Your frontend only can make connections out using any port it likes, but
> > there are only 65k of them, and at any one time, a fraction of those
> > will be tied up doing other things.
> 
> actually, it should only be the sourceIP,destIP,sourceport,destport that
> needs 
> to be unique for a TCP connection so the ports that are in use going to
> other 
> destinations should not affect you.

Does the TCP stack know that when you request a port?  My understanding
was that when you requested an address/port pair you weren't given one
already used by another program.  I could be wrong though.

> > We actually had to set a TCP parameter on our backends
> > (net.ipv4.tcp_rfc1337 = 0) so that when a busy frontend re-used a socket
> > pair for a previously closed connection it didn't just get dropped
> > immediately and return a "connection failed" back to nginx and hence the
> > client.
> >
> > That's fine for now, but if we ever hit 65k connections from a frontend
> > to a single imap server (eek) then we suspect that it will either lock
> > up trying to bind the connection or nginx will fail back to the client,
> > not a good look.
> >
> > (my just-this-second thought involves multiple listening ports on the
> > IMAP backends with multiple imapd lines in cyrus.conf and a randomiser
> > in the nginx authentication proxy, but I haven't talked to Rob about
> > that yet... since I only came up with it as I wrote this!)
> 
> this should be simple. tell the murder server that you have multiple
> backends 
> (but those backends just happen to live on the same box, either through 
> listening on multiple IPs/ ports, or in the worst case running multiple 
> instances of cyrus.

Yeah, we don't do murder - just nginx.  It will be easy to just make the
nginx authentication daemon return the different addresses at random.

We already start up one IP address in 10. range for each IMAP master in the
cyrus init script, easy enough to add more if we were so inclined.
-- 
  Bron Gondwana
  brong at fastmail.fm