Performance and cheap storage

[Bron Gondwana]

On Mon, Aug 07, 2006 at 05:59:33PM +0200, Phil Pennock wrote:
> On 2006-08-07 at 12:15 -0300, Henrique de Moraes Holschuh wrote:
> > On Mon, 07 Aug 2006, Kjetil Torgrim Homme wrote:
> > > I think David is missing the issue: it's the proxied connection which is
> > > problematic, not the connection to the client.  this locks the IP
> > > addresses to the frontend's and the backend's, and the port on the
> > > backend side is always 143 (or whatever you prefer), so the only
> > > variable part of the tuple is the port number on the frontend.  this
> > > restricts a frontend to 65k connections to each backend.
> > 
> > Not if the two hosts are capable of TCP timestamps, AFAIK.
> 
> TCP timestamps affects the quantity of data that can be in flight but
> doesn't affect the identification of the connection.  [RFC1323].
> 
> The main issue is that a connection is identifed by its two end-points,
> where an end-point is the IP address and the port.  Various other fields
> can be used to discard packets as being bad, not to multiplex
> connections.
> 
> So with one port on the backend, you have as many connections as there
> are available ports on the front-end.  Typically 32k-ish.
> 
> The "easy" fix is theoretically to configure up extra private addresses
> as aliases on the backend, and distribute the load over all of them.
> This avoids having multiple ports and multiple entries -- it's one
> cyrus.conf listening.  The problem may be making sure that the front-end
> knows that several backends are tied together as being one real system,
> to avoid interesting failover effects.

Actually, in our case that doesn't work too nicely because we already
listen on separate IP addresses.

There can be two cyrus replication masters on the one host if the replica
machine is down - otherwise there will be a master and a replica, each 
with their own IP address.  IPs follow roles, so 'imap5m' is always
10.202.10.5 and imap5r is always 10.202.11.5.  This means we can just
connect to 10.202.10.5 and know that it's the current replication
master.

But adding extra lines to cyrus.conf, now that sounds really easy -
I only have to do that to one template and then svn commit, roll
out and make install on the servers.  Yay makefile systems.

Bron.