LARGE single-system Cyrus installs?

[Vincent Fox]

Bron Gondwana wrote:
> Also virtual interfaces means you can move an instance without having
> to tell anyone else about it (but it sounds like you're going with an
> "all eggs in one basket" approach anyway)
>
>   

No, not "all eggs in one basket", but better usage of resources.

It seems silly to spend all the money for a T2000 with redundancies
and SAN and etc.  And then have it choke up when it hits (for us) about
10K users.  It seems everyone we talk to scratches their head why this
system with all it's cores would choke.  We have found even older Sun
V210 are handling 4K users with negligible load.

Our working hypothesis is that CYRUS is what is choking up at a certain
activity level due to bottlenecks with simultaneous access to some shared
resource for each instance.

By keeping instances down to say around 5K users maybe we can fit 4 
instances
onto a T2000 and thus for us have 20K users per server, which would be a 
better
cost efficiency than current 10K per server.

And yes, if we need to move an instance to another backend because the
activity level has increased radically we could do that.  Just use ZFS 
snapshot
to send and receive the filesystems and move the virtual IP for that 
instance
over to a new server.

We have 2 T2000 setups and are about to add a 3rd one.
For now over half our user-base is spread onto older V210 systems
that were scavenged up when Fall Quarter crushed our original vision.
We are hoping to migrate users back off the ancient/unsupported
hardware onto the newer stuff through the next few months.