choosing a file system

[Bron Gondwana]

On Thu, Jan 08, 2009 at 10:13:25PM -0800, Robert Banz wrote:
>>
>> There's a significant upfront cost to learning a whole new system
>> for one killer feature, especially if it comes along with signifiant
>> regressions in lots of other features (like a non-sucky userland
>> out of the box).
>
> The "non-sucky" userland comment is simply a matter of preference, and  
> bait for a religious war, which I'm not going to bite.

Well, yeah.  Point.  Though most Solaris admins I know tend to pull in
gnu or bsd utilities pretty quickly.  I'll take that one back, it was
baity.

> What I will say is that switching between Solaris, Linux, IRIX, Ultrix, 
> FreeBSD, HP-UX, OSF/1 -- any *nix variant, should not be considered a 
> stumbling block. Your comment shows the narrow-mindedness of the current 
> Linux culture, many of us were brought up supporting and using a 
> collection of these platforms at any one time.

There's a switching cost, particularly if you don't have any experience
with a new system.  You have to consider that cost when making an
upgrade choice.  I agree that ZFS is better than anything currently
available on Linux - but the question is "does that outweight the 
disadvantages of learning and supporting a new platform?".

There are basically two worthwhile things on Solaris: ZFS and DTrace.
Other things - fork behaviour caused us pain recently, it's just not
as cheap as on Linux, and forking from a big process caused lots of
swapping because even though it was execing pretty quickly, it had
to commit the memory first.  Oops.  There are downsides to Linux's
overcommit, but having to add complexity to our backup manager
because forking for every backup was too expensive was annoying.

(I do take offence to being considered narrow minded for not blindly
following the latest fashion and wanting to switch everything over to
Solaris because it has the latest bling - I've considered it, but the
numbers just don't add up.  We have something that works, is reliable
and is fast.  Our redundancy is just at a different level)

Hey - back on topic for cyrus.  We store sha1s of message files in
the index file now.  We don't have checksums on index files (yet, I
have crc32 patches half finished somewhere), but we're at a point
where userland scrubs are possible.  Along with replication, you can
restore any damaged file from the replica.  Actually, with out backup
system, you can even pull the original file from the backup, knowing
its sha1 because it gets recalculated again and checked during the
backup phase.

> (notice, didn't mention AIX. I've got my standards ;)

Hey - I have a friend who _likes_ AIX.  There are odd people in the
world.

> Patching is always an issue on any OS, and you do have the choice of  
> running X applications remotely (booting an entire graphic  
> environment!?), and many other tools available such as pca to help you  
> patch on Solaris, which provide many of the features that you're used  
> to.

I take it you haven't run X applications remotely from the other side of
the world before?  I'd hardly call it "running".  Crawling maybe.

My current approach is to run up an vncserver on a box in the same colo
and run X applications remotely to there.  It's significantly less
painful, and also gives me a place to run an iceweasel to talk to the
web interfaces of things things that won't talk to me any other way.
Uploading firmware via the web from locally is similarly less sucky
than pushing it out from Australia.

And I'm seeing there are quite a few third party tools that people have
written to ease the pain of patch management on Solaris (I believe it's
actually one of the nicer unixes to manage patches on, but when you're
used to apt-get, there's a whole world of WTFery in manually downloading
and applying patch sets - especially when you get permission denied on
a bunch of things that the tool has just suggested as being missing)

In short - I'm not sold on the value to FastMail of at least two of us
(bus factor) learning to maintain Solaris to the level that we'd want
for running something so core to our operations as the IMAP servers.

Bron ( happy to either stop the flamewar or take it off list at this
       point.  I don't think we're contributing anything meaningful
       any more )