Cyrus with a NFS storage. random DBERROR

[Paul Dekkers]

Hi,

I recently tried to use NFS (on a RedHat client, both to a NetApp filer
as well as a RedHat NFS server) and I'll share my experiences:

Michael Menge wrote:
> Cyrus has 2 problems with NFS.
>
> 1. Cyrus depends on filesystem locking. NFS-4 should have solved this
> problem
> but i have not tested it.
>
> 2. BerkleyDB uses shared Memory which does not work accros multiple
> servers.

I used skiplist in the tests (default with Simon's RPM), and initially
just used NFSv3 (and I also tested NFSv4): as long as I mounted with the
-o nolock option it actually worked quite well (also on NFSv3). The
performance was even better with the NetApp as target than with a local
filesystem (and NFSv3 was faster than v4).

The nolock options does not disable locking (as I understand it) for the
filesystem, it just disables locking over NFS, so other nodes won't have
the same file locked. (Correct me if I'm wrong.) My intention was not to
have an active-active setup, so in that regard this might not be that
bad. Not sure what other catches there are though.

I stressed the setup with the imaptest tool from Dovecot, I saw problems
with that in the past (also with NFSv3 and v4, but in combination with
Cyrus 2.2 and I'm not sure if I tried nolock), now it seemed to do just
fine. Only NFSv4 does not seem to be the answer, it seems that -o nolock
is (on Linux as client).

I'm very hesitant to put this into production, I just wanted to do some
more tests and ask others after that if they think this is wise or
not... I couldn't find the time to do more tests... (like see how RedHat
5 behaves instead of RedHat 4, if the tric also works on FreeBSD, if I
can make it fail one way or another... suggestions always welcome...)

Paul

P.S. For us it would be interesting to use this. I can still keep
replication as fallback too, but other than that I can have faster
failover to another primary server with NFS and something like
heartbeat... and I think a lot of overhead is right now in the ext3 fs.
(And the server would be a NetApp with WAFL, probably more efficient.)