Cyrus with a NFS storage. random DBERROR

[Paul Dekkers]

Hi,

Michael Menge wrote:
> after the problem with the wiki was solved, i added a summery about
> CyrusCluster
> http://cyrusimap.web.cmu.edu/twiki/bin/view/Cyrus/CyrusCluster .
> Please feel free to add infos about your experience with NFS

Good suggestion. I will do some more testing, for now: I did some
experiments on FreeBSD. I noticed that NFSv3 with -L (the -o nolock
equivalent) works too for storing both metadata and data on NFSv3, and
that separating metadata to a local (UFS) partition while mounting with
normal NFS options for the data partition works too. (As with Linux. But
a little bit faster actually, while storing both metadata and data on
NFS with local locking seems to be a bit slower.)

FreeBSD and NFSv4 is a no-go: I get a bunch of "Fatal error: failed to
mmap" errors.
And I guess this also indicates what makes NFS(v3) tricky: the mmaps
have to work, not sure if this can be considered 100% safe. (Still
reluctant to put this in production therefore, but perhaps there's
nothing to fear.)

For Linux the NFSv4 worked better, seperating metadata and storing the
data on NFSv4 is fine, but putting the metadata on NFSv4 too doesn't
work that well. Performance is bad, throughput decreases and things
stall for a while eventually. I just checked with Fedora 7 too, similar
results as with RHEL 4.

So it sounds like Linux and FreeBSD share the same matrix for NFSv3,
either local locking (no NFS) for all data, or NFS locking for the data
and local (ext3/UFS in my case) metadata. NFSv4 doesn't really add much.

Not sure what much testing I can do to assure that it is safe to put
data on NFS while storing metadata on the local filesystem. It doesn't
give an active-active setup, but there are certainly advantages in
performance and reliability. (But perhaps GFS is still worth checking,
if not just for the metadata ;-))

Paul