Put only cyrus.index and cyrus.cache in metadata partitions ?

[Bron Gondwana]

On Thu, Aug 26, 2010 at 05:19:25PM +0200, Eric Luyten wrote:
> Hello,
> 
> 
> I understand the cyrus.header file is the only irreplacable one, i.e.
> cyrus.index and cyrus.cache can be regenerated using 'reconstruct'
> while cyrus.header, a very small (150 bytes or thereabout) and stable
> file, contains some vital mailbox identification data.

cyrus.index contains both the "INTERNALDATE" (re-creatable from the file
timestamp if you're lucky) and all your flags.  You'll be sad if you lose
it.  It also contains UIDVALIDITY.

cyrus.header contains the only copy of the mailbox UNIQUEID - which is
what the \Seen flag uses, so you'll be sad if you lose that.

cyrus.cache is totally re-generatable using reconstruct.  You won't care
too much if you lose that.

Mind you - cyrus.index is only 96 bytes per message or so (88 if you're
not using my patches)

> I observe a cyrus.header file only gets modified when the mailbox ACL is
> changed or when annotations are added. Am I correct in assuming this ?

Does it get changed for annotations?  It gets changed if you add a new flag,
or if you update the ACL (though it's only a backup copy of the ACL.  The
active version is in mailboxes.db)

> So most accesses to cyrus.header files are read operations.

Yeah, true.

> Does it therefore make (some) sense to leave the cyrus.header files with
> the message files and not with the other metadata ?
> I am not considering cyrus.expunge and cyrus.squat here for the sake of
> simplicity.

No, not really.  It makes more sense to leave cyrus.cache with the message
files because it's large, and put your meta files on small fast disk.

The next version of cyrus will sport "delayed cache loading" - where the
cache file doesn't even get statted until it's first needed.  This
reduces the IO cost off having it off on slow big disks.  We're hoping to
have an alpha to release soon.  We could do it now - but I have one more
pretty invasive change needed to ensure locking consistency, and I'd
rather make it first!

Bron.