[PATCH] Changing folder paths on disk to support fast-rename

[Bron Gondwana]

This is a very small patch for a very big change (and incomplete,
it doesn't alter rehash, dohash, etc)

Anyway...

Considering the fast-rename and extending it to delete, we have
the following problem:

folders:

user.brong
user.brong.foo
user.brong.foo.bar

Delete/rename user.brong.foo WITHOUT affecting user.brong.foo.bar.

On disk these are:

BASEDIR/b/user/brong/
BASEDIR/b/user/brong/foo/
BASEDIR/b/user/brong/foo/bar/

So you either have to:

a) move every single non "bar" file to the new location, or

b) move user/brong/foo to the new location, then create a new
   user/brong/foo and move bar back into it.

I realised there was:

c) change folder layout on disk such that sub folders in the
   IMAP world aren't sub-folders in the on-disk world.


(c) started looking more attractive all the time, especially
since I've seen it at work with my Maildir copy which
offlineimap keeps up to date for me.


The following patch creates:

BASEDIR/user/b/brong/user.brong/
BASEDIR/user/b/brong/user.brong.foo/
BASEDIR/user/b/brong/user.brong.foo.bar/

and handles shared namespace, etc as:

BASEDIR/shared/shared
BASEDIR/shared/shared.foo
BASEDIR/shared/shared.foo.bar

Basically, if mboxlist_isusermailbox(name, 0) is true then
it gets hashed the first way, otherwise the second.

This has the nice property with our rename/delete patch
(also attached) and required for this patch you get:

BASEDIR/user/b/brong/DELETED.user.brong.foo.TIMESTAMP/

keeping the user's stuff all together on disk.


I really do prefer the "each IMAP folder is a different
folder" and "hashing based on realistic use patterns" (at
least for us... and I'm a believer in Hans Reiser's attitude
that if your filesystem can't handle a folder with thousands
of items then your filesystem needs fixing)


I'll be working on fast-rename in this new universe next,
but I thought I should throw this out there for comments.
So, what do you think (and yes, I'll be making rehash at
least work happily with this, because we'll need to do it
ourselves.  Current plan - down the replica, rehash it,
change the config, bring it back up.  Failover, rinse,
repeat)

Bron.

P.S. I'm using quilt now to work on these patches.  It's an
order of magnitude nicer than working on each patch in
isolation and using diff/patch by hand, but it means that
patches are now in a series that apply one after the other
rather than all against the raw upstream source.  I hope to
post my entire quilt series once I fix the one thing that
has a FastMail specific key encoded in it to use a config
option instead.
-- 
  Bron Gondwana
  brong at fastmail.fm

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: cyrus-hashuser-2.3.9.diff
Url: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20070821/51e48c80/attachment-0002.ksh 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: cyrus-delete-rename-2.3.8.diff
Url: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20070821/51e48c80/attachment-0003.ksh