Replication and single instance store

[David Carter]

On Tue, 4 Sep 2007, Bron Gondwana wrote:

> Ah - yeah, that's right.  Except that the restart only got negotiated 
> after each folder was processed, and if you're pushing a new folder with 
> 200,000 messages (say, after a user move in our case) then that got a 
> bit memory hungry and all sorts insane.

Yes, this was the 5*MAX_MAILBOX_PATH allocation for each message when
support for partitions was added.

My original code cached 300k messages on the server between restarts 
(without any substantial memory leak). Better, but not perfect.

> Does this mean there is no way to get single-instance-store on a replica
> if you're rebuilding it from scratch?

No. You would need a database which maintained a persistent mapping 
between UUID and a list of files on each partitition which are that UUID.

I'm open to suggestions.

-- 
David Carter                             Email: David.Carter at ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.