...and a question (SHA1 UUIDs)

[David Carter]

All from:

   http://www-uxsup.csx.cam.ac.uk/~dpc22/cyrus/patches/2.3.8/

3 _incompatible_ spins on SHA1 UUID support
============================================

Each of the following requires an index upgrade and a new MINOR_VERSION.
Serious business, needs discussion.

sha1_uuid_add.patch:
--------------------

Extend UUIDs to 20 bytes: 1 byte version (schema) plus 19 bytes payload.

uuid_mode:
   none
   prealloc  (schema 1)  Current 2.3.8 behaviour with fix to master applied
   shortmd5  (schema 2)  First 11 bytes of message MD5, as per Fastmail.
        md5  (schema 3)  Full 16 bytes of message MD5
   shortsha1 (schema 4)  First 19 bytes of message SHA1

Existing schema 1 UUIDs are filled to 20 bytes

sha1_uuid_replace.patch
-----------------------

Throw away the existing UUID apparatus (in particular the environment
value CYRUS_UUID_PREFIX used to pass state from master to imapd and
lmtpd). Replace it with simple 20 byte UUIDs which are the message SHA1.

Any existing UUIDs are reset to NIL. I suppose that it would be possible
to pad the existing 12 byte UUIDs to 20 bytes: the chance of a collision
is remote.

sha1_uuid_full.patch
--------------------

Extend UUIDs to 21 bytes: 1 byte version (schema) plus 20 payload.

uuid_mode:
   none
   prealloc  (schema 1)  Current 2.3.8 behaviour with fix to master applied
   shortmd5  (schema 2)  First 11 bytes of message MD5, as per Fastmail.
        md5  (schema 3)  Full 16 bytes of message MD5
        sha1 (schema 4)  Full 20 bytes of message SHA1

Existing schema 1 UUIDs are filled to 21 bytes.

sha1.patch :: required by all of the above.
==========

SHA1 library that I found with Google, released under LGPL. If the
licensing is a problem, then any alternative which provides:

   void sha1( unsigned char *input, int ilen,
              unsigned char *output );

will do just fine. I imagine that it appears in a lot of crypto libraries.

Discussion
==========

sha1_uuid_full.patch is the most flexible of the three versions, and 
allows us to add further schemas (another 252) in the future.

12 to 21 bytes is an awkward change. I have done this by shrinking the 
preceding item in the index (cache_version) from 4 to 3 bytes and punting 
modseq and modseq64 up by 8 bytes. This did involve some work on 
mailbox_upgrade_index_work(), which wasn't expecting things in the middle 
of the index record to expand (and in both directions at once!).

Thoughts? I appreciate that it will take some time for these patches to be 
reviewed properly, but a ruling on which of the three potential index 
formats are preferred would allow early adopters (I imagine Fastmail and 
myself) to get on without worrying that we have an unsupported database.

-- 
David Carter                             Email: David.Carter at ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.