Future Ideas wiki page

[David Lang]

On Wed, 13 Jan 2010, Bron Gondwana wrote:

> On Wed, 13 Jan 2010 17:40 -0800, "David Lang" <david.lang at digitalinsight.com> wrote:
>>> Absolutely - two issues.
>>>
>>> 1: how to you give folders UIDs?
>>
>> I thought that there was mention in your list of addressing folders by
>> UID for
>> replication purposes.
>
> UniqueID - it's an internal 16 hex digit field.  Tacking one of those on every
> result item would be expensive for bandwidth.


If you have a lot of of messages being identified it will be noticable, but I'll 
point out that the user dowloading one word doc attachement is several thousand 
messages worth of IDs. Another option would be to use the folder name (but you 
would have to worry about escaping some characters, etc)

>>> The first one is the killer, because you'd have to be able to map back to select
>>> by UID as well.  SELECT 195455, or something.
>>>
>>> Actually, the whole "SELECTED" idea would probably be discarded, instead just
>>> addressing everything by UID and never actually selecting folders.
>>
>> yes, for some things this is extremely useful for others it's just added
>> complication.
>
> There's not much that it's useful for with cross folder work!
>
> Internally our web interface actually already does exactly what you describe so it
> can support cross folder searching.  Every UnqMsg is <folderid>:<uid>, where
> FolderId is a reference to a database field which contains a mapping to the
> underlying folder.  We also store a bunch of metadata about how to present that
> particular folder to the user (number of messages to display, type of preview,
> default sort, etc)

if the only thing that can really use this is cross folder searching, then only 
implement this as output from the server in search results (and possibly sorting 
commands), allow it for input (message specification) for other commands. Worst 
case you could require the client to select the folder.

One other thing that something like this would be useful for is for things that 
want to watch for messages from any of several folders. Today they need to 
either open a seperate connection for each folder they want to watch, or cycle 
through the folders polling them.

I suspect that if this addressing mechanism was implemented, it would be about 
as easy to add it everywhere that UID is an option as it would be to only add it 
to some commands.

>>> By which time, why not just define a brand new protocol not called IMAP which
>>> includes the good bits of what IMAP currently does, and discards anything that
>>> doesn't fit the multi-folder worldview.  So long as you made the storage and
>>> meta-data requirements compatible with already existing Cyrus and other IMAP
>>> servers, you could just write a whole new daemon that talked your new protocol
>>> and be happy with that.
>>
>> you could, but just like not every client uses UIDs and still use message
>> numbers, this is being done in a backwards compatible manner so that the
>> client
>> can use either one, and a server can support both.
>
> The problem is you keep paying a higher and higher complexity cost at the server end.
> Once you start talking not having a folder selected, that cost (and associated
> locking issues) is going to blow your implementation complexity way up.  Cross folder is
> far enough away from the initial design of IMAP that the impedance mismatch is grating
> pretty badly!

I'm missing the locking issues here. even with single folder searching you can 
have a message disappear immediatly after the search results have been provided. 
Folders don't get locked when they are selected. What is it that needs to be 
locked for this sort of addressing?

I agree with the climbing complexity, but is there really so much stuff that 
_everyone_ agrees should be dumped with IMAPv4 that it's worth the break? If so 
it may be time for IMAPv5, but have we really moved that far?

Most of the time you really do want to act on one folder at a time, and as such 
selecting that folder makes sense.

David Lang