Cyrus CalDAV design decision

Wed Sep 7 03:29:48 EDT 2011

Hi Robert,

Glad to see that you've been looking in a bit more detail.

Some of the issues you've raised are definitely valid, and being addressed.

On Wednesday 07 September 2011 14.45:37 Robert Mueller wrote:
>  1. The format never seems to have ever made it to the "finished" state.
>     It's been stuck in RC states for several years. I don't think that
>     gives strong confidence to implementers that it's actually stable.

This is primarily an issue of convention, but yes, the format has not been 
formally declared final, although clients have successfully worked against the 
latest version for 7+ years, including in production.

>  2. The description is that's it's in XML, but there's no DTD or ABNF,
>     just a pseudo english format description. So there's no way to
>     clearly test that you can create/read conforming XML blobs in a
>     client implementation.

Yes, a test suite is one of the points that people in the community agree is a 
thing we want to have, and want to work on together.

>  3. The document is very incomplete in places, for example this node:

Yes. The initial authors of this went with a reference application approach.

Kontact E35 & Outlook + Toltec provide the reference for this specification. 

There are advantages and arguments why that is a useful idea in some ways, in 
particular if the reference is Free Software, and it *has* enabled a higher 
level of interoperability that other formats were able to provide.

That said, the current people driving the technology also agree it is 
underspecified and that this is not developer friendly.

So we're working on cleaning up and fleshing out the specification, for which 
we've modeled a process after the Python Enhancement Process (PEP), and 
ingeniously called the Kolab Enhancement Process (KEP), see 
http://wiki.kolab.org/KEP

>  6. For that matter, there's no complete set of examples (an mbox full
>     of example Events, Tasks, Contacts, etc) or a test suite that
>     clients should be able to read and interoperate with.

In the old thinking, the test suite are the reference implementations, which 
are put through published test plans. In the new group we agree we want more.

>  7. The Datetime format seems arbitrarily made up, but is actually the
>     ISO8601 combined extended UTC format with second precision, but you
>     don't reference ISO8601 or RFC3339 which also defines "Internet
>     time" the same way.  Likewise Date.

It is actually a RFC3339 subset, although not specifically defined as such.

Unfortunately it had some real issues which have to do with time zones and 
recurring events across them, so we had to do a bit more work on them, see
http://wiki.kolab.org/KEP:2

>  8. Recurrence seems primitive. Recurrence across DST boundaries seems
>     to be as broken as every other format.  There doesn't seem to be a
>     way to specify public holiday exclusions.

The DST boundary thing should be fixed with KEP 2. 

The public holiday exclusions & co are a good point, and something we could 
address through another KEP.

>  9. The docs talk about ANNOTATEMORE more everywhere, which really needs
>     updating to include METADATA and the final RFC

KEP 9 will fix this:

http://wiki.kolab.org/User:Greve/Drafts/KEP:9#Configuration_storage_in_folder_annotations

> 10. The /vendor/kolab/folder-type annotation should be updated now that
>     SPECIALUSE has been made an RFC

I think that is a very good idea. This should likely be added to KEP 9, which 
is still in drafting stage, so easy to extend.

> When you add these things together, I don't think it makes it easy to
> create a client or server for the Kolab format.

There are quite a few practical counterexamples to that, e.g. the recent 
client for the Evolution Data Server to hook in GNOME Evolution, or the people 
who are playing with Kolab Android plugins & a Free Software Outlook plugin.

But yes, I think we can reduce the barrier more.

> And I think that goes part of the way to explaining why the quality of
> alternate clients out there hasn't exactly been great. I've tried Bynari
> (Outlook) and SyncKolab (Thunderbird), and their implementations are
> pretty bad.

SyncKolab was a vacation project of a single person, and done without much 
interaction. We're looking at changing that, but right now it is clearly not 
on the list of recommended clients.

As to the Bynari plugin, I wonder which version you tried, because when 
testing this against a Kolab Server with the Kontact test plan, we got not 
perfect, but fairly decent results.

> 1. The Kolab Format spec needs to be completely finished. Edge cases
>    cleaned up, todo parts finished, and unspecified bits specified
>    completely.

Agreed.

> 2. It needs to be marked as a true 2.0 version, not an RC. You can add
>    2.1 later if you want, but I think the eternal RC status just leaves
>    it feeling abandoned.

Actually we're thinking of bouncing versions to at least 2.1, and re-sync the 
version of the overall format, and the XML object version, because the current 
state of "Kolab Format Version 2.0 defines Kolab XML Version 1.0" is confusing.

> 3. We need some sort of test suite/code. That should consist of at least
>    two main things: [...]

Agreed.

> Yes, that would require a bit of work from someone, but I totally think
> it's worth it, and I think it would go a long way to helping more
> implementers work on the Kolab format in the future.

Fully agreed.

We've begun the work, as you can see, and are cleaning things up step by step.

Because we have a grown community with productive environments and customers 
we cannot just have one person go through there and redefine things to his or 
her whim, it needs to be coordinated with the various client applications, 
which is at least in part a consensus building process.

But keeping the level of interoperability between clients high seems a strong 
enough incentive to go through this with the community.

And yes, we're contributing substantial resources to this process, and are 
willing to do more, especially in collaboration with others. For some of these 
aspects the lack of feedback can be more blocking than a lack of resource 
within the company. 

So the engagement of yourself and others would be very much welcome, and 
you're even invited to work with us driving KEPs on issues where you feel the 
motivation and need to speed things up.

Best regards,
Georg

-- 
Georg C. F. Greve
Chief Executive Officer

Kolab Systems AG
Zürich, Switzerland

e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com

pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
Url : http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20110907/99f9e9b0/attachment.bin 