LIST-EXT and list_data rewrite
murch at andrew.cmu.edu
Thu Jun 30 09:39:33 EDT 2011
On 6/30/2011 9:35 AM, Bron Gondwana wrote:
> I've been looking at various bugs with LIST throughout 2.4.x, with
> LSUB and LIST and stuff issues. Most of them are now fixed in stable and
> in master, but there's still some things that the dovecot 'imaptest' tool
> The core problem is this:
> we have two functions, mboxlist_findall and mboxlist_findsub, which come in
> both altnamespace and regular flavours.
> If you have a query on (children subscribed) then you need to find anything
> that either has children OR is subscribed, which requires both lots of find
> to run kind of concurrently, which is impossible.
> At the moment, the callback function to either of these checks the name to
> see if either:
> a) it can be matched exactly by the LIST filter
> b) one of its parents can be matched by the list filter.
> if (b), then if the parent has already been seen, we just mark it as having
> children, and move on. Otherwise we create a "NonExistant" item and set
> HasChildren on it.
> Whenever we get another match, we move on. This kind of works, until you
> have a LIST-EXT query that wants to find folder that are either subscribed
> or have children!
> So what I'm looking at doing is replacing all this with an in-memory list
> of folders which will be in the final result, and we can add to it and
> then sort it and merge before output. This means we can fully pass
> (I guess the other alternative is to build an in-memory list from the LSUB
> stuff if required, and just walk that list. It's guaranteed to be less
> wasteful of memory, but probably more complex...)
Whatever you think is easiest to implement and easiest to understand.
The LIST/LSUB code has always been complex and somewhat fragile -- its
like that Jenga game, you move the wrong piece and the whole thing falls
Principal Systems Software Engineer
Carnegie Mellon University
More information about the Cyrus-devel