slow index lookups on large mailboxes

[Bron Gondwana]

On Wed, Aug 1, 2012, at 11:20 PM, Deniss wrote:
> hello here,
> 
> after upgrade of cyrus from 2.3 to 2.4 I noticed some slowdown on few 
> huge mailboxes doing multiple FETCH UID queries. I briefly checked a 
> codebase of new cyrus and found out that to find any required record in 
> index the new code loops through all records and output matched data.
> for example:
>      for (msgno = 1; msgno <= state->exists; msgno++) {
>          im = &state->map[msgno-1];
>          checkval = usinguid ? im->record.uid : msgno;
>          if (!seqset_ismember(seq, checkval))
>              continue;
>          r = index_fetchreply(state, msgno, fetchargs);
>          if (r) break;
>          fetched = 1;
>      }
> 
> This approach brings constant time of index query for any numbers of 
> requested records. But for large indexes and single record requested 
> this algorithm is clearly ineffective.
> 
> is there plans to tune index lookups in future ?

Yes:

http://git.cyrusimap.org/cyrus-imapd/commit/?id=6b8c92d9ea92a74e508a6399a1a1dc5fe55936d3

(and bug fixes afterwards)

it's in -master, so will be in 2.5.

Bron.
-- 
  Bron Gondwana
  brong at fastmail.fm