slow index lookups on large mailboxes

[Deniss]

will that commit be backported to 2.4 branch ?
is here any problem to backport it ?

On 2012.08.02. 11:05, Bron Gondwana wrote:
> 
> 
> On Wed, Aug 1, 2012, at 11:20 PM, Deniss wrote:
>> hello here,
>>
>> after upgrade of cyrus from 2.3 to 2.4 I noticed some slowdown on few 
>> huge mailboxes doing multiple FETCH UID queries. I briefly checked a 
>> codebase of new cyrus and found out that to find any required record in 
>> index the new code loops through all records and output matched data.
>> for example:
>>      for (msgno = 1; msgno <= state->exists; msgno++) {
>>          im = &state->map[msgno-1];
>>          checkval = usinguid ? im->record.uid : msgno;
>>          if (!seqset_ismember(seq, checkval))
>>              continue;
>>          r = index_fetchreply(state, msgno, fetchargs);
>>          if (r) break;
>>          fetched = 1;
>>      }
>>
>> This approach brings constant time of index query for any numbers of 
>> requested records. But for large indexes and single record requested 
>> this algorithm is clearly ineffective.
>>
>> is there plans to tune index lookups in future ?
> 
> Yes:
> 
> http://git.cyrusimap.org/cyrus-imapd/commit/?id=6b8c92d9ea92a74e508a6399a1a1dc5fe55936d3
> 
> (and bug fixes afterwards)
> 
> it's in -master, so will be in 2.5.
> 
> Bron.
>