squatter not used after upgrade to cyrus 3.0.8

Fri May 10 08:18:12 EDT 2019

Hi Carlos,

Quoting Carlos Larrañaga <clarra at cc.upv.es>:

> Hi Michael,
>
> Do you resolved this problem?
> I'm having the same issue with cyrus 3.0.9 not accessing cyrus.squat files.
>

no the problem is not completly resolved jet.
See https://github.com/cyrusimap/cyrus-imapd/issues/2598 for details.

> I've put in my impad.conf:
>   search_engine: squat
>   search_fuzzy_always: 1

AFAIK the squat search engine does not support fuzzy search.
I am sure in my testing this setting didn't resolve the slow
search, but i don't remember if this setting did nothing,
or  failed to find anything at all or what else happened
with this configuration.

>
> Any recomendation will be appreciated.
>
> Best regards,
> Carlos.
>
>
> El 25/10/2018 a las 15:09, Michael Menge escribió:
>> Hi,
>>
>> Quoting Michael Menge <michael.menge at zdv.uni-tuebingen.de>:
>>
>>> Hi,
>>>
>>> Quoting Albert Shih <Albert.Shih at obspm.fr>:
>>>
>>>> Le 17/09/2018 à 14:01:52+0200, Michael Menge a écrit
>>>>> Hi,
>>>>>
>>>>> we recently upgrade from Cyrus-Imapd 2.4.x to 3.0.8. After some initial
>>>>> problems
>>>>> which we could fix cyrus imapd 3.0.8 is running stable. The one remaining
>>>>> problem
>>>>> we receive reports about is, that the search is not working/too slow.
>>>>>
>>>>> First we discovered that one of the options for Squatter are not
>>>>> supported
>>>>> anymore, "-s Skip mailboxes whose index file is older than their current
>>>>> squat file (within a small time delta)." and that squatter does not like
>>>>> "-r" in combination with the source "user"
>>>>>
>>>>>   > squatter -C /etc/imapd_be.conf -r  user
>>>>>   fatal error: Internal error: assertion failed: lib/cyrusdb_twoskip.c:
>>>>> 2339: keylen
>>>>>
>>>>>
>>>>> But after reindexing all mailboxes the search is still slow. I tried to
>>>>> debug this and
>>>>> found with strace that cyrus didn't try to open the cyrus.squat
>>>>> file for the
>>>>> mailbox.
>>>>>
>>>>> I suspect that I am missed some configuration change. So here is our
>>>>> imapd.conf for our backends
>>>>
>>>> If I'm correct you need :
>>>>
>>>>  search_fuzzy_always: on
>>>>
>>>> in your config.
>>>>
>>>> If you can tell me if it's work...I would really appreciate. Because I
>>>> activated that but I'm not able to see if it's work really.
>>>>
>>> Thanks for your help.
>>>
>>> I did tried it on my test server and it seems to be a bit faster,
>>> but that could be due to caching. Strace still didn't show any access
>>> to the cyrus.squat file.
>>>
>>> For information: We only use squatter. No Xapia. And we had much faster
>>> search with Cyrus-Imapd 2.3.x and 2.4.x. I don't have the timings form
>>> the old system but our users are complaining and they receive timeouts
>>> in our horde/imp webmailer, which they did't receive before.
>>>
>>> Any other ideas are appreciated.
>>
>> I still have the problem that search in cyrus imap 3.0.8 with search engine
>> squatter is slow compared to 2.4.20. I try to figure out if the squatter
>> search engine is working in cyurs imapd 3.0 and I messed up my
>> configuration,
>> or if my configuration should work but squatter is broken.
>>
>> I did set up a test environment to compare the old and new versions.
>> To verify that the search is indeed slower with 3.0.8
>>
>> I used two different searches 'B SEARCH TEXT "Test"' and 'B SEARCH
>> HEADER X-comment Unirundmail'
>>
>> === 2.4.20 === SEARCH TEXT
>>
>> A SELECT INBOX
>> * 4321 EXISTS
>> * 4321 RECENT
>> * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
>> * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] Ok
>> * OK [UNSEEN 1] Ok
>> * OK [UIDVALIDITY 1540372444] Ok
>> * OK [UIDNEXT 93369] Ok
>> * OK [HIGHESTMODSEQ 2] Ok
>> * OK [URLMECH INTERNAL] Ok
>> A OK [READ-WRITE] Completed
>> B SEARCH TEXT "Test"
>> * SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
>> 23 24 25 26 27 28 29 30 31 33 34 35 36 37 38 39
>> ....
>> 4292 4294 4295 4296 4298 4299 4300 4301 4303 4306 4307 4309 4310
>> 4315 4316 4317 4318 4321
>> B OK Completed (1996 msgs in 0.690 secs)
>>
>>
>> === 3.0.8 === SEARCH TEXT
>>
>> a SELECT INBOX
>> * 4321 EXISTS
>> * 0 RECENT
>> * FLAGS (\Answered \Flagged \Draft \Deleted \Seen NonJunk
>> $Forwarded $mdnsent $label1 $label2 $label3 hordetest testflag)
>> * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen
>> NonJunk $Forwarded $mdnsent $label1 $label2 $label3 hordetest
>> testflag \*)] Ok
>> * OK [UNSEEN 3737] Ok
>> * OK [UIDVALIDITY 1238498084] Ok
>> * OK [UIDNEXT 93373] Ok
>> * OK [HIGHESTMODSEQ 98491] Ok
>> * OK [URLMECH INTERNAL] Ok
>> * OK [ANNOTATIONS 65536] Ok
>> a OK [READ-WRITE] Completed
>> B SEARCH TEXT "Test"
>> * SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
>> 23 24 25 26 27 28 29 30 31 33 34 35 36 37 38 39
>> ....
>> 4274 4275 4276 4277 4278 4279 4285 4286 4287 4288 4292 4294 4295
>> 4296 4298 4299 4300 4301 4303 4306 4307 4309 4310 4315 4316 4317
>> 4318 4321
>> B OK Completed (1935 msgs in 2.580 secs)
>>
>> ==== 2.4.20 === SEARCH HEADER
>>
>> a SELECT INBOX
>> * 4321 EXISTS
>> * 0 RECENT
>> * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
>> * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] Ok
>> * OK [UNSEEN 1] Ok
>> * OK [UIDVALIDITY 1540372444] Ok
>> * OK [UIDNEXT 93369] Ok
>> * OK [HIGHESTMODSEQ 2] Ok
>> * OK [URLMECH INTERNAL] Ok
>> a OK [READ-WRITE] Completed
>> b SEARCH HEADER X-comment Unirundmail
>> * SEARCH 4283 4291 4313 4319 4320
>> b OK Completed (5 msgs in 0.020 secs)
>>
>> ==== 3.0.8 === SEARCH HEADER
>>
>> a SELECT INBOX
>> * 4321 EXISTS
>> * 0 RECENT
>> * FLAGS (\Answered \Flagged \Draft \Deleted \Seen NonJunk
>> $Forwarded $mdnsent $label1 $label2 $label3 hordetest testflag)
>> * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen
>> NonJunk $Forwarded $mdnsent $label1 $label2 $label3 hordetest
>> testflag \*)] Ok
>> * OK [UNSEEN 3737] Ok
>> * OK [UIDVALIDITY 1238498084] Ok
>> * OK [UIDNEXT 93373] Ok
>> * OK [HIGHESTMODSEQ 98491] Ok
>> * OK [URLMECH INTERNAL] Ok
>> * OK [ANNOTATIONS 65536] Ok
>> a OK [READ-WRITE] Completed
>> b SEARCH HEADER X-comment Unirundmail
>> * SEARCH 4283 4291 4313 4319 4320
>> b OK Completed (5 msgs in 0.370 secs)
>>
>> ===
>>
>> There is also a big discrepancy between time indicated in the "OK
>> Completed" and the time from
>> sending the search command till the return of the result, which is
>> 0.890 sec compared to ~30 sec
>> on the production system.
>>
>> I used strace on the imapd processes while searching to verify that
>> the squat file was used
>> in 2.4 but not in 3.0.
>> I could see open events for the squat file and the messages that
>> where found for 2.4.20
>> and no open event (not even a failed one) to the squat file but
>> instead open events for
>> all message files in that folder for 3.0.8
>>
>> I read the documentation and source code and tried to figure out if
>> i am missing some
>> config options, or if i could pinpoint a function where the search
>> was turning the
>> wrong way. I used "perf -g" the see the call graphs and to figure
>> out where the
>> call graphs change
>>
>> I can see that the same functions are called up to "index_search",
>> and that the called functions
>> change at that point. I know that the search code got restructured
>> to support different search
>> engines and that some functions got renamed. I have attached the
>> perf report output, so that
>> someone with a better understanding of the code can see whats going
>> on. I can provide the
>> perf.data files if it helps.
>>
>> Can someone confirm or refute that the squatter search engine is
>> working with cyrus imapd 3.0.x?
>>
>> Is "search_engine: squat" in imapd.conf and a "squatter" event in
>> cyrus.conf is sufficient to
>> use the squatter search index in 3.0 or are there other options
>> steps required.
>>
>> Regards
>>
>>    Michael Menge
>>
>> PS. link to my original post with my imapd.conf
>> https://lists.andrew.cmu.edu/pipermail/info-cyrus/2018-September/040395.html
>>
>>

--------------------------------------------------------------------------------
M.Menge                                Tel.: (49) 7071/29-70316
Universität Tübingen                   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung          mail:  
michael.menge at zdv.uni-tuebingen.de
Wächterstraße 76
72074 Tübingen