<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<font face="Calibri">Hi Michael,<br>
<br>
Do you resolved this problem?<br>
I'm having the same issue with cyrus 3.0.9 not accessing
cyrus.squat files. <br>
<br>
I've put in my impad.conf:<br>
search_engine: squat<br>
search_fuzzy_always: 1<br>
<br>
Any recomendation will be appreciated.<br>
<br>
Best regards,<br>
Carlos.<br>
<br>
</font><br>
<div class="moz-cite-prefix">El 25/10/2018 a las 15:09, Michael
Menge escribió:<br>
</div>
<blockquote type="cite"
cite="mid:20181025150940.Horde.ZSBGZc-bdo4Ww0nHKDroxYH@webmail.uni-tuebingen.de">Hi,
<br>
<br>
Quoting Michael Menge <a class="moz-txt-link-rfc2396E" href="mailto:michael.menge@zdv.uni-tuebingen.de"><michael.menge@zdv.uni-tuebingen.de></a>:
<br>
<br>
<blockquote type="cite">Hi,
<br>
<br>
Quoting Albert Shih <a class="moz-txt-link-rfc2396E" href="mailto:Albert.Shih@obspm.fr"><Albert.Shih@obspm.fr></a>:
<br>
<br>
<blockquote type="cite">Le 17/09/2018 à 14:01:52+0200, Michael
Menge a écrit
<br>
<blockquote type="cite">Hi,
<br>
<br>
we recently upgrade from Cyrus-Imapd 2.4.x to 3.0.8. After
some initial
<br>
problems
<br>
which we could fix cyrus imapd 3.0.8 is running stable. The
one remaining
<br>
problem
<br>
we receive reports about is, that the search is not
working/too slow.
<br>
<br>
First we discovered that one of the options for Squatter are
not supported
<br>
anymore, "-s Skip mailboxes whose index file is older than
their current
<br>
squat file (within a small time delta)." and that squatter
does not like
<br>
"-r" in combination with the source "user"
<br>
<br>
> squatter -C /etc/imapd_be.conf -r user
<br>
fatal error: Internal error: assertion failed:
lib/cyrusdb_twoskip.c:
<br>
2339: keylen
<br>
<br>
<br>
But after reindexing all mailboxes the search is still slow.
I tried to
<br>
debug this and
<br>
found with strace that cyrus didn't try to open the
cyrus.squat file for the
<br>
mailbox.
<br>
<br>
I suspect that I am missed some configuration change. So
here is our
<br>
imapd.conf for our backends
<br>
</blockquote>
<br>
If I'm correct you need :
<br>
<br>
search_fuzzy_always: on
<br>
<br>
in your config.
<br>
<br>
If you can tell me if it's work...I would really appreciate.
Because I
<br>
activated that but I'm not able to see if it's work really.
<br>
<br>
</blockquote>
Thanks for your help.
<br>
<br>
I did tried it on my test server and it seems to be a bit
faster,
<br>
but that could be due to caching. Strace still didn't show any
access
<br>
to the cyrus.squat file.
<br>
<br>
For information: We only use squatter. No Xapia. And we had much
faster
<br>
search with Cyrus-Imapd 2.3.x and 2.4.x. I don't have the
timings form
<br>
the old system but our users are complaining and they receive
timeouts
<br>
in our horde/imp webmailer, which they did't receive before.
<br>
<br>
Any other ideas are appreciated.
<br>
</blockquote>
<br>
I still have the problem that search in cyrus imap 3.0.8 with
search engine
<br>
squatter is slow compared to 2.4.20. I try to figure out if the
squatter
<br>
search engine is working in cyurs imapd 3.0 and I messed up my
configuration,
<br>
or if my configuration should work but squatter is broken.
<br>
<br>
I did set up a test environment to compare the old and new
versions.
<br>
To verify that the search is indeed slower with 3.0.8
<br>
<br>
I used two different searches 'B SEARCH TEXT "Test"' and 'B SEARCH
HEADER X-comment Unirundmail'
<br>
<br>
=== 2.4.20 === SEARCH TEXT
<br>
<br>
A SELECT INBOX
<br>
* 4321 EXISTS
<br>
* 4321 RECENT
<br>
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
<br>
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen
\*)] Ok
<br>
* OK [UNSEEN 1] Ok
<br>
* OK [UIDVALIDITY 1540372444] Ok
<br>
* OK [UIDNEXT 93369] Ok
<br>
* OK [HIGHESTMODSEQ 2] Ok
<br>
* OK [URLMECH INTERNAL] Ok
<br>
A OK [READ-WRITE] Completed
<br>
B SEARCH TEXT "Test"
<br>
* SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
23 24 25 26 27 28 29 30 31 33 34 35 36 37 38 39
<br>
....
<br>
4292 4294 4295 4296 4298 4299 4300 4301 4303 4306 4307 4309 4310
4315 4316 4317 4318 4321
<br>
B OK Completed (1996 msgs in 0.690 secs)
<br>
<br>
<br>
=== 3.0.8 === SEARCH TEXT
<br>
<br>
a SELECT INBOX
<br>
* 4321 EXISTS
<br>
* 0 RECENT
<br>
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen NonJunk
$Forwarded $mdnsent $label1 $label2 $label3 hordetest testflag)
<br>
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen
NonJunk $Forwarded $mdnsent $label1 $label2 $label3 hordetest
testflag \*)] Ok
<br>
* OK [UNSEEN 3737] Ok
<br>
* OK [UIDVALIDITY 1238498084] Ok
<br>
* OK [UIDNEXT 93373] Ok
<br>
* OK [HIGHESTMODSEQ 98491] Ok
<br>
* OK [URLMECH INTERNAL] Ok
<br>
* OK [ANNOTATIONS 65536] Ok
<br>
a OK [READ-WRITE] Completed
<br>
B SEARCH TEXT "Test"
<br>
* SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
23 24 25 26 27 28 29 30 31 33 34 35 36 37 38 39
<br>
....
<br>
4274 4275 4276 4277 4278 4279 4285 4286 4287 4288 4292 4294 4295
4296 4298 4299 4300 4301 4303 4306 4307 4309 4310 4315 4316 4317
4318 4321
<br>
B OK Completed (1935 msgs in 2.580 secs)
<br>
<br>
==== 2.4.20 === SEARCH HEADER
<br>
<br>
a SELECT INBOX
<br>
* 4321 EXISTS
<br>
* 0 RECENT
<br>
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
<br>
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen
\*)] Ok
<br>
* OK [UNSEEN 1] Ok
<br>
* OK [UIDVALIDITY 1540372444] Ok
<br>
* OK [UIDNEXT 93369] Ok
<br>
* OK [HIGHESTMODSEQ 2] Ok
<br>
* OK [URLMECH INTERNAL] Ok
<br>
a OK [READ-WRITE] Completed
<br>
b SEARCH HEADER X-comment Unirundmail
<br>
* SEARCH 4283 4291 4313 4319 4320
<br>
b OK Completed (5 msgs in 0.020 secs)
<br>
<br>
==== 3.0.8 === SEARCH HEADER
<br>
<br>
a SELECT INBOX
<br>
* 4321 EXISTS
<br>
* 0 RECENT
<br>
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen NonJunk
$Forwarded $mdnsent $label1 $label2 $label3 hordetest testflag)
<br>
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen
NonJunk $Forwarded $mdnsent $label1 $label2 $label3 hordetest
testflag \*)] Ok
<br>
* OK [UNSEEN 3737] Ok
<br>
* OK [UIDVALIDITY 1238498084] Ok
<br>
* OK [UIDNEXT 93373] Ok
<br>
* OK [HIGHESTMODSEQ 98491] Ok
<br>
* OK [URLMECH INTERNAL] Ok
<br>
* OK [ANNOTATIONS 65536] Ok
<br>
a OK [READ-WRITE] Completed
<br>
b SEARCH HEADER X-comment Unirundmail
<br>
* SEARCH 4283 4291 4313 4319 4320
<br>
b OK Completed (5 msgs in 0.370 secs)
<br>
<br>
===
<br>
<br>
There is also a big discrepancy between time indicated in the "OK
Completed" and the time from
<br>
sending the search command till the return of the result, which is
0.890 sec compared to ~30 sec
<br>
on the production system.
<br>
<br>
I used strace on the imapd processes while searching to verify
that the squat file was used
<br>
in 2.4 but not in 3.0.
<br>
I could see open events for the squat file and the messages that
where found for 2.4.20
<br>
and no open event (not even a failed one) to the squat file but
instead open events for
<br>
all message files in that folder for 3.0.8
<br>
<br>
I read the documentation and source code and tried to figure out
if i am missing some
<br>
config options, or if i could pinpoint a function where the search
was turning the
<br>
wrong way. I used "perf -g" the see the call graphs and to figure
out where the
<br>
call graphs change
<br>
<br>
I can see that the same functions are called up to "index_search",
and that the called functions
<br>
change at that point. I know that the search code got restructured
to support different search
<br>
engines and that some functions got renamed. I have attached the
perf report output, so that
<br>
someone with a better understanding of the code can see whats
going on. I can provide the
<br>
perf.data files if it helps.
<br>
<br>
Can someone confirm or refute that the squatter search engine is
working with cyrus imapd 3.0.x?
<br>
<br>
Is "search_engine: squat" in imapd.conf and a "squatter" event in
cyrus.conf is sufficient to
<br>
use the squatter search index in 3.0 or are there other options
steps required.
<br>
<br>
Regards
<br>
<br>
Michael Menge
<br>
<br>
PS. link to my original post with my imapd.conf
<br>
<a class="moz-txt-link-freetext" href="https://lists.andrew.cmu.edu/pipermail/info-cyrus/2018-September/040395.html">https://lists.andrew.cmu.edu/pipermail/info-cyrus/2018-September/040395.html</a>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
--------------------------------------------------------------------------------
<br>
M.Menge Tel.: (49) 7071/29-70316
<br>
Universität Tübingen Fax.: (49) 7071/29-5912
<br>
Zentrum für Datenverarbeitung mail:
<a class="moz-txt-link-abbreviated" href="mailto:michael.menge@zdv.uni-tuebingen.de">michael.menge@zdv.uni-tuebingen.de</a>
<br>
Wächterstraße 76
<br>
72074 Tübingen
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">----
Cyrus Home Page: <a class="moz-txt-link-freetext" href="http://www.cyrusimap.org/">http://www.cyrusimap.org/</a>
List Archives/Info: <a class="moz-txt-link-freetext" href="http://lists.andrew.cmu.edu/pipermail/info-cyrus/">http://lists.andrew.cmu.edu/pipermail/info-cyrus/</a>
To Unsubscribe:
<a class="moz-txt-link-freetext" href="https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus">https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus</a></pre>
</blockquote>
<br>
</body>
</html>