<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /></head><body style='font-size: 10pt; font-family: Verdana,Geneva,sans-serif'>
<p>Hi Sebastian!!</p>
<p><br /></p>
<p>Thank you so much mate!!. Point 2 is the most uncertain for me. I answer your comments below :)</p>
<p><br /></p>
<p>Point 1 -> It's clear... yes I think so too... I was planning to force all searches as FUZZY with <span style="white-space: nowrap;">search_fuzzy_always</span>: 1 as it seems really nice :)</p>
<p>Point 2 -> So, you do a df every two minutes to see free space in tmpfs?... and when you see it starts growing you launch a "<span style="white-space: nowrap;">sudo -u cyrus </span><span style="white-space: nowrap;">/usr/cyrus/bin/squatter -C /etc/cyrus/imapd-sloti30t01.conf -v -z </span><span style="white-space: nowrap;">archive -t temp,meta,data,archive -u brong</span>" equivalent command?. You would launch it without -u I suppose, for doing Squatter for all users (no sense of doing it for just one user... isn't it?)... I assume this command would move all items from temp, meta and data to archive database. So, for instance if you launch a "<span style="white-space: nowrap;">sudo -u cyrus </span><span style="white-space: nowrap;">/usr/cyrus/bin/squatter -C /etc/cyrus/imapd-sloti30t01.conf -v -z </span><span style="white-space: nowrap;">data -t temp,meta,data -u brong</span>" it moves (and then empties temp and meta) temp and meta to data and even compacts the own data database?. We usually have 2000 imap concurrent connections, 150 concurrent pop3 and receive 120 messages/min more or less... and SSD array of disks yes.. for each server...</p>
<p>Point 3 -> Totally agree. Just for confirming.</p>
<p>Point 4 -> If having a crash in a server... the Cyrus log file, would throw which mailboxes need a ctl_conversations -z and -b or "it's better" doing it for all accounts (with -r and without -u?)?. When talking about Xapian databases (not conversations) I assume some error would appear for them surely... the same way as for instance if cyrus.seen would be corrupt... isn't it?.</p>
<p><br /></p>
<p>Thanks a lot again. Your help is really really important for at this stage...</p>
<p><br /></p>
<p> Cheers!!</p>
<div>---<br />
<div class="firmasarenet" style="clear: both; text-align: left;">
<div class="imgsarenet" style="margin: 0 0 10px 0;"><img src="https://www.sarenet.es/estaticos/LogoSarenetEmails.png?newlogo" alt="sarenet" /></div>
<div class="titulosarenet" style="font-family: Helvetica, Arial, sans-serif; font-weight: bold; font-size: 14px; color: #333333;"><strong>Egoitz Aurrekoetxea</strong></div>
<div class="dptosarenet" style="font-family: Helvetica, Arial, sans-serif; font-weight: normal; font-size: 12px; color: #333333;">Departamento de sistemas</div>
<div class="textosarenet" style="font-family: Helvetica, Arial, sans-serif; font-weight: normal; font-size: 12px; color: #333333;">944 209 470<br />Parque Tecnológico. Edificio 103<br />48170 Zamudio (Bizkaia)</div>
<div class="lnksarenet" style="font-family: Helvetica, Arial, sans-serif; font-weight: normal; font-size: 12px; color: #007ac4; line-height: 1.2;"><a id="mailto" style="font-size: 12px; color: #007ac4; text-decoration: underline;" href="mailto:egoitz@sarenet.es"><label id="label_email">egoitz@sarenet.es</label></a></div>
<div class="lnksarenet" style="font-family: Helvetica, Arial, sans-serif; font-weight: normal; font-size: 12px; color: #007ac4; line-height: 1.2;"><a style="font-size: 12px; color: #007ac4; text-decoration: underline;" href="http://www.sarenet.es">www.sarenet.es</a></div>
<br />
<div class="imprimirsarenet" style="font-family: Helvetica, Arial, sans-serif; font-weight: normal; font-size: 10px; color: #a0c361;">Antes de imprimir este correo electrónico piense si es necesario hacerlo.</div>
</div>
</div>
<p><br /></p>
<p>El 09-01-2019 12:59, Sebastian Hagedorn escribió:</p>
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><!-- html ignored --><!-- head ignored --><!-- meta ignored -->
<div class="pre" style="margin: 0; padding: 0; font-family: monospace"><span style="white-space: nowrap;">Hi,</span><br /> <br /> <span style="white-space: nowrap;">I will try to answer what I can ... see below.</span><br /> <br /> --On 8. Januar 2019 um 18:40:17 +0100 Egoitz Aurrekoetxea <<a href="mailto:egoitz@sarenet.es">egoitz@sarenet.es</a>> wrote:<br /> <br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><span style="white-space: nowrap;">- search_fuzzy_always: 1 , causes all searches to go through Xapian</span><br /> <span style="white-space: nowrap;">engine. Being so good as it seems (and the way it speeds in my testing</span><br /> <span style="white-space: nowrap;">env search operations, it's nice!!), what could be the reason for not</span><br /> <span style="white-space: nowrap;">having it enabled by default?. Can it have some kind of problem?. I</span><br /> <span style="white-space: nowrap;">can't see them... Just for avoiding surprises.</span></blockquote>
<br /> With FUZZY you may get more matches than without. Look here for an explanation:<br /> <br /> <span style="white-space: nowrap;"><<a href="https://tools.ietf.org/html/rfc6203" target="_blank" rel="noopener noreferrer">https://tools.ietf.org/html/rfc6203</a>></span><br /> <br /> <span style="white-space: nowrap;">The example in 3. is a good one:</span><br /> <br /> <span style="white-space: nowrap;">3. The FUZZY Search Key</span><br /> <br /> <span style="white-space: nowrap;"> The FUZZY search key takes another search key as its argument. The</span><br /> <span style="white-space: nowrap;"> server is allowed to perform all matching in an implementation-</span><br /> <span style="white-space: nowrap;"> defined manner for this search key, including ignoring the active</span><br /> <span style="white-space: nowrap;"> comparator as defined by [RFC5255]. Typically, this would be used to</span><br /> <span style="white-space: nowrap;"> search for strings. For example:</span><br /> <br /> <span style="white-space: nowrap;"> C: A1 SEARCH FUZZY (SUBJECT "IMAP break")</span><br /> <span style="white-space: nowrap;"> S: * SEARCH 1 5 10</span><br /> <span style="white-space: nowrap;"> S: A1 OK Search completed.</span><br /> <br /> <span style="white-space: nowrap;"> Besides matching messages with a subject of "IMAP break", the above</span><br /> <span style="white-space: nowrap;"> search may also match messages with subjects "broken IMAP", "IMAP is</span><br /> <span style="white-space: nowrap;"> broken", or anything else the server decides that might be a good</span><br /> <span style="white-space: nowrap;"> match.</span><br /> <br /> Note that the *server* decides what "might be a good match". When all searches become FUZZY it might confuse users, but on the other hand I doubt there is a single IMAP client that lets the user choose whether a particular search should be FUZZY or not ...<br /> <br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><span style="white-space: nowrap;">- Bron, in this post</span><br /> <span style="white-space: nowrap;"><a href="https://fastmail.blog/2014/12/01/email-search-system/" target="_blank" rel="noopener noreferrer">https://fastmail.blog/2014/12/01/email-search-system/</a> told that Fastmail</span><br /> <span style="white-space: nowrap;">was not able to handle with just one Xapian default database (even on</span><br /> <span style="white-space: nowrap;">SSD disks) all traffic. So he said Fastmail was using a in-memory</span><br /> <span style="white-space: nowrap;">filesystem for a database (called temp) for new email. Later another for</span><br /> <span style="white-space: nowrap;">cleaning up that in memory filesystem. And later one more, for keeping</span><br /> <span style="white-space: nowrap;">definitively the content. You seemed to use a Squatter command for</span><br /> <span style="white-space: nowrap;">moving elements between databases. Concretely (sudo -u cyrus</span><br /> <span style="white-space: nowrap;">/usr/cyrus/bin/squatter -C /etc/cyrus/imapd-sloti30t01.conf -v -z</span><br /> <span style="white-space: nowrap;">archive -t temp,meta,data,archive -u brong). I assume that compacts all</span><br /> <span style="white-space: nowrap;">elements from all databases to archive?. If I wanted to compact elements</span><br /> <span style="white-space: nowrap;">from temp to data, the command should be "sudo -u cyrus</span><br /> <span style="white-space: nowrap;">/usr/cyrus/bin/squatter -C /etc/cyrus/imapd-sloti30t01.conf -v -z data</span><br /> <span style="white-space: nowrap;">-t temp -u brong" (in this example for user brong) ??. I assume you</span><br /> <span style="white-space: nowrap;">launch something like it with a cron weekly and it's done?.</span></blockquote>
<br /> That depends entirely on your system. Without knowing the number of users you have, the number of mails that arrive per hour, what kind of storage systems you have, how much free RAM your servers have, it is impossible to say how often you need to squatter in compact mode. On a test server I set up I run a cron job every 2 minutes to check if tmpfs is getting tight ...<br /> <br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><span style="white-space: nowrap;">- If something went wrong when the upgrade proccess from 2.4 to 3.0,</span><br /> <span style="white-space: nowrap;">could I setup 3.0 as master of the 2.4 and later make 2.4 master again?.</span><br /> <span style="white-space: nowrap;">Could that cause info loosing?. I assume yes but just</span><br /> <span style="white-space: nowrap;">for knowing posibilities.</span></blockquote>
<br /> You have to describe in more detail what you mean. The point of no return is usually when you start to deliver new mail to the new server. You cannot sync from 3.0 to 2.4. That means if you start to deliver new mail to the 3.0 server, you can't go back to 2.4 without losing the messages that have arrived n the meantime. Strictly speaking to could manually copy the mailboxes from the 3.0 server to the 2.4 and run reconstruct, but I don't consider that a viable option.<br /> <br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><span style="white-space: nowrap;">- When a Xapian or conversation index becomes broken, a reconstruct</span><br /> <span style="white-space: nowrap;">could recover?. What could be the repairing procedure?.</span></blockquote>
<br /> ctl_conversationsdb -z "zaps" the conversationsdb – it basically empties it. Then you can recreate it with ctl_conversationsdb -b. The "reconstruct" command does not touch the conversationsdb. If the actual Xapian index should be broken, I guess you'd have to delete the index files and run squatter again.<br /> <br /> Cheers<br /> Sebastian</div>
</blockquote>
</body></html>