<!DOCTYPE html><html><head><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div style="font-family:Arial;">On Wed, Feb 13, 2019, at 00:39, Michael Menge wrote:<br></div><blockquote type="cite" id="fastmail-quoted"><div style="font-family:Arial;">Hi Bron,<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">sorry, i had to rearrange some quotes to put them my answers in a more <br></div><div style="font-family:Arial;">meaningful order.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">Quoting Bron Gondwana <brong@fastmailteam.com>:<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">>> >> The file was already at <br></div><div style="font-family:Arial;">>> /srv/cyrus-hdd-be/archive/ssd-part/L/user/XXXX/2185.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">I was able to fix these problems with reconstruct, and the didn't <br></div><div style="font-family:Arial;">reappear till now.<br></div><div style="font-family:Arial;">Also there where other accounts which had IOERRORS regarding the <br></div><div style="font-family:Arial;">conversation db,<br></div><div style="font-family:Arial;">with no cyr_expire archive errors, so i believe that these problems <br></div><div style="font-family:Arial;">are not related.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">I tried rebuilding the conversation db for the accounts with errors, <br></div><div style="font-family:Arial;">but some other<br></div><div style="font-family:Arial;">accounts will show up with errors some time later. I counldn't find a <br></div><div style="font-family:Arial;">some thing in<br></div><div style="font-family:Arial;">common jet.<br></div></blockquote><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">OK. That's hard to disagnore from remotely :(<br></div><div style="font-family:Arial;"><br></div><blockquote type="cite" id="fastmail-quoted"><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">>> >> > Anyway, I don't think that would break anything.<br></div><div style="font-family:Arial;">>> >> ><br></div><div style="font-family:Arial;">>> >> > metapartition-ssd: /srv/cyrus-ssd-be/meta/ssd-part<br></div><div style="font-family:Arial;">>> >> > metapartition_files: header index cache expunge squat annotations<br></div><div style="font-family:Arial;">>> >> > lock dav archivecache<br></div><div style="font-family:Arial;">>> >> ><br></div><div style="font-family:Arial;">>> >> > Ooh, I haven't tested having cache and archivecache on the same<br></div><div style="font-family:Arial;">>> >> > location. That's really interesting. Again, I'd be in favour of<br></div><div style="font-family:Arial;">>> >> > separation here, give them different paths. That might be tricky<br></div><div style="font-family:Arial;">>> >> > with ssd though, the way this is laid out. I assume you have some<br></div><div style="font-family:Arial;">>> >> > kind of symlink farm going on?<br></div><div style="font-family:Arial;">>> >> ><br></div><div style="font-family:Arial;">>> >><br></div><div style="font-family:Arial;">>> >> I didn't know that there could be a problem with cache and archivecache.<br></div><div style="font-family:Arial;">>> >> At the time we decided on the configuration for cyrus 3.0 I looked at the<br></div><div style="font-family:Arial;">>> >> imapd.conf man page and for metapartition_files decided that I want all<br></div><div style="font-family:Arial;">>> >> meta files on the ssd storage. There was no indication in the man page<br></div><div style="font-family:Arial;">>> >> that there could be a problem.<br></div><div style="font-family:Arial;">>> ><br></div><div style="font-family:Arial;">>> > Fair. I'd have to test that to see if it works correctly. I would<br></div><div style="font-family:Arial;">>> > hope so, but I haven't tested that configuration. This is the<br></div><div style="font-family:Arial;">>> > downside with having lots of different ways to do things!<br></div><div style="font-family:Arial;">>> ><br></div><div style="font-family:Arial;">>> >> How do I separate location of archivecache from the other<br></div><div style="font-family:Arial;">>> >> metapartition path?<br></div><div style="font-family:Arial;">>> >> And fix the cache and archivecache files?<br></div><div style="font-family:Arial;">>> ><br></div><div style="font-family:Arial;">>> > This I don't know a good answer for. I will test if having the same<br></div><div style="font-family:Arial;">>> > path for cache and archivecache could fail. I THINK that I made the<br></div><div style="font-family:Arial;">>> > code safe for it, but I'm not sure that it's been tested.<br></div><div style="font-family:Arial;">>> ><br></div><div style="font-family:Arial;">>> >> No there is no sysmlink farm. We have mounted different iSCSI volumes to<br></div><div style="font-family:Arial;">>> >> /srv/cyrus-ssd-be, /srv/cyrus-hdd-be and /srv/cyrus-be<br></div><div style="font-family:Arial;">>> ><br></div><div style="font-family:Arial;">>> > Right. That makes sense.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">Did you have time to look into the cache/archivecache situation jet?<br></div></blockquote><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">Yes, I've looked at the code!<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">in mailbox_archive():<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;"> /* got a new cache record to write */<br></div><div style="font-family:Arial;"> if (differentcache)<br></div><div style="font-family:Arial;"> {<br></div><div style="font-family:Arial;"> dirtycache = 1;<br></div><div style="font-family:Arial;"> copyrecord.cache_offset = 0;<br></div><div style="font-family:Arial;"> if (mailbox_append_cache(mailbox, ©record))<br></div><div style="font-family:Arial;"> continue;<br></div><div style="font-family:Arial;"> }<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">And the code for differentcache is:<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;"> char *spoolcache = xstrdup(mailbox_meta_fname(mailbox, META_CACHE));<br></div><div style="font-family:Arial;"> char *archivecache = xstrdup(mailbox_meta_fname(mailbox, META_ARCHIVECACHE));<br></div><div style="font-family:Arial;"> int differentcache = strcmp(spoolcache, archivecache);<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">So it looks like the answer is cache/archivecache is fine. It is not your problem.<br></div><div style="font-family:Arial;"><br></div><blockquote type="cite" id="fastmail-quoted"><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">>> > Right! I do wonder if there are some bugs in 3.0.x which are fixed<br></div><div style="font-family:Arial;">>> > on master around delivery to archive partition. We definitely had<br></div><div style="font-family:Arial;">>> > bugs on master, but I thought they were newly introduced on master<br></div><div style="font-family:Arial;">>> > as well, which is why the fixes weren't backported. But if you're<br></div><div style="font-family:Arial;">>> > having files be in the wrong location, maybe there are bugs on 3.0.x<br></div><div style="font-family:Arial;">>> > as well.<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">Are all fixes from master backported to 3.0?<br></div></blockquote><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">Unfortunately it's hard to tell, because many of the fixes on master are fixes to bugs that were only introduced on master, and some bugs on 3.0 we just say "the fix is so invasive that it's basically just backporting 90% of master, which is pointless for a stable release".<br></div><div style="font-family:Arial;"><br></div><blockquote type="cite" id="fastmail-quoted"><div style="font-family:Arial;">Is the new Commit "I will try your new commits regarding CID" related to the<br></div><div style="font-family:Arial;">"IOERROR: conversations_audit on load:" and "IOERROR: <br></div><div style="font-family:Arial;">conversations_audit on store"?<br></div></blockquote><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">Shouldn't be. It just means we store the G keys regardless of whether the record has a CID.<br></div><div style="font-family:Arial;"><br></div><blockquote type="cite" id="fastmail-quoted"><div style="font-family:Arial;">I will try your new commits in the next days on my test servers to sea <br></div><div style="font-family:Arial;">if the fix<br></div><div style="font-family:Arial;">the endless loop in ctl_conversationsdb I have seen for some accounts.<br></div></blockquote><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">I guess one more question - are you running the most recent index version? (reconstruct -V max)<br></div><div style="font-family:Arial;"><br></div><blockquote type="cite" id="fastmail-quoted"><div style="font-family:Arial;">Quoting myself (Re: prblems rebuilding conversations db) Jan 24, 2019<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">> The program loops in build_cid_cb (imap/ctl_conversationsdb.c:189)<br></div><div style="font-family:Arial;">><br></div><div style="font-family:Arial;">> For the problematic mailbox that I tested, for every message<br></div><div style="font-family:Arial;">> record->cid was NULLCONVERSATION, so mailbox_cacherecord,<br></div><div style="font-family:Arial;">> message_update_conversations and mailbox_rewrite_index_record<br></div><div style="font-family:Arial;">> where called, each returned 0.<br></div><div style="font-family:Arial;">><br></div><div style="font-family:Arial;">> After iterating trough all messages in the mailbox count was > 0, and r==0,<br></div><div style="font-family:Arial;">> so the while condition (!r && count) was true for the next run.<br></div><div style="font-family:Arial;">> At this point record->cid was still NULLCONVERSATION for every message,<br></div><div style="font-family:Arial;">> which I guess should not be the case.<br></div></blockquote><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">ctl_conversationsdb -b should update the cid. BUT - if you're running old mailboxes which have a format which doesn't support saving the CID, that would for sure break things!<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">Bron.<br></div><div style="font-family:Arial;"><br></div><div id="sig56629417"><div class="signature">--<br></div><div class="signature"> Bron Gondwana, CEO, FastMail Pty Ltd<br></div><div class="signature"> brong@fastmailteam.com<br></div><div class="signature"><br></div></div><div style="font-family:Arial;"><br></div></body></html>