UID THREAD REFS US-ASCII ALL slow / stalls forever on one folder.

[ellie timoney]

Hi,

I think I would do something like:

0) set client_timeout to a big value (see below)
1) let the imapd start normally
2) connect to it with a minimal imap client (like imtest or telnet)
3) check logs to see which imapd process id your client is connected to (if there's more than one)
4) use the "gdb /path/to/binary pid" invocation to attach gdb to the running imap process
5) repeat #4 in a few different ways if it's unable to find symbols
6) set a breakpoint on the cmd_thread function (since that's what handles the THREAD command) and then continue
7) back in your imap client, send the UID THREAD REFS US-ASCII ALL" command and step through to see what happens

Note that these are not exhaustive steps, more of a "get started, and then react accordingly, depending on what you see"

I would also try variations of that THREAD command -- if you add/remove options, does it start working?  Can you isolate the problem to a specific combination of options?  Does it only happen for some mailboxes?

You probably want to set client_timeout to a big value.  The default is 10 seconds, so the server will probably throw your client off while you're reading output from gdb, and then you'll wind up debugging the "disconnect an unresponsive client code" instead.  I usually set this to be 30 minutes or so for debugging.  For 3.2 and newer, you can spell this as "30m".  For older versions, the value is in seconds, so you want "1800".

Permissions may be awkward.  You will need to be the cyrus user (or root) to connect gdb to the running imapd, but you will also need permission to read the source that it was built from, which is probably not owned by the cyrus user.  In my case it's under my user account's home directory, so I add the cyrus user to my user account's group, and make sure the path to the source directory is g+rx (because I don't like running gdb as root).

Cheers,

ellie

On Fri, Jul 3, 2020, at 5:23 AM, Jesper Schmitz Mouridsen via Info-cyrus wrote:
> Hi.
> 
> I recently upgraded Cyrus to 3.2.2 from 3.0.13.
> 
> Now threading "UID THREAD REFS US-ASCII ALL" on one specific folder with 
>  >50K mails,
> 
> makes imapd process use 100 CPU% without any progress. truss[1] or dtrace
> 
> shows no output. The process seems totally stuck.
> 
> I installed in a FreeBSD 12.1 jail. Any hints beside the online docs of 
> how to use gdb to
> 
> see what  is going one. I could not make imapd run under gdb even with 
> the -D option
> 
> and debug_command setting or reading 
> https://www.cyrusimap.org/imap/developer/developer-testing.html?highlight=debug
> 
> It is fast enough on other folders also with ~50k mails < 3 secs.
> 
> Any hints, highly appreciated :-)
> 
> [1] https://www.freebsd.org/cgi/man.cgi?truss
> 
> Regards
> 
> Jesper
> 
> ----
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus