use valgrind / Re: SIGSEGV in cyrus-imapd 3.0.7 mupdate

[Дилян Палаузов]

Hello Michael,

this is likely either a memory mishandling issue (use after free(),
double free(), invalid read()/write()...), which gets evident if cyrus
is run under valgrind --tool=memcheck.  I run it with

valgrind --num-callers=30 --leak-check=full --track-origins=yes --read-
var-info=yes --show-leak-kinds=all --trace-children=yes --track-fds=yes 
/usr/local/cyrus/bin/master -D &> log-file-memcheck

It runs then very slow, but shows both cause of troubles and memory
leaks.

Another reason can be multi-threaded inconsistencies: mutexes locked in
inconsistent order by differnt threads (while this is not a cause for
crash, it leads to deadlock), mutexes locked in one thread and unlocked
in another or alike.  This can be detected by valgrind/helgrind

valgrind --tool=helgrind --num-callers=30 --leak-check=full --track-
origins=yes --read-var-info=yes --trace-children=yes --track-fds=yes
/usr/local/cyrus/bin/master -D &> log-file-helgrind

Of course, it is useful to have debugging symbols.  Compiling with -O3
migh make things faster, while compiling with -O0 will make it run
considerably slower. If ld (=ld.bfd) is very, very recent (e.g. 2.31),
then link with  -Wl,-z,noseparate-code otherwise valgrind cannot read
the debug info (https://bugs.kde.org/show_bug.cgi?id=395682).

Then it shall be possible to see in one or the other log files what is
relevant for causing SIGSERV, in particular the first place where bad
operation happens, which might be some time before the crash.

Regards
  Дилян

On Mon, 2018-07-02 at 17:24 +0200, Michael Menge wrote:
> Hi,
> 
> we are in the process of setting up our new production mailserver with
> cyrus-imapd 3.0.7 on RHEL 7.5 Servers.
> 
> At the moment we encounter many crashes (SIGSEGV) of the mupdate process on
> the mupdate master instance. As soon as we issue a command that updates
> multiple mailboxes in short time we trigger a SIGSEGV.
> 
> > sam user/zrstes* cyrus all
> 
> Setting ACL on user/zrstes1...OK.
> Setting ACL on user/zrstes1/Mail...OK.
> Setting ACL on user/zrstes1/Mail/drafts...OK.
> Setting ACL on user/zrstes1/Mail/s-spam...OK.
> Setting ACL on user/zrstes1/Mail/sent...OK.
> Setting ACL on user/zrstes1/Mail/trash...OK.
> Setting ACL on user/zrstes1/Mail/v-spam...OK.
> Setting ACL on user/zrstes2...cyrus: lrswipkxtea: no connection to server
> 
> I suspect that at this time the connection from the backend to the  
> mupdate master
> is lost as the mupdate process received the SIGSEGV
> 
> (gdb) bt
> #0  0x00007fb49c000098 in ?? ()
> #1  0x00007fb4a9a59a85 in DH_free (r=0x7fb49c1b9600) at dh_lib.c:194
> #2  0x00007fb4aa84dbef in tls_shutdown_serverengine () at imap/tls.c:1311
> #3  0x0000000000404075 in conn_free (C=0x7fb4840009f0) at imap/mupdate.c:379
> #4  0x00000000004062c0 in thread_main (rock=0x0) at imap/mupdate.c:1330
> #5  0x00007fb4a771add5 in start_thread (arg=0x7fb4a22d9700) at  
> pthread_create.c:308
> #6  0x00007fb4a718fb3d in clone () at  
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
> 
> 
> Further information, as DH_free is in the bt, we have NOT configured  
> dh_params in
> our ssl certificate/key or in the imapd.conf
> 
> Kind regards
> 
>     Michael Menge
> 
> --------------------------------------------------------------------------------
> M.Menge                                Tel.: (49) 7071/29-70316
> Universität Tübingen                   Fax.: (49) 7071/29-5912
> Zentrum für Datenverarbeitung          mail:  
> michael.menge at zdv.uni-tuebingen.de
> Wächterstraße 76
> 72074 Tübingen
>