Contents mysql problem
Rudolf Gabler
rug at usm.lmu.de
Thu Sep 12 08:19:54 EDT 2013
Hi,
I examined my mysql problem a little more and got the following:
the following log (2) shows, I preforked 7 imapd -s childs (by /etc/cyrus.conf) and while one of the processes still reads from the local mysql server (see process 32711), newly forked imapd childs are loosing this ability (see process 9881). From this time on nearly all newly created childs can't connect to the mysql server (even then when the "good ones" = 32711 are terminated).
In a very rapid process ( because childs with connection errors are terminated see log 1) new childs are forked and forked (in my case 1 fork/second the next 2 minutes - as well imaps as https processes) until without restarting the master process some childs are forked, which are again able to connect to the mysql server. But the forking process goes on and tries to reach the number of preforked childs. This fills the logs because only any then and now a "good" process is created.
But only if I restart the master any child is working. The default max connection limit is set to the default (150) for the mysql server. This is still a testing environment with only one user connected by one open mailer.
Many regards,
Rudi Gabler
---
log 1 shows the typically error when a child is unsuccessfully forked. Interestingly it tries to connect to the mysql server for the information of quotas and then to the file /var/lib/imap/quotas.db. But there is no specification in which format this file should be (because it should be in a sql database).
log 1:
Sep 12 12:45:16 xmailer imaps[31321]: SQL backend trying to connect to a host
Sep 12 12:45:16 xmailer imaps[31321]: SQL backend trying to open db 'cyrusdb' on host 'localhost'
Sep 12 12:45:16 xmailer imaps[31321]: executing SQL cmd: SELECT * FROM mailboxes_db LIMIT 0;
Sep 12 12:45:16 xmailer imaps[31321]: SQL backend trying to connect to a host
Sep 12 12:45:16 xmailer imaps[31321]: SQL backend trying to open db 'cyrusdb' on host 'localhost'
Sep 12 12:45:16 xmailer imaps[31321]: DBERROR: SQL backend could not connect to host localhost
Sep 12 12:45:16 xmailer imaps[31321]: DBERROR: could not open SQL database 'cyrusdb'
Sep 12 12:45:16 xmailer imaps[31321]: DBERROR: opening /var/lib/imap/quotas.db: cyrusdb error
Sep 12 12:45:16 xmailer imaps[31321]: Fatal error: can't read quotas file
Sep 12 12:45:16 xmailer master[32309]: process 31321 exited, status 75
Sep 12 12:45:16 xmailer master[32309]: service imaps pid 31321 in READY state: terminated abnormally
log 2:
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.emulex';
Sep 12 12:23:56 xmailer imaps[32711]: fetching user_deny.db entry for 'rug'
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.FFM';
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.FFM';
Sep 12 12:23:56 xmailer imaps[32711]: fetching user_deny.db entry for 'rug'
Sep 12 12:23:56 xmailer imaps[4803]: fetching user_deny.db entry for 'rug'
Sep 12 12:23:56 xmailer imaps[9881]: executed
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.Fireball';
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.Fireball';
Sep 12 12:23:56 xmailer imaps[9881]: SQL backend trying to connect to a host
Sep 12 12:23:56 xmailer imaps[9881]: SQL backend trying to open db 'cyrusdb' on host 'localhost'
Sep 12 12:23:56 xmailer imaps[4803]: fetching user_deny.db entry for 'rug'
Sep 12 12:23:56 xmailer imaps[32711]: fetching user_deny.db entry for 'rug'
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.Fu&AN8-ball';
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.Fu&AN8-ball';
Sep 12 12:23:56 xmailer imaps[9881]: executing SQL cmd: SELECT * FROM mailboxes_db LIMIT 0;
Sep 12 12:23:56 xmailer imaps[32711]: fetching user_deny.db entry for 'rug'
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.Hawaii';
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.Hawaii';
Sep 12 12:23:56 xmailer imaps[32711]: fetching user_deny.db entry for 'rug'
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.iostat';
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.iostat';
Sep 12 12:23:56 xmailer imaps[9881]: SQL backend trying to connect to a host
Sep 12 12:23:56 xmailer imaps[9881]: SQL backend trying to open db 'cyrusdb' on host 'localhost'
Sep 12 12:23:56 xmailer imaps[32711]: fetching user_deny.db entry for 'rug'
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.isdndec';
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.isdndec';
Sep 12 12:23:56 xmailer imaps[9881]: DBERROR: SQL backend could not connect to host localhost
Sep 12 12:23:56 xmailer imaps[9881]: DBERROR: could not open SQL database 'cyrusdb'
Sep 12 12:23:56 xmailer imaps[9881]: DBERROR: opening /var/lib/imap/quotas.db: cyrusdb error
Sep 12 12:23:56 xmailer imaps[9881]: Fatal error: can't read quotas file
Sep 12 12:23:56 xmailer imaps[32711]: fetching user_deny.db entry for 'rug'
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.JENAM';
Sep 12 12:23:56 xmailer imaps[32711]: executing SQL cmd: SELECT * FROM mailboxes_db WHERE dbkey = 'user.rug.Solved.JENAM';
Sep 12 12:23:56 xmailer imaps[32711]: fetching user_deny.db entry for 'rug'
..
Sep 12 12:23:56 xmailer master[32309]: process 9881 exited, status 75
Sep 12 12:23:56 xmailer master[32309]: service imaps pid 9881 in READY state: terminated abnormally
----------------------------------------------------------------------
Message: 1
Date: Wed, 11 Sep 2013 09:46:17 +0200
From: Rudolf Gabler <rug at usm.lmu.de>
Subject: mysql problem
To: info-cyrus at lists.andrew.cmu.edu
Message-ID: <ADBE1AB6-540B-4FA2-ADBE-98FDA0973586 at usm.lmu.de>
Content-Type: text/plain; charset="windows-1252"
Hi,
the cyrus-imapd-2.4.17-caldav-beta6 release shows the following:
After a fresh start with a mysql database for several purposes
duplicate_db: sql
mboxlist_db: sql
quota_db: sql
tlscache_db: sql
the system is running as expected. After approximately the 10 imaps contact suddenly the following occurs:
Sep 11 09:43:29 xmailer imaps[8168]: DBERROR: SQL query failed: MySQL server has gone away
Sep 11 09:43:29 xmailer imaps[8168]: DBERROR: SQL failed SELECT * FROM mailboxes_db WHERE dbkey = 'user?..
But the local mysql server is o.k. and running. After a restart of cyrus (/etc/init.d/cyrus-imapd restart) everything is working until the next appr. 10 connection times by the user.
If I use sqlite instead everything is stable for at least a month.
Any hint what may occur?
Regards,
Rudi Gabler
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20130911/7b0ff055/attachment-0001.bin
------------------------------
Message: 2
Date: Wed, 11 Sep 2013 19:43:33 +1000
From: Bron Gondwana <brong at fastmail.fm>
Subject: Re: mysql problem
To: info-cyrus at lists.andrew.cmu.edu
Cc: Ken Murchison <murch at andrew.cmu.edu>
Message-ID:
<1378892613.11046.20578185.3FFF81BF at webmail.messagingengine.com>
Content-Type: text/plain; charset="utf-8"
On Wed, Sep 11, 2013, at 05:46 PM, Rudolf Gabler wrote:
> Hi,
>
> the cyrus-imapd-2.4.17-caldav-beta6 release shows the following:
>
> After a fresh start with a mysql database for several purposes
>
> duplicate_db: sql
> mboxlist_db: sql
> quota_db: sql
> tlscache_db: sql
>
> the system is running as expected. After approximately the 10 imaps contact suddenly the following occurs:
>
> Sep 11 09:43:29 xmailer imaps[8168]: DBERROR: SQL query failed: MySQL server has gone away
> Sep 11 09:43:29 xmailer imaps[8168]: DBERROR: SQL failed SELECT * FROM mailboxes_db WHERE dbkey = 'user?..
>
> But the local mysql server is o.k. and running. After a restart of cyrus (/etc/init.d/cyrus-imapd restart) everything is working until the next appr. 10 connection times by the user.
Smells like connection handle leakage, or leakage of some other finite resource in the mysql library.
> Any hint what may occur?
Ken - any ideas? You know this code better than I do.
Bron.
--
Bron Gondwana
brong at fastmail.fm
------------------------------
Message: 3
Date: Wed, 11 Sep 2013 07:00:09 -0400
From: Ken Murchison <murch at andrew.cmu.edu>
Subject: Re: mysql problem
To: Bron Gondwana <brong at fastmail.fm>
Cc: info-cyrus at lists.andrew.cmu.edu
Message-ID: <52304D39.2030804 at andrew.cmu.edu>
Content-Type: text/plain; charset=UTF-8; format=flowed
On 9/11/13 5:43 AM, Bron Gondwana wrote:
> On Wed, Sep 11, 2013, at 05:46 PM, Rudolf Gabler wrote:
>> Hi,
>>
>> the cyrus-imapd-2.4.17-caldav-beta6 release shows the following:
>>
>> After a fresh start with a mysql database for several purposes
>>
>> duplicate_db: sql
>> mboxlist_db: sql
>> quota_db: sql
>> tlscache_db: sql
>>
>> the system is running as expected. After approximately the 10 imaps contact suddenly the following occurs:
>>
>> Sep 11 09:43:29 xmailer imaps[8168]: DBERROR: SQL query failed: MySQL server has gone away
>> Sep 11 09:43:29 xmailer imaps[8168]: DBERROR: SQL failed SELECT * FROM mailboxes_db WHERE dbkey = 'user?..
>>
>> But the local mysql server is o.k. and running. After a restart of cyrus (/etc/init.d/cyrus-imapd restart) everything is working until the next appr. 10 connection times by the user.
> Smells like connection handle leakage, or leakage of some other finite resource in the mysql library.
>
>> Any hint what may occur?
> Ken - any ideas? You know this code better than I do.
>
> Bron.
>
No, I don't have any ideas. I don't have any experience using MySQL for
all Cyrus databases (or any Cyrus db for that matter), and the *DAV code
uses SQLite, not mySQL, and does so directly without going through the
cyrusdb API.
--
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University
------------------------------
Message: 4
Date: Wed, 11 Sep 2013 08:53:09 -0500
From: Blake Hudson <blake at ispn.net>
Subject: Re: mysql problem
To: info-cyrus at lists.andrew.cmu.edu
Message-ID: <523075C5.80705 at ispn.net>
Content-Type: text/plain; charset=UTF-8; format=flowed
Rudolf Gabler wrote the following on 9/11/2013 2:46 AM:
> the system is running as expected. After approximately the 10 imaps contact suddenly the following occurs:
>
> Sep 11 09:43:29 xmailer imaps[8168]: DBERROR: SQL query failed: MySQL server has gone away
> Sep 11 09:43:29 xmailer imaps[8168]: DBERROR: SQL failed SELECT * FROM mailboxes_db WHERE dbkey = 'user?..
>
> But the local mysql server is o.k. and running. After a restart of cyrus (/etc/init.d/cyrus-imapd restart) everything is working until the next appr. 10 connection times by the user.
>
> If I use sqlite instead everything is stable for at least a month.
>
> Any hint what may occur?
>
I'm not familiar with using MySQL for anything more than authentication
so I'm not sure how Cyrus makes it's db connections, but is there a
limit on the number of allowed client connections in my.cnf - or per
user on the MySQL account you're using for Cyrus?
When Cyrus is broke, can you login to MySQL, use the Cyrus databases,
and run the queries that Cyrus would?
--Blake
------------------------------
_______________________________________________
Info-cyrus mailing list
Info-cyrus at lists.andrew.cmu.edu
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
End of Info-cyrus Digest, Vol 98, Issue 4
*****************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20130912/b0ed4849/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20130912/b0ed4849/attachment-0001.bin
More information about the Info-cyrus
mailing list