cyrus-imapd with lmtpd + postfix slow delivery with group email ids
Stephen Ulmer
ulmer at ufl.edu
Mon May 18 15:26:58 EDT 2015
I mostly waned top make sure that your delivery concurrency wasn’t limited by the number of processes that could access a named pipe. That doesn’t seem to be your problem, though.
--
Stephen
> On May 18, 2015, at 4:21 AM, jayesh shinde <jayesh.shinde at netcore.co.in> wrote:
>
> Hi Stephen ,
>
> I am handling via TCP i.e from smtp server to mailbox server.
>
> on smtp server postfix's main.cf transport_maps is maintain , which deliver the emails to lmtp.
>
> Example like below.
> /@mydomain.com/ lmtp:[192.168.1.1]
>
> Request you to please guide here or any fine tuning etc ....
>
> Regards
> Jayesh Shinde
>
>
> On 05/15/2015 10:11 PM, Stephen Ulmer wrote:
>> Are you handing off to LMTP via TCP or a pipe?
>>
>> --
>> Stephen
>>
>>
>>
>>> On May 15, 2015, at 2:18 AM, jayesh shinde <jayesh.shinde at netcore.co.in <mailto:jayesh.shinde at netcore.co.in>> wrote:
>>>
>>> Thanks Dan for your feedback.
>>> I verified the suggested points , its working as require. Please find below inline reply for each.
>>>
>>> I found that when big group email is getting deliver ( i.e more than 500 - 4K members ) at that time other emails are not getting deliver simultaneously to user's mailbox.
>>> I verified this from live maillog i.e by doing grep key words like "to mailbox:" / "Delivered:" etc ...
>>>
>>> My mail queue issue is getting critical in peak hours !! Kindly suggest.
>>>
>>> On 05/14/2015 08:22 PM, Dan White wrote:
>>>> On 05/14/15 05:42 +0000, jayesh.shinde wrote:
>>>>> Please suggest , for faster delivery what could be the best config in
>>>>> postfix + cyrus-imapd .
>>>>>
>>>>> I am using the cyrus-imapd-2.4.17-6.el6.x86_64 for mailbox server and for
>>>>> smtp server postfix-2.10.0-1.el6.x86_64 Both server have SAS hdd , 16 core
>>>>> , 16 GB RAM . There is no i/o issues
>>>>
>>>>> Mailbox server have 21K email ids. Cyrus running with tcp socket on
>>>>> port 24 i.e lmtpd . cyrus singleinstance is maintained
>>>>
>>>>> For group id mapping is under virtual_alias_maps of smtp's main.cf i.e no
>>>>> /etc/aliases. From smtp server emails are getting deliver by
>>>>> transport_maps to mailbox server .
>>>>
>>>>> My problem observe :--
>>>>> --------------------------
>>>>> 1) When HOD send email to 3-5 big group email ids then , then its postfix
>>>>> delivery get slow. What I observe is postfix getting delivered the traffic
>>>>> per email and not parallel. Each group contain 3k or 7k email ids.
>>>>
>>>> Verify that single instance store is working. If not, you may have a
>>>> Postfix configuration issue. Find one of these mass emails in a mailbox and
>>>> stat it to find out:
>>>>
>>>> stat -c %h <file>
>>>>
>>>> A returned value of 1 means single instance store isn't working.
>>>>
>>>> If that's not the issue, you can determine if this is a postfix issue or a
>>>> cyrus issue by sending a mass email directly to lmtpd, with lmtptest.
>>> Cyrus Singleinstance is working , I have verified like below. In imapd.conf man page its define as default 1 ( i.e default enable )
>>>
>>> [root at Mailbox ~]# ls -lirth /var/spool/imap/domain/v/vdomain/m/user/muser1/ | tail -n 20
>>>
>>> 163326970 -rw------- 198 cyrus mail 413K May 15 10:07 64828.
>>> 163396232 -rw------- 145 cyrus mail 92K May 15 10:13 64829.
>>> 163379367 -rw------- 151 cyrus mail 96K May 15 10:29 64830.
>>> 163323993 -rw------- 496 cyrus mail 13K May 15 10:30 64831.
>>> 163398621 -rw------- 516 cyrus mail 850K May 15 10:42 64832.
>>> 163398568 -rw------- 523 cyrus mail 455K May 15 10:43 64833.
>>> 163398933 -rw------- 617 cyrus mail 3.7M May 15 10:47 64834.
>>> 163398962 -rw------- 522 cyrus mail 1.3M May 15 10:48 64835.
>>> 163398990 -rw------- 617 cyrus mail 58K May 15 10:49 64836.
>>> 163396678 -rw------- 465 cyrus mail 207K May 15 10:56 64837.
>>> 163399529 -rw------- 576 cyrus mail 332K May 15 10:57 64838.
>>> 163399554 -rw------- 2 cyrus mail 54K May 15 10:57 64839.
>>> 163346523 -rw------- 36 cyrus mail 767K May 15 11:01 64840.
>>>
>>> [root at Mailbox muser1]# stat -c %h 64829.
>>> 145
>>> [root at Mailbox muser1]# stat -c %h 64828.
>>> 198
>>>
>>>>
>>>>> In peak hours the queue on postfix get high and other normal emails also
>>>>> getting stuck in queue. Once the group email get clear after that other
>>>>> emails also get clear.
>>>>
>>>>> in mailbox cyrus.conf :-- lmtp cmd="lmtpd -a" listen="lmtp" prefork=0
>>>>> in main.cf of smtp server :--
>>>>> lmtp_destination_concurrency_limit = 100
>>>>> lmtp_destination_recipient_limit = 0
>>>>
>>>> How many lmtp processes do you see spawned in this scenario?
>>>>
>>> [root at Mailbox ~]# ps auxwww| grep lmtp
>>>
>>> cyrus 3224 0.0 0.0 73376 4648 ? S 10:55 0:00 lmtpd
>>> cyrus 4043 0.0 0.0 73376 4156 ? S 10:59 0:00 lmtpd
>>> postfix 6972 0.0 0.0 75708 4336 ? S 11:10 0:00 lmtp -t unix -u
>>> postfix 6974 0.0 0.0 75708 4340 ? S 11:10 0:00 lmtp -t unix -u
>>> postfix 6975 0.0 0.0 75708 4336 ? S 11:10 0:00 lmtp -t unix -u
>>> postfix 6977 0.0 0.0 75708 4344 ? S 11:10 0:00 lmtp -t unix -u
>>> postfix 6979 0.0 0.0 75708 4336 ? S 11:10 0:00 lmtp -t unix -u
>>> postfix 6981 0.0 0.0 75708 4336 ? S 11:10 0:00 lmtp -t unix -u
>>> postfix 6985 0.0 0.0 75708 4348 ? S 11:10 0:00 lmtp -t unix -u
>>> cyrus 6998 0.0 0.0 73376 4052 ? S 11:10 0:00 lmtpd
>>> cyrus 7001 0.0 0.0 73376 4044 ? S 11:10 0:00 lmtpd
>>> cyrus 7234 0.9 0.0 75456 4976 ? S 11:12 0:04 lmtpd -a
>>> cyrus 8758 0.6 0.0 75452 4112 ? S 11:19 0:00 lmtpd -a
>>> root 8898 0.0 0.0 103236 972 pts/1 S+ 11:19 0:00 grep --colour=auto lmtp
>>> cyrus 46046 0.0 0.0 73376 4648 ? S 10:28 0:01 lmtpd
>>> cyrus 48107 0.0 0.0 73372 4740 ? S 10:37 0:01 lmtpd
>>> cyrus 48108 0.0 0.0 73376 4700 ? S 10:37 0:01 lmtpd
>>>
>>> lmtpd :-- 7 process
>>> lmtp -t unix -u :-- 7 process
>>> lmtpd -a :-- 2 process
>>>
>>> Cyrus config is as follows:
>>> -----------------------------------
>>>
>>> SERVICES {
>>> imap cmd="imapd" listen="imap" prefork=5
>>> imaps cmd="imapd -s" listen="imaps" prefork=1
>>> pop3 cmd="pop3d" listen="pop3" prefork=3
>>> pop3s cmd="pop3d -s" listen="pop3s" prefork=1
>>> sieve cmd="timsieved" listen="sieve" prefork=0
>>>
>>> lmtp cmd="lmtpd -a" listen="lmtp" prefork=0
>>> lmtpunix cmd="lmtpd" listen="/var/lib/imap/socket/lmtp" prefork=1
>>> }
>>>
>>> [root at Mailbox ~]# cat /etc/imapd.conf
>>> configdirectory: /var/lib/imap
>>> partition-default: /var/spool/imap
>>> admins: cyrus
>>> sievedir: /var/lib/imap/sieve
>>> sendmail: /usr/sbin/sendmail
>>> hashimapspool: true
>>> sasl_pwcheck_method: saslauthd
>>> sasl_mech_list: PLAIN
>>> allowplaintext: yes
>>> unixhierarchysep: 1
>>> virtdomains: yes
>>> altnamespace: on
>>> defaultdomain: mydomain.com <http://mydomain.com/>
>>> expunge_mode: immediate
>>> delete_mode: immediate
>>> tls_cert_file: /usr/share/ssl/certs/server.pem
>>> tls_key_file: /usr/share/ssl/certs/server.key
>>> tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
>>> lmtp_over_quota_perm_failure: 1
>>> duplicatesuppression: 1
>>> duplicate_db_path: /var/lib/imap/cyrus-ram-tmpfs/deliver.db
>>> statuscache_db_path: /var/lib/imap/cyrus-ram-tmpfs/statuscache.db
>>> tlscache_db_path: /var/lib/imap/cyrus-ram-tmpfs/tls_sessions.db
>>>
>>> Regards
>>> Jayesh Shinde
>>> ----
>>> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20150518/c96560c5/attachment-0001.html
More information about the Info-cyrus
mailing list