Cyrus IMAPd 2.3.7 Released

[André Böhm]

Ken Murchison schrieb:
> Andre Böhm wrote:
>> Ken Murchison schrieb:
>>> André Böhm wrote:
>>>> Hello,
>>>>
>>>> after upgrading from 2.2.13 to 2.3.7, I have a very worrying problem
>>>> with lmtp.
>>>>
>>>> When procmail tries to deliver mail via the "deliver" command, with
>>>> the "-a" option I get "master[96435]: process 99783 exited, signaled
>>>> to death by 11".
>>>>
>>>> This is only on murder master and murder frontends. On the backend
>>>> seems not to be any problem.
>>>>
>>>> Without the "-a" argument, deliver does work, but I cannot use any
>>>> extensions anymore because ACL for anyone would be needed.
>>>>
>>>> What can I do to fix this? Or is there some changes in LMTP I were not
>>>> aware of?
>>> Just to clarify, you only have this problem on frontend (proxy) machines
>>> when using the -a option?
>>>
>>>
>>
>> Sorry, my information was incomplete and not quite correct. Now I think
>> I mixed up two different things, but they may be connected.
>>
>> On the frontend/proxy, the main problem is that delivery to the postuser
>> fails, with lmtpunix reporting: "verify_user() failed: Mailbox does not
>> exist"
> 
> By 'postuser' do you mean the value of the 'postuser' option in 
> imapd.conf?  Are you trying to deliver to a shared mailbox?
> 
> 
>> The problem with "signalled to death" happened 4 times as in syslog, but
>> I cannot repeat it.
>>
>> On the murder master, the same problem exists with the postuser.
>> Additionally, attempts to use the -a parameter with deliver always
>> result in this:
>> lmtpunix[28915]: accepted connection
>> lmtpunix[28915]: lmtp connection preauth'd as postman
>> mupdate[5989]: cmd_find(fd:13, user.andre)
>> master[5981]: process 28915 exited, signaled to death by 11
>> master[5981]: service lmtpunix pid 28915 in BUSY state: terminated
>> abnormally
> 
> Is the murder master also a frontend machine (I assume it has to be if 
> lmtpproxyd is listening)?

Yes.

> 
> I can't reproduce this problem the problem here, but maybe I don't quite 
> understand your config.
> 
> 

O.k., I just try to list all of it, you have to pick the interesting 
parts...

I have several servers, all running FreeBSD. All mailservers live in a 
FreeBSD jail and consist of cyrus-imapd and postfix plus some procmail 
"magic".

Two of these servers are cyrus imapd murder frontends (proxies) that 
also have a postfix configured as mx server for several domains.
Out of these two servers one is actually the murder master.

When email comes in, postfix does not deliver directly to lmtp, instead 
every email is given to procmail, and after some clamav, spamd, and 
homebrewn conversion scripts (8bit header and so on).
Now the important part is when the email finally gets through all this 
procmail processing, it will be piped to cyrus/bin/deliver.

This is now (2.3.7) working nearly as it has been working before 
(2.2.x), with practically same configuration as before, but with only 
two exceptions.

The first exception when email is destined for the "postuser", defined 
in imapd.conf. In my case the postuser is called "shared".
On all imapd.confs in the murder environment the "postuser" 
configuration option is the same, and it worked for really a long time 
with 2.2.

This is the "critical" part at the end of my procmail recipe:

DELIVERMAIL=/usr/local/cyrus/bin/deliver

:0 w
* EXTENSION ?? [^ ]
| $DELIVERMAIL -a "$ABMAILDELIVERTO" -r "$SENDER" -m "$EXTENSION" 
"$ABMAILDELIVERTO"

:0 w
| $DELIVERMAIL -a "$ABMAILDELIVERTO" -r "$SENDER" "$ABMAILDELIVERTO"

:0 w
| $DELIVERMAIL -a "andre" -r "$SENDER" -m "unzustellbar" -q "andre"


The variable ABMAILDELIVERTO is set to the user by the script before, 
i.e. "andre" (for myself) or "shared" for the postuser.

Note, that normally procmail would stop processing after a successful 
delivery, so that the last line will only be reached if all delivery 
attempts before were unsuccessful.
The extension "unzustellbar" is german for "undeliverable" and is one of 
my own mail folder, that I watch for problems (like remote backends 
being unreachable).

Now, on the frontend/proxy, when emails arrive that would normally (like 
for the last 2 years) be delivered to a shared folder called 
"Postmaster/Bad-address", the procmail recipe does the following:

/usr/local/cyrus/bin/deliver -a "shared" -r "postmaster at abmail.de" -m 
"Postmaster/Bad-address" "shared"

with the result:
lmtpunix[30300]: verify_user(Postmaster.Bad-address) failed: Mailbox 
does not exist

After that, because of failure, procmail continues in the script to the 
next lines to deliver without the extension:

/usr/local/cyrus/bin/deliver -a "shared" -r "postmaster at abmail.de" "shared"

with the result:
lmtpunix[30356]: verify_user() failed: Mailbox does not exist

Which, o.k., is correct. My script should not try to put something into 
shareds inbox, so I have to fix it myself. This line should never be 
executed for the "shared" (postuser) user. It is obsolete as I learned 
lmtp tries by itself to put the mail into the inbox when the extension 
is not available.

However, the last resort is:
/usr/local/cyrus/bin/deliver -a "andre" -r "postmaster at abmail.de" -m 
"unzustellbar" "andre"

So finally all the emails for the shared user end up in my 
"unzustellbar" mail folder.



O.k., the same problem I have on the other frontend, which is at the 
same time the murder master.
But on this server there is also the additional problem. I now have 
removed the -a parameter from my delivery recipe in procmail to have it 
working again (but I cannot use any subfolders/extensions from procmail 
recipes anymore). Whenever I try to deliver with -a parameter, I get:
lmtpunix[66205]: accepted connection
lmtpunix[66205]: lmtp connection preauth'd as postman
mupdate[57984]: cmd_find(fd:15, Postmaster.Bad-address)
Jul 13 15:22:30 mtb master[57976]: process 66205 exited, signaled to 
death by 11
Jul 13 15:22:30 mtb master[57976]: service lmtpunix pid 66205 in BUSY 
state: terminated abnormally

This only happens on murder master with deliver -a.
This is the cyrus.conf from murder master:

START {
  recover        cmd="ctl_cyrusdb -r"
  idled          cmd="idled"
}

# UNIX sockets start with a slash and are put into /var/imap/socket
SERVICES {
   imap          cmd="imapd" listen="imap" prefork=1
   imapunix      cmd="imapd" listen="/var/imap/socket/imap" prefork=1
   imaps         cmd="imapd -s" listen="imaps" prefork=1
   pop3          cmd="pop3d" listen="pop3" prefork=0
   pop3s         cmd="pop3d -s" listen="pop3s" prefork=0
#  nntp          cmd="nntpd" listen="nntp" prefork=0
#  nntps         cmd="nntpd -s" listen="nntps" prefork=0
   sieve         cmd="timsieved" listen="sieve" prefork=0
#  lmtp          cmd="lmtpd" listen="lmtp" prefork=1
   lmtpunix      cmd="lmtpd" listen="/var/imap/socket/lmtp" prefork=0
   mupdate       cmd="/usr/local/cyrus/bin/mupdate -m" listen=3905 prefork=1
}

EVENTS {
   checkpoint    cmd="ctl_cyrusdb -c" period=30
   delprune      cmd="cyr_expire -E 3" at=0400
   tlsprune      cmd="tls_prune" at=0400
}




Some lines from my imapd.conf which are the same on all servers:

unixhierarchysep: yes
altnamespace: yes
userprefix: Andere Benutzer
sharedprefix: Gemeinsame Ordner
postuser: shared
username_tolower: 1
lmtp_downcase_rcpt: 0
lmtp_fuzzy_mailbox_match: 1
virtdomains: userid
defaultdomain: abmail.de

(I cycled the new lmtp_downcase_rcpt and lmtp_fuzzy_mailbox_match 
options, but that did not change anything for the "postuser" problem. 
However, addressing these particular folders worked before for years, so 
it should not be the broken part.)

The mupdate server is configured in imapd.conf on all servers, also on 
the mupdate master itself where it connects to itself.


I hope that shows the general picture of my setup. Please tell me if I 
missed some crucial parts.
Last night was a little stressful, so forgive me that I didn't post all 
relevant information at once.


Would you recommend separating the murder master and not use it as a 
frontend?



André Böhm