cyrus-imapd 2.1.15, sieve, lmtpd, and return-path header

[Rob Siemborski]

On Tue, 23 Sep 2003, Chris Stromsoe wrote:

> LMTP is not a dialect of SMTP (unless I'm reading 2033 wrong -- feel free
> to correct me if I am):
>
>    Although LMTP is an alternative protocol to ESMTP, it uses (with a
>    few changes) the syntax and semantics of ESMTP.  This design permits
>    LMTP to utilize the extensions defined for ESMTP.  LMTP should be
>    used only by specific prior arrangement and configuration, and it
>    MUST NOT be used on TCP port 25.

I suppose whether LMTP is a "dialect of SMTP" or not is open to
interpretation.  It is, however, certainly part of the delivery process.
And, for the record, Sieve processing happens before the 200 OK result is
returned to the transmitting MTA, so technically delivery has not been
completed at the time Sieve is executing.

> > But presumably, conceptually at least, -after- sieve has had a whack at
> > the message.  Note that the absence of a sieve script for a given
> > local-part does not mean that sieve doesn't process the message; only
> > that it uses the default action.
>
> No.  It is added regardless of whether or not sieve ever looks at the
> message.  If you have no sieve script, sieve is never invoked.  Yet
> x-sieve is still added.

Well, if there's no sieve script, then sieve did process the message --
and did nothing to it.  Note that X-Sieve is *not* added if Cyrus is
compiled without sieve support.

> LMTP is not a specialized dialect of SMTP.

Again, open to interpretation.  While LMTP is definitely *NOT* SMTP, LMTP
could be described as SMTP without the requirement for the server to queue
messages.

> It is its own separate transport protocol that just happens to share
> some of the language of smtp.  smtp passing off a message to an lmtp
> server is semantically the same as smtp passing it off via ftp or nntp
> or whatever else.  From the perspecitve of the mta, it is the functional
> endpoint.

Only if a 200 OK response is received.  Otherwise it can have a temporary
or permanent failure (just like many other delivery mechanisms).

> > That's where we disagree.  I claim that final delivery doesn't occur
> > until it is actually placed into a mailbox.  Apparently the authors of
> > the lmtpd code tended more towards my view than yours.
>
> Hrm.  Very well.  I claim that when sieve is being used, final delivery is
> done by sieve.  When it is not in use, final delivery is done by lmtpd.
> Ie, one of
>
>   smtp -> lmtp -> mailbox
>   smtp -> lmtp -> sieve -> mailbox
>
> and not
>
>   smtp -> lmtp -> sieve -> lmtp -> mailbox

Actually, this is probably the closest of the three to reality, since lmtp
has yet to issue an OK back to the mail server at the point that the
script is run.

Of course, a sieve failure in our implementation doesn't result in a LMTP
error, it results in an attempt to deliver to the INBOX directly.  And
only if that fails is an LMTP error generated.

Perhaps this is somewhat more accurate

   smtp -> lmtp -> sieve -> (failure?)-yes-> lmtp -> mailbox
                                      \
                                       no-> (sieve actions)

> It may, I don't know (I've never looked).  In any event, procmail doesn't
> have to take the return-path -- my mailer that calls procmail inserts a
> return-path and feeds the message to procmail, which filters and delivers
> my mail.  It is (as far as I'm aware without doing a lot more reading than
> I have time for right now) 100% normal operating behavior for sendmail
> using procmail as a delivery agent.

Here, procmail is the final delivery agent.  That isn't really the case
for Cyrus's sieve implementation.

This all being said, I really am having trouble finding the Return-path
header as a compelling argument to changing the behavior of the
processing.  The contents of the Return-path header are really a part of
the envelope, and an envelope test exists (and it probably shouldn't be
included in bounces/redirects).

If anything, I'd think checking the version of sieve via the X-Sieve
header is more interesting, but I guess that is because I fail to see the
absolute need to check Return-path directly.

However, I am currently leaning towards adopting (a corrected version of)
the patch, since the behavior of including the headers is probably less
confusing to users in general.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456
Research Systems Programmer * /usr/contributed Gatekeeper