Double Carriage return breaks header ..
Greg A. Woods
woods at weird.com
Tue May 31 17:47:37 EDT 2005
[ On Tuesday, May 31, 2005 at 10:12:10 (+0200), Patrik Henningsson wrote: ]
> Subject: Re: Double Carriage return breaks header ..
>
> So now we know that Cyrus presumably is doing wrong, any brave soul
> contribute with a patch?
I may try my hand at a patch, though not today and not likely this week.
BTW, here's some more detail, and yet another problem, about all this
EOL business.
The big worry in SMTP is, and always has been, the recognition of a bare
<LF> as a line terminator (and thus "<LF>.<LF>" as end of message),
_not_ bare <CR> -- so far as I know there has never been any useful SMTP
implementation that recognized bare <CR> as a line terminator.
The problem with recognizing bare <LF> as EOL being that the client-SMTP
cannot know whether or not to do dot-stuffing to protect the dot in a
"<LF>.<LF>" sequence it wants to send. If it guesses wrong about the
server it's talking to then it either risks message truncation or
message corruption (adding an extra, unwanted, ".").
Despite the fact that RFC 2821 has explicitly forbidden new conforming
implementations from recognizing anything but <CRLF> as a line
terminator the "Right(tm)" solution to this issue is of course to always
MIME encode any message part containing the "<LF>.<LF>" sequence in such
a way that the sequence no longer appears that way in the SMTP data,
since of course one can no more predict the RFC 2821 conformance of a
server-SMTP than once can predict it's use of <LF>. I.e. RFC 2821's
rule is completely useless here -- too many servers will likely always
recognize "<LF>.<LF>" as the end of a message -- MIME encoding is the
only true solution.
Presumably though Cyrus wouldn't have to worry about any of this if its
LMTP server only recognized <CRLF> as the line terminator, as per RFC
2821 (though RFC 2033 is only "just" updated to point to RFC 2821
courtesy RFC 3848), and thus only allowed "<CRLF>.<CRLF>" as the end of
the message.
Unfortunately this is yet another problem with Cyrus (or another
manifestation of the same problem, or maybe just a clearer illustration
of the same problem) -- its LMTP engine not only stuffs newlines before
bare <CR>s in the headers (thus corrupting message headers as we've been
discussing in this thread), but it also recognizes the "<LF>.<LF>"
sequence as the end of the message (thus also risking message truncation
if the client-LMTP doesn't do dot-stuffing):
# echo -e "LHLO building.weird.com\r\nMAIL FROM:<woods at weird.com>\r\nRCPT TO:<woods at building.weird.com>\r\nDATA\r\nTo: gwoods\r\nSubject: TEST\r\r\nX-header: bar\r\n\r\n\n.\ncrline\r\r\nfoo\r\n\r\n.\r\nQUIT\r" | nc -U /var/imap/socket/lmtp
220 building LMTP Cyrus v2.2.10 ready
250-building
250-8BITMIME
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-SIZE 4000000
250-AUTH EXTERNAL
250 IGNOREQUOTA
250 2.1.0 ok
250 2.1.5 ok
354 go ahead
250 2.1.5 Ok
500 5.5.2 Syntax error
500 5.5.2 Syntax error
500 5.5.2 Syntax error
500 5.5.2 Syntax error
221 2.0.0 bye
# od -c /var/spool/imap/w/user/woods/44.
0000000 R e t u r n - P a t h : < w o
0000020 o d s @ w e i r d . c o m > \r \n
0000040 R e c e i v e d : f r o m b
0000060 u i l d i n g . w e i r d . c o
0000100 m ( [ u n i x s o c k e t ]
0000120 ) \r \n \t b y b u i l d i n g
0000140 ( C y r u s v 2 . 2 . 1 0 )
0000160 w i t h L M T P A ; \r \n \t
0000200 T u e , 3 1 M a y 2 0 0 5
0000220 1 7 : 1 2 : 4 2 - 0 4 0 0 \r
0000240 \n X - S i e v e : C M U S i
0000260 e v e 2 . 2 \r \n T o : g w o
0000300 o d s \r \n S u b j e c t : T E
0000320 S T \r \n M e s s a g e - I D :
0000340 < c m u - l m t p d - 2 2 9 7 2
0000360 - 1 1 1 7 5 7 3 9 6 2 - 0 @ b u
0000400 i l d i n g > \r \n D a t e : T
0000420 u e , 3 1 M a y 2 0 0 5
0000440 1 7 : 1 2 : 4 2 - 0 4 0 0 \r \n
0000460 \r \n X - h e a d e r : b a r \r
0000500 \n \r \n \r \n
0000505
Note there are no lines "crline" or "foo" in the body, and presumably
they're the cause of the "syntax error" responses during the transaction.
And also note the problem of the headers ending before the "X-header:",
finally clearly reproducing the problem that was reported at the start
of this thread.
Removing the "\n.\n" from the data stream still reproduces the split
headers problem, but also reveals that the bare <CR> botch inserts a
newline in the headers, not in the body -- "crline" is not followed by a
blank line, but it is missing the extra <CR> that should be there:
# echo -e "LHLO building.weird.com\r\nMAIL FROM:<woods at weird.com>\r\nRCPT TO:<woods at building.weird.com>\r\nDATA\r\nTo: gwoods\r\nSubject: TEST\r\r\nX-header: bar\r\n\r\ncrline\r\r\nfoo\r\n\r\n.\r\nQUIT\r" | nc -U /var/imap/socket/lmtp
220 building LMTP Cyrus v2.2.10 ready
250-building
250-8BITMIME
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-SIZE 4000000
250-AUTH EXTERNAL
250 IGNOREQUOTA
250 2.1.0 ok
250 2.1.5 ok
354 go ahead
250 2.1.5 Ok
221 2.0.0 bye
# od -c /var/spool/imap/w/user/woods/45.
0000000 R e t u r n - P a t h : < w o
0000020 o d s @ w e i r d . c o m > \r \n
0000040 R e c e i v e d : f r o m b
0000060 u i l d i n g . w e i r d . c o
0000100 m ( [ u n i x s o c k e t ]
0000120 ) \r \n \t b y b u i l d i n g
0000140 ( C y r u s v 2 . 2 . 1 0 )
0000160 w i t h L M T P A ; \r \n \t
0000200 T u e , 3 1 M a y 2 0 0 5
0000220 1 7 : 2 3 : 0 8 - 0 4 0 0 \r
0000240 \n X - S i e v e : C M U S i
0000260 e v e 2 . 2 \r \n T o : g w o
0000300 o d s \r \n S u b j e c t : T E
0000320 S T \r \n M e s s a g e - I D :
0000340 < c m u - l m t p d - 2 3 0 4 4
0000360 - 1 1 1 7 5 7 4 5 8 8 - 0 @ b u
0000400 i l d i n g > \r \n D a t e : T
0000420 u e , 3 1 M a y 2 0 0 5
0000440 1 7 : 2 3 : 0 8 - 0 4 0 0 \r \n
0000460 \r \n X - h e a d e r : b a r \r
0000500 \n \r \n c r l i n e \r \n f o o \r \n
0000520 \r \n
0000522
At least Cyrus LMTP doesn't accept <CR> as EOL anywhere but in the
headers. Small consolation, but something....
I guess the question "we" need to ask is whether there are any existing
client-LMTP implementations which might send just <LF> as their
end-of-line terminator. I should hope not....
I'm going to guess that humans don't test LMTP by hand nearly so often
as they do SMTP, so the need for recognizing bare <LF> as a line
terminator, for use with telnet for example, should be nearly nil.
--
Greg A. Woods
H:+1 416 218-0098 W:+1 416 489-5852 x122 VE3TCP RoboHack <woods at robohack.ca>
Planix, Inc. <woods at planix.com> Secrets of the Weird <woods at weird.com>
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
More information about the Info-cyrus
mailing list