APPEND vs RFC2822 vs STD0011

[Edward Reid]

At 12:28 PM -0400 7/9/03, Cyrus Daboo wrote:
> On the NULL issue, IMAP does not allow bare NULLs in any data that
>either
> the server or client sends. If you check the formal syntax you will
>see
> that the 'literal' element used to send the message content in an
>APPEND
> explicitly excludes NULL as a valid character.

Ah-ha. Excellent. Thanks, I can definitely run with that.

> Bare CR or LF is another issue...

That's the one that's left.

BTW, in response to email off the list, I should make it clear that I'm
only concerned with the body, not headers.

At 01:35 PM -0400 7/9/03, John Alton Tamplin wrote:
> Clearly a client is required to properly handle a NO or BAD response
>to
> an APPEND command.

I agree. I have other issues with what Eudora considers "proper
handling". They are saying that a dialog box and ceasing operation is
proper handling, I say it's not. Them: "Where does it say that Eudora
is an autoresponder?" Me: "Well, duh, in the User Manual." I did not
mention this part because it's not an issue for this list. It's
certainly going to be part of my next response to Qualcomm, but I
wanted to gather information on the other issues as well.

> The RFC specifies the behavior of
> both the client and the server, and saying the message SHOULD be in
> RFC2822 format means that the server can choose to relax that rule if
>it
> has a good reason just as well as it does for the client.

What RFC3501 says is

      The APPEND command appends the literal argument as a new message
      to the end of the specified destination mailbox.  This argument
      SHOULD be in the format of an [RFC-2822] message.

Since it's the client that constructs the literal, it appears to me
that the RFC is giving the client the choice. The language appears to
specify only the client/server combination. Obviously it would be
better if the language were clearer, but it's not. In other places, the
RFC explicitly refers to the server, for example, the lines immediately
after the above:

      8-bit
      characters are permitted in the message.  A server implementation
      that is unable to preserve 8-bit data properly MUST be able to
      reversibly convert 8-bit APPEND data to 7-bit using a [MIME-IMB]
      content transfer encoding.

           Note: There MAY be exceptions, e.g., draft messages, in
           which required [RFC-2822] header lines are omitted in
           the message literal argument to APPEND.  The full
           implications of doing so MUST be understood and
           carefully weighed.

I had not previously noted the implications of that last paragraph. It
implies that even the headers are not necessarily required to be
RFC2822 compliant. But it's even less clear about whether the client or
the server makes the choice. But again, I'm not personally concerned
about headers.

The formal syntax is no help in this case, because it includes no
restriction beyond "literal".

> RFC3501 specifically requires 2822 rather than 822, and even says that
> all references to 822 should be considered as 2822.  If a mail client
> claims to conform to RFC3501, then the mail messages it sends should
> conform to RFC2822 not RFC822.  If it wants to claim conformance only
>to
> an older IMAP RFC, that is fine.

We are not talking about a case where RFC3501 mentions RFC822. We're
talking about a case where RFC3501 mentions RFC2822 but does not make
it a requirement, and is vague on exactly what is allowed and whether
the client or the server decides what is allowed.

> The data is not allowed to contain nulls -- from RFC3501, 4.3.1:

Thanks. The formal syntax, as Cyrus Daboo mentioned, is clearer. But in
any case, I now have more than adequate documentation about nulls to
hit the Eudora people with.

> Regarding bare newlines, RFC3501 2.2 states:
>
>    All interactions transmitted by client and server are in the form
>of
>    lines, that is, strings that end with a CRLF.  The protocol
>receiver
>    of an IMAP4rev1 client or server is either reading a line, or is
>    reading a sequence of octets with a known count followed by a line.

This clearly does not mean that a literal cannot contain CR or LF -- in
fact, in general the APPEND literal will always contain CRLFs which do
not end the literal. The literal is prefix-coded with the octet count,
and so the rule about lines ending in CRLF does not apply until the
octet count is exhausted. The part of 2.2 that applies to this
situation is the last line, "reading a sequence of octets with a known
count".

Mind, you, I'd prefer to find solid reasons to tell Eudora to clean up
its act. I have that now with respect to nulls. Bare newlines still
look ambiguous.

Edward