Recent (probably MacOS) mail app provoking endless cyrus.index writes on 2.3 server.

[Bron Gondwana]

On Wed, Oct 14, 2015, at 22:49, Eric Luyten wrote:
> On Wed, October 14, 2015 11:55 am, Eric Luyten wrote:
> > On Tue, October 13, 2015 10:43 pm, Bron Gondwana wrote:
> >
> >> On Wed, Oct 14, 2015, at 00:26, ktm at rice.edu wrote:
> >>
> >>
> >>> The El Capitan Mac Mail was recently released. It definitely has a
> >>> problem. We saw our typical sync log size go from 10k when busy to 100k+.
> >>> We are
> >>> going to recommend that people wait to upgrade to that release until the
> >>> problem is addressed.
> >>
> >> I'll look into whether this is a problem (or as much of a problem) on later
> >>  releases! Just to be clear - have you tracked the issue down to the
> >> EXPUNGE
> >> command using IO?
> >
> >
> >
> > Bron,
> >
> >
> >
> > I am more leaning towards issues in the handling of "IDLE"
> >
> >
> >
> > Is that a 'DONE' with a tag in the (almost) middle ???
> >
> >
> >
> >
> > <1444816251<8<1444816251<7314.6476 IDLE
> >
> >> 1444816251>+ idling
> >>
> > <1444816251<D<1444816251<ONE
> >
> >> 1444816251>87314.6476 OK Completed
> >>
> > <1444816251<8<1444816251<7315.6476 EXPUNGE
> >
> >> 1444816251>* 0 EXISTS
> >>
> > * 0 RECENT
> > 87315.6476 OK Completed
> > <1444816251<8<1444816251<7316.6476 IDLE
> >
> >> 1444816251>+ idling
> >>
> > <1444816251<D<1444816251<ONE
> >
> >> 1444816251>87316.6476 OK Completed
> >>
> > <1444816251<8<1444816251<7317.6476 EXPUNGE
> >
> >> 1444816251>* 0 EXISTS
> >>
> > * 0 RECENT
> > 87317.6476 OK Completed
> > <1444816251<8<1444816251<7318.6476 IDLE
> >
> >> 1444816251>+ idling
> >>
> > <1444816251<D<1444816251<ONE
> >
> >> 1444816251>87318.6476 OK Completed
> >>
> > <1444816251<8<1444816251<7319.6476 EXPUNGE
> >
> >> 1444816251>* 0 EXISTS
> >>
> > * 0 RECENT
> > 87319.6476 OK Completed
> >
> >
> >
> >
> >> From http://tools.ietf.org/html/rfc2177 :
> >>
> >
> >
> > "The IDLE command is sent from the client to the server when the
> > client is ready to accept unsolicited mailbox update messages.  The server
> > requests a response to the IDLE command using the continuation ("+") response.
> > The IDLE command remains active until the client
> > responds to the continuation, and as long as an IDLE command is active, the
> > server is now free to send untagged EXISTS, EXPUNGE, and other messages at any
> > time.
> >
> > The IDLE command is terminated by the receipt of a "DONE"
> > continuation from the client; such response satisfies the server's continuation
> > request.  "
> >
> >
> >
> > Again ... if this has been solved in 2.4 or 2.5 branches, we'll upgrade
> > (management decisions on moving to Google Apps or Office365 taken into
> > account)
> 
> 
> 
> Apologies for following up on follow-ups to myself...
> 
> 
> It looks like a timestamp, not a tag.
> Perhaps the telemetry logging is 2.3 was not 100% bugfree...

Yeah, it's interesting that it puts a timestamp AFTER the 'D'.  I wonder
if it's being sent the D first, and then the ONE in a separate packet.

> Q: Why does Mac Mail generate an unneeded 'EXPUNGE' ?

Absolutely no idea.  I think we have an upgraded test mac here in the
office.  I'll set it up and see what it does.

> Bron, how can I turn off 'IDLE' in my 2.3 server capabilities ?

Ooh - there's suppress_capabilities in 2.4, but I'm not sure that ever got
backported.  You could certainly just comment it out in imapd.c and
rebuild.  Yeah... looking at the history this was added early in the 2.4
series.

The commit is 0a82c8e952afa70d69cc15840d912331fcfba262 from Dec 2010.
It will probably backport cleanly.

Bron.

-- 
  Bron Gondwana
  brong at fastmail.fm