2.3.9 mod time bug

[Bron Gondwana]

On Fri, Aug 31, 2007 at 08:19:58PM -0400, John Capo wrote:
> For as long as I can remember I have sent the file modtime to the
> sync_server in the protocol stream.  Keeping the modtimes on the
> master and the replica in sync becomes important when doing rsync
> snapshots to a backup machine from the replica.  When switching to
> the replica and then doing backups from the new replica (old master),
> out of sync modtimes causes rather large data transfers to the
> backup machine. :-(

Ahh - this isn't an issue for us because we parse the index files
and only sync new files.  Cyrus spool files never change - or if
they do it's a bug!  We optimise that case away and will only
refetch if the UUID has changed (with md5uuid, this means only
if the message content has changed)

> So I though I would try the new internaldate scheme in 2.3.9 for
> setting modtimes instead of my scheme but it does not work right
> on FreeBSD 4.11 and probably not on other BSD systems.  Dunno about
> Linux.

I'm not sure about to-a-second accuracy.  A few seconds either side
didn't really worry me.

> The calls to utime() (obsolete on BSD) in cmd_upload() set the
> modtime OK but some data written with fwrite() is still buffered
> in the stdio buffers.  The buffered data is not flushed to disk
> till an fclose() call which may be a few to many seconds later.
> The data written with fclose() sets the modtime overwriting the
> time set to the internaldate with utime().

Yeah, I have to admit - now that we parse the internaldate from
the Received headers anyway we don't care so much so I haven't
been maintaining and testing this patch.

I'd be happy to abstract out this section and put in a test in 
configure.in for the BSD equivalent for utime if it's going to
be necessary in the near future.  Depends how "obsolete" it is!
 
> Another thing, the fsync()/fclose() calls in this loop in
> sync_message_fsync() should be reversed, at least on on BSD systems.
> 
>     /* fsync() files in reverse order: ReiserFS FAQ indicates that this
>      * gives best potential for optimisation */
>     for (i = (l->file_count-1) ; i >= 0 ; i--) {
>         fsync(fileno(l->file[i]));
>         fclose(l->file[i]);
>         l->file[i] = NULL;
>     }

Presumably this would solve your issue completely, since there would
be nothing left to flush?
 
> fsync() is a system call on BSD systems.  It will flush the system
> I/O buffers but the stdio buffers may still contain unwritten data.
> fsync() may be hooked into stdio on Linux but not on BSD.

I'm not sure to be honest - though I imagine this was originally
designed on Solaris anyway?

Bron.