sync_client failing with "Fatal error: failed to finish reading file!"

[Nic Bernstein]

List,
Checking the source for dlist.c, here is the context for this error
[printfile()]:

    static void printfile(struct protstream *out, const struct dlist *dl)
    {
        char buf[4096];
        struct stat sbuf;
        FILE *f;
        unsigned long size;

        f = fopen(dl->sval, "r");
        if (!f) {
    	syslog(LOG_ERR, "IOERROR: Failed to read file %s", dl->sval);
    	prot_printf(out, "NIL");
    	return;
        }
        if (fstat(fileno(f), &sbuf) == -1) {
    	syslog(LOG_ERR, "IOERROR: Failed to stat file %s", dl->sval);
    	prot_printf(out, "NIL");
    	fclose(f);
    	return;
        }
        size = sbuf.st_size;
        if (size != dl->nval) {
    	syslog(LOG_ERR, "IOERROR: Size mismatch %s (%lu != " MODSEQ_FMT ")",
    	       dl->sval, size, dl->nval);
    	prot_printf(out, "NIL");
    	fclose(f);
    	return;
        }

        prot_printf(out, "%%{");
        prot_printastring(out, dl->part);
        prot_printf(out, " ");
        prot_printastring(out, message_guid_encode(&dl->gval));
        prot_printf(out, " %lu}\r\n", size);

        while (size) {
    	int n = fread(buf, 1, (size > 4096 ? 4096 : size), f);
    	if (n <= 0) break;
    	prot_write(out, buf, n);
    	size -= n;
        }
        fclose(f);

        if (size) fatal("failed to finish reading file!", EC_IOERR);
    }

The only way we can see this happening is if size <0; in other words,
something has written more into the file since we opened it.  Is that a
correct interpretation?  If so, the error message doesn't really jive
with the error condition.  Shouldn't the test be:
    if (size > 0) fatal...
instead?  If size < 0, then manifestly we have finished reading the file.

Cheers,
    -nic
On 06/11/2013 01:34 PM, Nic Bernstein wrote:
> Friends,
> We've been running a pair of replicas with Ken's
> cyrus-imapd-2.4.17-caldav-beta (4 and 5) recently, in preparation for
> a migration away from a single 2.4.11 server.  The configuration is
> the 2.4.11 box is the master, it replicates to the first 2.4.17 box,
> which is also running a sync_client instance to replicate to the
> second 2.4.17 box.  The intent is that once the client is ready, we
> abandon the 2.4.11 box and the first 2.4.17 instance becomes the
> master and the second the replica.
>
> This has all been running pretty well, but lately the sync_client on
> the middle box has failed a few times.  Each time, I see this in the
> logs (The 2.4.11 master server has nothing in this time interval):
>
>     # The 2.4.17 replica -- in the middle:
>     Jun 11 13:36:09 mailbox cyrus/sync_client[29338]: sync_client RESTART succeeded
>     Jun 11 13:36:10 mailbox cyrus/sync_client[29338]: COMPRESS received NO response: Compression already active: DEFLATE
>     Jun 11 13:36:10 mailbox cyrus/sync_client[29338]: Failed to enable compression, continuing uncompressed
>     Jun 11 13:36:52 mailbox cyrus/sync_client[29338]: Fatal error: failed to finish reading file!
>
>     # The 2.4.17 replica, end of the chain:
>     Jun 11 13:36:16 ia24 cyrus/syncserver[21234]: accepted connection
>     Jun 11 13:36:16 ia24 cyrus/syncserver[21234]: cmdloop(): startup
>     Jun 11 13:36:16 ia24 cyrus/syncserver[21234]: login: mailbox.wi.occinc.com [192.168.190.24] mailproxy PLAIN User logged in
>     Jun 11 13:36:59 ia24 cyrus/syncserver[21234]: IOERROR: reading message: unexpected end of file
>
> Can anyone tell me what this is indicative of?  A search of the web
> doesn't turn up any hits for "Fatal error: failed to finish reading file!"
>
> Thanks in advance!
>     -nic
> -- 
> Nic Bernstein                             nic at onlight.com
> Onlight, Inc.                             www.onlight.com
> 219 N. Milwaukee St., Suite 2a            v. 414.272.4477
> Milwaukee, Wisconsin  53202
>
>
> ----
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-- 
Nic Bernstein                             nic at onlight.com
Onlight, Inc.                             www.onlight.com
219 N. Milwaukee St., Suite 2a            v. 414.272.4477
Milwaukee, Wisconsin  53202

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20130611/42daddf3/attachment.html