backupd and sync_client IOERROR

ellie timoney ellie at fastmail.com
Thu Jun 18 21:01:02 EDT 2020


Hi Marco,

On Thu, Jun 18, 2020, at 10:19 PM, Marco wrote:
> Hello,
> 
>   I'm trying to configure backupd in rolling mode as a final setup. 
> Running a first backup on few users
> 
> sync_client -A -n bck -z -v -v
> 
> after a while the process die with:
> 
> cyrus/sync_client[9540]: MESSAGE received NO response: 
> IMAP_PROTOCOL_ERROR Protocol error
> cyrus/sync_client[9540]: do_folders(): update failed: 
> example.com!user.utente^archivista 'Bad protocol'
> cyrus/sync_client[9540]: IOERROR: do_user_main: Bad protocol for 
> utente.archivista at example.com to [no channel] (tst-msg03-bck.example.com)
> Error from do_user(utente.archivista at example.com): bailing out!
> cyrus/sync_client[9540]: Error in 
> do_user(utente.archivista at example.com): bailing out!
> 
> The backupd host doesn't complain:
> 
> 2020-06-18T11:50:30.528584+02:00 tst-msg03-bck backupd[1308172]: 
> creating sql_db 
> /var/spool/cyr_backup/u/utente.archivista at example.com_cxlsmX.index
> 2020-06-18T11:56:44.971042+02:00 tst-msg03-bck backupd[1308172]: login: 
> tst-msg03.example.com [10.102.42.168] cyr_backup LOGIN User logged in
> 
> So I repeated the same command, but with less verbosity:
> 
> sync_client -A -n bck -z -v
> 
> and it worked well, without errors. Uhm...
> 
> Maybe, before to run the first initial backup (sync_client -A), do I 
> have to stop the sync_client already working in rolling mode?

Rolling mode only makes incremental updates, so if you're starting from a server that already has existing data, you should do the first manual initial backups before enabling the rolling mode.

It's generally safe to run a manual sync_client side-by-side with a rolling one (they will lock things properly and keep out of each other's way).  But the exception to this is that a rolling mode replication can't coherently update an uninitialised replica (since it only sends new changes recorded in the sync log, but not the pre-existing data).  You need to do a full manual replication first, to ensure both sides are the same, before rolling mode can work sensibly.  This is the same regardless of whether your replica is a normal replica or a backupd one.

As you saw, replicating again manually will probably fix you up, at least. :)

> http://www.cyrusimap.org/imap/reference/admin/backups.html says "Update 
> cyrus.conf(5) to add a sync_client(8) invocation to the SERVICES section 
> specifying (at least) the -r and -n channel options.". Really maybe you 
> intend START. In SERVICE I need to specify a listen too, this is not 
> very clear to me.

This is wrong.  It should be the DAEMON section, not the SERVICES section.  I'm fixing this now, so the website should be updated shortly.

You can also run it from the START section, but note that if the sync_client exits early for some reason, master will not restart it.  If you run it from the START section, you will need to monitor/restart it yourself.

> Around of backupd I would ask two other questions:
> 
> 1) The backupd host always claims this:
> 
> backupd[1309970]: DBERROR: reading /var/lib/imap/db/skipstamp, assuming 
> the worst: No such file or directory
> 
> Could you tell me how to fix that DBERROR too?

I think you might need to add the usual recover entry to the START section:
    
recover       cmd="ctl_cyrusdb -r"

I notice this is missing from the backups documentation -- please let me know if it this sorts it out, and I'll fix it up!
 
> 2) Is there a way to delete a user from backupd host?

That's an interesting question...

Under normal use, if a user is deleted from their imap server, then, after the deletion has replicated to the backup server, and after the "backup_retention" period has elapsed, and after a subsequent "ctl_backups compact ..." has been run, then the backup file should be approximately empty.  But I think it will still exist; nothing actually deletes it yet.

To get rid of it manually, you can use "ctl_backups list ..." to find the filename where it exists on disk.  You can also find it by using cyr_dbtool to look up the user's key in backups.db.  You will then need to:

a) use cyr_dbtool to delete that user's key from backups.db
b) delete the named file (it will be like "/some/path/userid_XXXXXX")
c) also delete the corresponding "/some/path/userid_XXXXXX.index" file

Note that if the user still really exists, and a rolling sync_client replicates them to the same backup server again, then the backup file will be recreated (with a new XXXXXX) -- with the same problems as above wrt no-initial-sync.

Cheers,

ellie


More information about the Info-cyrus mailing list