Replication and Virtual Domains

[Bron Gondwana]

Ken and David - a couple of questions:
1) is replication expected to work with virtual domains at all?
2) if not - how difficult do you think it would be to add - since
   you've both worked with that code?

I've had a quick glance, but it looks like I need to understand
layers of indirection to get at how mailboxes are looked up during
the sync process.

Silly me, I'm trying to set up both replication and virtual domain support
at the same time - having previously had neither.  I have all the machines
set up, the config in place, and I've even tried running "sync_client -u
brong5m at fastmail.fm", that being my test user account.

No luck though.

The two servers are called imap5 and imap6.  Imap5 is the master, imap6 the
replica in this particular case.  Since we're doing active/active with two
instances of cyrus on each machine, we have:

[root at imap5 root]$ ls -la /etc | grep imap5
-rw-r--r--   1 root  root     1270 May 12 01:37 cyrus-imap5-master.conf
-rw-r--r--   1 root  root     1218 May 12 01:37 cyrus-imap5-replica.conf
lrwxrwxrwx   1 root  root       28 May 12 03:09 cyrus-imap5.conf -> /etc/cyrus-imap5-master.conf
-rw-r--r--   1 root  root     1273 May 12 03:09 imapd-imap5-master.conf
-rw-r--r--   1 root  root     1155 May 12 01:37 imapd-imap5-replica.conf
lrwxrwxrwx   1 root  root       28 May 12 03:09 imapd-imap5.conf -> /etc/imapd-imap5-master.conf

[root at imap6 root]$ ls -la /etc | grep imap5
-rw-r--r--   1 root  root     1270 May 12 01:52 cyrus-imap5-master.conf
-rw-r--r--   1 root  root     1218 May 12 01:52 cyrus-imap5-replica.conf
lrwxrwxrwx   1 root  root       29 May 12 03:31 cyrus-imap5.conf -> /etc/cyrus-imap5-replica.conf
-rw-r--r--   1 root  root     1272 May 12 01:52 imapd-imap5-master.conf
-rw-r--r--   1 root  root     1156 May 12 03:31 imapd-imap5-replica.conf
lrwxrwxrwx   1 root  root       29 May 12 03:31 imapd-imap5.conf -> /etc/imapd-imap5-replica.conf

Allowing us to quickly switch between the various configurations.
The symlink is created by the init.d script after it checks that
there is no other instance running, and the "service IP" for that
machine is also bound using heartbeat's send_arp so that we can
just put 'imap5m.internal' as the hostname for the user's IMAP
server and know that whichever machine has that IP address is the
current master for that data pool.

Similarly, we use symlinks such that /var/cyrus/imap5 is the base
directory for cyrus on both machines, even though the data is on
different disks.  This allows the configs to be entirely identical
on both machines.

So - here's a directory listing after a bunch of changes and forced
replications:

[root at imap5 root]$ find /var/cyrus/imap5/data1/
/var/cyrus/imap5/data1/
/var/cyrus/imap5/data1/a
/var/cyrus/imap5/data1/a/user
/var/cyrus/imap5/data1/a/user/another
/var/cyrus/imap5/data1/n
/var/cyrus/imap5/data1/n/user
/var/cyrus/imap5/data1/n/user/newtest
/var/cyrus/imap5/data1/sync.
/var/cyrus/imap5/data1/domain
/var/cyrus/imap5/data1/domain/e
/var/cyrus/imap5/data1/domain/e/eml.cc
/var/cyrus/imap5/data1/domain/e/eml.cc/t
/var/cyrus/imap5/data1/domain/e/eml.cc/t/user
/var/cyrus/imap5/data1/domain/e/eml.cc/t/user/testing
/var/cyrus/imap5/data1/domain/f
/var/cyrus/imap5/data1/domain/f/fastmail.fm
/var/cyrus/imap5/data1/domain/f/fastmail.fm/b
/var/cyrus/imap5/data1/domain/f/fastmail.fm/b/user
/var/cyrus/imap5/data1/domain/f/fastmail.fm/b/user/brong5m
/var/cyrus/imap5/data1/domain/f/fastmail.fm/b/user/brong5m/2.
/var/cyrus/imap5/data1/domain/f/fastmail.fm/b/user/brong5m/Trash
/var/cyrus/imap5/data1/domain/f/fastmail.fm/b/user/brong5m/Trash/1.
/var/cyrus/imap5/data1/domain/f/fastmail.fm/b/user/brong5m/Sent Items
/var/cyrus/imap5/data1/domain/f/fastmail.fm/b/user/brong5m/Sent Items/1.
/var/cyrus/imap5/data1/domain/f/fastmail.fm/b/user/brong5m/Sent Items/2.
/var/cyrus/imap5/data1/stage.

[root at imap6 root]$ find /var/cyrus/imap5/data1/
/var/cyrus/imap5/data1/
/var/cyrus/imap5/data1/a
/var/cyrus/imap5/data1/a/user
/var/cyrus/imap5/data1/a/user/another
/var/cyrus/imap5/data1/n
/var/cyrus/imap5/data1/n/user
/var/cyrus/imap5/data1/n/user/newtest
/var/cyrus/imap5/data1/sync.
/var/cyrus/imap5/data1/domain
/var/cyrus/imap5/data1/stage.

(note: we made /var/cyrus/imap5/data1/domain on imap6 by hand after
the first replication failed)

It appears that all users created in the default domain ('internal'
in our config) are replicated fine, but the users in other domains
are not.

An attempt to sync the brong5m at fastmail.fm users creates this in the
logs in imap6:

May 12 01:54:42 imap6 syncserver[16522]: login: imap5m.internal [10.202.10.5] repluser DIGEST-MD5 User logged in
May 12 01:54:42 imap6 syncserver[16522]: Failed to access inbox for brong5m at fastmail.fm
May 12 01:54:42 imap6 syncserver[16522]: Unlocked

Let me know if you need the entire configs (minus passwords, natch)

Bron.

-- 
  Bron Gondwana
  brong at fastmail.fm