<HTML><BODY>Thank you for your experience.<br><br>Best regards!<br><br><br><blockquote style="border-left:1px solid #0857A6; margin:10px; padding:0 0 0 10px;">
Четверг, 13 сентября 2018, 19:26 +05:00 от Michael Menge <michael.menge@zdv.uni-tuebingen.de>:<br>
<br>
<div id="">
<div class="js-helper js-readmsg-msg">
<style type="text/css"></style>
<div>
<base target="_self" href="https://e.mail.ru/">
<div id="style_15368487760000000383_BODY">Quoting Evgeniy Kononov <<a href="mailto:egenius@inbox.ru">egenius@inbox.ru</a>>:<br>
<br>
> Hi!<br>
><br>
> Thank you for reply.<br>
> Users can connect to only one server at a time. I move the master <br>
> server to another hardware and at this time it is necessary for <br>
> users to use the mail.<br>
> If this is not a secure configuration, then can I just run <br>
> "sync_client -A" from the master server, and then switch users to a <br>
> replica?<br>
> After that, swap the roles of master-replica between the servers? I'm right ?<br>
><br>
>> We use cyrus aggregator aka cyrus murder, and AFAIK fastmail also uses <br>
>> multiple<br>
>> instances on one server with nginx frontends<br>
><br>
> Can you give an example of the configuration?<br>
<br>
Sure,<br>
<br>
first of some background Infos:<br>
<br>
We recently switched from Cyrus 2.4.20 on SLES 11 SP4 to Cyrus 3.0.8 <br>
on RHEL 7.5 so consult<br>
the man pages for your version.<br>
<br>
Our Mailserver are running as 6 KVM VMs (RHEV) with 20 GB Ram, 8 Cores each on<br>
two locations. We have a total of ~44000 accounts, ~457000 Mailboxes, <br>
and 2x6.5 TB Mails<br>
<br>
Each server is running 3-4 instances. One frontend, two backend/replic<br>
and on one of the servers the cyrus mupdate master. Each Server on one<br>
location is paired with one server on the other location for replication<br>
so in normal operation one backend on server A replicates to a replic on<br>
server B and the backend on server B replicates to the replica on server A.<br>
<br>
Keepalived and ipvs loadbalancer distribute the the load to the <br>
frontend servers.<br>
We use a private subnet for our backend and replic und mupdate instances and a<br>
service ip address for the frontends.<br>
<br>
We move the ip address with the role, so that ma01.mail.localhost on server A<br>
replicate to sl01.mail.localhost on server B. But if we need to switch <br>
to the replic<br>
we will start it with ma01.mail.localhost on server B<br>
<br>
Keeping the master instance for mailbox on the same IP is important, <br>
because updating the<br>
location for all mailboxes in the mupdate master would take to long. <br>
(the mupdate protocol<br>
knows nothing about replication)<br>
<br>
<br>
The main trick to run multiple instances on one server is to use <br>
different cyrus.conf<br>
and imapd.conf files for each instance. We use cyrus_INSTANCE.conf and <br>
imapd_INSTANCE.conf<br>
where INSTANCE is replaced by mu for mupdate, fe for the frontend, be <br>
for the first<br>
backend/replic and re of the second backend/replic<br>
<br>
The choosing of "be" and "re" was not the best as it is easily <br>
confused with the role<br>
in wich each of these instances can run.<br>
<br>
The masterproces is started with "master -C /etc/imapd_INSTANCE.conf <br>
-M /etc/cyrus_INSTANCE.conf -p /var/run/cyrus_instance.pid"<br>
and in the cyrus_INSTANCE.conf you must also use "-C <br>
/etc/imapd_INSTANCE.conf" service, start and event<br>
"cmd" so that the correct conf file is used. For services you also <br>
have to configure "listen="<br>
so that each instance has its own ip to listen on as only one process <br>
can listen on 0.0.0.0 for each port.<br>
In the imapd_INSTANC.conf many directories must be configured.<br>
<br>
We generate the conf files from templates. Where TYPE = INSTANCES<br>
Here are the main parts of our templates<br>
<br>
<br>
========== Cyrus Master ============<br>
# cyrus_@@TYPE@@.conf<br>
# Template MD5SUM: @@MD5SUM@@<br>
<br>
START {<br>
@@TYPE@@recover cmd="ctl_cyrusdb -r -C /etc/imapd_@@TYPE@@.conf"<br>
@@TYPE@@mupdatepush cmd="ctl_mboxlist -m -a -C /etc/imapd_@@TYPE@@.conf"<br>
@@TYPE@@idled cmd="idled -C /etc/imapd_@@TYPE@@.conf"<br>
}<br>
<br>
SERVICES {<br>
@@TYPE@@imap cmd="imapd -U 50 -C /etc/imapd_@@TYPE@@.conf" <br>
listen="@@HOSTNAME@@:imap" prefork=1 maxfds=1024<br>
@@TYPE@@imaps cmd="imapd -U 50 -s -C <br>
/etc/imapd_@@TYPE@@.conf" listen="@@HOSTNAME@@:imaps" prefork=1 <br>
maxfds=1024<br>
@@TYPE@@pop3 cmd="pop3d -C /etc/imapd_@@TYPE@@.conf" <br>
listen="@@HOSTNAME@@:pop3" prefork=1 maxfds=1024<br>
@@TYPE@@pop3s cmd="pop3d -s -C /etc/imapd_@@TYPE@@.conf" <br>
listen="@@HOSTNAME@@:pop3s" prefork=1 maxfds=1024<br>
@@TYPE@@sieve cmd="timsieved -C /etc/imapd_@@TYPE@@.conf" <br>
listen="@@HOSTNAME@@:sieve" prefork=0 maxfds=1024<br>
@@TYPE@@lmtp cmd="lmtpd -U 5 -C /etc/imapd_@@TYPE@@.conf" <br>
listen="@@HOSTNAME@@:lmtp" prefork=1 maxfds=1024<br>
@@TYPE@@lmtpunix cmd="lmtpd -U 5 -C /etc/imapd_@@TYPE@@.conf" <br>
listen="/srv/cyrus-@@TYPE@@/socket/lmtp" prefork=1 maxfds=1024<br>
}<br>
<br>
EVENTS {<br>
@@TYPE@@checkpoint cmd="ctl_cyrusdb -c -C <br>
/etc/imapd_@@TYPE@@.conf" period=30<br>
@@TYPE@@delprune cmd="cyr_expire -E 3 -X 60 -D 60 -C <br>
/etc/imapd_@@TYPE@@.conf" at=0100<br>
@@TYPE@@tlsprune cmd="tls_prune -C /etc/imapd_@@TYPE@@.conf" at=0430<br>
@@TYPE@@squatter cmd="squatter -C /etc/imapd_@@TYPE@@.conf -i" at=2200<br>
}<br>
<br>
======= Cyrus Replic ==============<br>
# cyrus_@@TYPE@@.conf<br>
# Template MD5SUM: @@MD5SUM@@<br>
<br>
START {<br>
@@TYPE@@recover cmd="ctl_cyrusdb -r -C /etc/imapd_@@TYPE@@.conf"<br>
}<br>
<br>
SERVICES {<br>
@@TYPE@@syncserver cmd="sync_server -C /etc/imapd_@@TYPE@@.conf" <br>
listen="@@HOSTNAME@@:csync" prefork=1 maxfds=1024<br>
@@TYPE@@imap cmd="imapd -U 50 -C /etc/imapd_@@TYPE@@.conf" <br>
listen="@@HOSTNAME@@:imap" prefork=1 maxfds=1024<br>
}<br>
<br>
EVENTS {<br>
@@TYPE@@checkpoint cmd="ctl_cyrusdb -c -C <br>
/etc/imapd_@@TYPE@@.conf" period=30<br>
@@TYPE@@delprune cmd="cyr_expire -E 3 -X 60 -D 60 -C <br>
/etc/imapd_@@TYPE@@.conf" at=0100<br>
}<br>
<br>
===============<br>
<br>
<br>
Configuration for Backend/Failover Instance<br>
# Template MD5SUM: @@MD5SUM@@<br>
servername: @@HOSTNAME@@<br>
configdirectory: /srv/cyrus-@@TYPE@@<br>
partition-default: /srv/cyrus-@@TYPE@@<br>
partition-ssd: /srv/cyrus-@@TYPE@@/ssd-part<br>
metapartition-ssd: /srv/cyrus-ssd-@@TYPE@@/meta/ssd-part<br>
metapartition_files: header index cache expunge squat annotations lock <br>
dav archivecache<br>
archivepartition-ssd: /srv/cyrus-hdd-@@TYPE@@/archive/ssd-part<br>
archive_enabled: 1<br>
proc_path: /srv/tmpfs/proc-@@TYPE@@<br>
mboxname_lockpath: /srv/tmpfs/lock-@@TYPE@@<br>
defaultpartition: ssd<br>
admins: XXX<br>
<br>
mupdate_server: @@MUPDATEHOSTNAME@@<br>
mupdate_port: 3905<br>
mupdate_authname: XXX<br>
mupdate_password: XXX<br>
proxy_authname: XXX<br>
proxy_password: XXX<br>
proxyservers: XXX<br>
<br>
allowallsubscribe: 1<br>
<br>
sync_host: @@SYNCHOST@@<br>
sync_authname: XXX<br>
sync_password: XXX<br>
sync_port: 2005<br>
guid_mode: sha1<br>
sync_log: 1<br>
sync_shutdown_file: /srv/cyrus-@@TYPE@@/sync/shutdown<br>
<br>
sievedir: /srv/cyrus-@@TYPE@@/sieve<br>
sieve_extensions: fileinto reject vacation imapflags notify include <br>
envelope body relational regex subaddress copy<br>
sieve_maxscriptsize: 150<br>
<br>
syslog_prefix: @@TYPE@@<br>
<br>
============== Imapd Replic ===============<br>
# Configuration for Slave (Replica) Instance<br>
# Template MD5SUM: @@MD5SUM@@<br>
servername: @@HOSTNAME@@<br>
configdirectory: /srv/cyrus-@@TYPE@@<br>
partition-default: /srv/cyrus-@@TYPE@@<br>
partition-ssd: /srv/cyrus-@@TYPE@@/ssd-part<br>
metapartition-ssd: /srv/cyrus-ssd-@@TYPE@@/meta/ssd-part<br>
metapartition_files: header index cache expunge squat annotations lock <br>
dav archivecache<br>
archivepartition-ssd: /srv/cyrus-hdd-@@TYPE@@/archive/ssd-part<br>
archive_enabled: 1<br>
<br>
proc_path: /srv/tmpfs/proc-@@TYPE@@<br>
mboxname_lockpath: /srv/tmpfs/lock-@@TYPE@@<br>
defaultpartition: ssd<br>
admins: XXX<br>
<br>
allowusermoves: 1<br>
allowallsubscribe: 1<br>
<br>
proxy_authname: XXX<br>
proxy_password: XXX<br>
proxyservers: XXX<br>
<br>
sievedir: /srv/cyrus-@@TYPE@@/sieve<br>
sieve_extensions: fileinto reject vacation imapflags notify include <br>
envelope body relational regex subaddress copy<br>
sieve_maxscriptsize: 150<br>
<br>
sasl_pwcheck_method: saslauthd<br>
sasl_mech_list: plain login<br>
allowanonymouslogin: no<br>
syslog_prefix: @@TYPE@@<br>
=================================<br>
<br>
The sync client is started as own service<br>
<br>
I hope it helps<br>
<br>
Regards<br>
<br>
Michael<br>
<br>
> Best regards.<br>
><br>
>> Четверг, 13 сентября 2018, 13:22 +05:00 от Michael Menge <br>
>> <<a href="mailto:michael.menge@zdv.uni-tuebingen.de">michael.menge@zdv.uni-tuebingen.de</a>>:<br>
>><br>
>> Hi,<br>
>><br>
>> This setup is NOT SUPPORTED and WILL BREAK if the replication process<br>
>> is triggered<br>
>> from the wrong server (user is active on both servers, user switched<br>
>> from one server<br>
>> to the other while the sync-log file is still processed, after split<br>
>> brain) and<br>
>> some mailboxes have been subscribed, renamed created deleted.....<br>
>><br>
>> Also there is the risk of a race condition with subscriptions, if a<br>
>> user subscribes<br>
>> to multiple folders, the first will trigger a sync from A to B, but as<br>
>> the folder<br>
>> is subscribed on B it will trigger a sync from B to A, witch can <br>
>> undo the next<br>
>> folder subscription.<br>
>><br>
>> These are only some cases that came to my mind. There will be more<br>
>> cases and it<br>
>> will be hard to debug. So DON'T DO IT!<br>
>><br>
>> What we do is, that we have distributed our users between multiple<br>
>> instances, and each server is running one instance as master and one other<br>
>> as replic. In case of failure or maintenance we stop the master <br>
>> instance, and<br>
>> promote the corresponding replic and configure them so that they will sync<br>
>> them back. If the old master is up to date we switch them back.<br>
>><br>
>> We use cyrus aggregator aka cyrus murder, and AFAIK fastmail also uses<br>
>> multiple<br>
>> instances on one server with nginx frontends<br>
>><br>
>> Regards,<br>
>><br>
>> Michael<br>
>><br>
<br>
<br>
--------------------------------------------------------------------------------<br>
M.Menge Tel.: (49) 7071/29-70316<br>
Universität Tübingen Fax.: (49) 7071/29-5912<br>
Zentrum für Datenverarbeitung mail: <br>
<a href="mailto:michael.menge@zdv.uni-tuebingen.de">michael.menge@zdv.uni-tuebingen.de</a><br>
Wächterstraße 76<br>
72074 Tübingen<br>
<br>
</div>
<base target="_self" href="https://e.mail.ru/">
</div>
</div>
</div>
</blockquote>
<br>
<br>-- <br>Evgeniy Kononov<br></BODY></HTML>