Folder subscription issue
Egoitz Aurrekoetxea
egoitz at sarenet.es
Thu Jul 18 10:12:58 EDT 2019
Hi!!,
Fine! Very happy sharing then :) :) . It only handles email. For Calendars/Contacts we have been long time now, using Davical (to which we contributed in it’s day https://wiki.davical.org/index.php/DAViCal-cli <https://wiki.davical.org/index.php/DAViCal-cli>) . We don’t refuse to use Caldav with Cyrus, it’s just we did the system previous to Cyrus Caldav system.
I attach the code in this email. I explain how we use it. We have each mailbox server running this code as a cron job and we have too some servers with Cyrus IMAP for just storing removed content (without the cron obviously). Each user in the restore server (a normal mailbox server but just for storing deleted content) is something like : user_domain at recuperaciones.saremail.com <mailto:user_domain at recuperaciones.saremail.com>. All our servers have autocreate feature (although in our mailbox servers is not being used nowadays). So, we keep track of what has been removed in a mailbox server with two elements… the Cyrus log and cyradm command. With cyradm command we keep track of deleted “folders" in each user account. With the log, we know where expunges had been run. Later, we take the DELETED mailboxes (the folders of each user) and upload them to Saremail-Restore. After that, we check the log (from some hours before till the present moment). Then we ask unexpunge to see what has been removed in each place. We upload them. We keep track in a database of what exactly has dealed with and what is remaining to deal with, so in the case a fail over to a slave is produced unexpunges can then be run there, even if there’s nothing in the logs that say that (because it’s obviously a slave).
If you think it could be useful, perhaps could be uploaded to contrib directory…
Cheers!
Egoitz Aurrekoetxea
Dpto. de sistemas
944 209 470
Parque Tecnológico. Edificio 103
48170 Zamudio (Bizkaia)
egoitz at sarenet.es <mailto:undefined>
www.sarenet.es <http://www.sarenet.es/>
Antes de imprimir este correo electrónico piense si es necesario hacerlo.
> El 18 jul 2019, a las 14:09, Bron Gondwana <brong at fastmail.fm> escribió:
>
> Awesome, I'll take a look. We're just talking about having some level of self-service restore in JMAP as well, so it'll be great to look at what you've done.
>
> Cheers,
>
> Bron.
>
> On Thu, Jul 18, 2019, at 20:30, Egoitz Aurrekoetxea wrote:
>> Thanks :) :)
>>
>> Take a look please at what I told you too in a new thread about the contribution with our deleted mail restoring system… perhaps it’s interesting for Cyrus?. For us it’s pretty useful at least… it allows users to restore deleted mail by their own…
>>
>> Cheers!!
>>
>>
>>
>> Egoitz Aurrekoetxea
>> Dpto. de sistemas
>> 944 209 470
>> Parque Tecnológico. Edificio 103
>> 48170 Zamudio (Bizkaia)
>> egoitz at sarenet.es
>> www.sarenet.es <http://www.sarenet.es/>
>>
>> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
>>
>>> El 18 jul 2019, a las 12:22, Bron Gondwana <brong at fastmail.fm <mailto:brong at fastmail.fm>> escribió:
>>>
>>> Hmmm.... so sync_client isn't validating the user match properly. Fun. That's definitely a bug that we should fix.
>>>
>>> I'll ask ellie (CC'd) to play with the test on that, or guide you through the process of adding one to Cassandane!
>>>
>>> Cheers,
>>>
>>> Bron.
>>>
>>> On Thu, Jul 18, 2019, at 17:22, Egoitz Aurrekoetxea wrote:
>>>> Hi!,
>>>>
>>>> Yes, I’m so sorry for the noise… but you know…. It’s an un modified script in more than 8 years, not done by me… which you assume it works… because indeed, it copies mailbox content (sync_client -u aaa^aaaa at bbb.es <mailto:aaa%5Eaaaa at bbb.es> seems to copy mailbox content as aaa.aaaa at bbb.es <mailto:aaa.aaaa at bbb.es> but not subs) but not “properly” the subs, sieve… Basically taking a look at some stored consoles (ssh sessions) of the migration…. I noted that were in fact those USER ___^____ at ____ so I did… I’m doing it by me!! Then looked at that script and saw….
>>>>
>>>> Sorry for the time wasted Ellie.. same Bron,
>>>>
>>>> Cheers!
>>>>
>>>>
>>>> Egoitz Aurrekoetxea
>>>> Dpto. de sistemas
>>>> 944 209 470
>>>> Parque Tecnológico. Edificio 103
>>>> 48170 Zamudio (Bizkaia)
>>>> egoitz at sarenet.es <mailto:egoitz at sarenet.es>
>>>> www.sarenet.es <http://www.sarenet.es/>
>>>>
>>>> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
>>>>
>>>>> El 17 jul 2019, a las 23:58, Bron Gondwana <brong at fastmailteam.com <mailto:brong at fastmailteam.com>> escribió:
>>>>>
>>>>> Oh awesome - thanks for letting us know :) This will save ellie spending more time on it, though it's good to have tests for it anyway.
>>>>>
>>>>> Bron.
>>>>>
>>>>> On Wed, Jul 17, 2019, at 23:56, Egoitz Aurrekoetxea wrote:
>>>>>> Hi mates,
>>>>>>
>>>>>> Reproduced and properly located (not a Cyrus bug). We have to apologize for the generated noise. Cyrus replication works just fine.
>>>>>>
>>>>>> As I explained before, we fetch from Mysql the accounts each server has (instead of using -A) for no reason… just for historical reason…
>>>>>>
>>>>>> So, when we fetched the emails, as in very old (now) versions of Cyrus a conversion from . to ^ seem to be needed (the first Cyrus version was 2.0 o even older), we were doing the following in the Perl script :
>>>>>>
>>>>>> while (@fila = $query->fetchrow()) {
>>>>>> $fila[0] =~ s/\./^/g;
>>>>>> $email = $fila[0]."@".$fila[1];
>>>>>> system("/usr/local/bin/sudo -u cyrus /usr/local/cyrus/sbin/sync_client -S xxxxxx.sarenet.es <http://xxxxxx.sarenet.es/> -v -u ".$email);
>>>>>>
>>>>>> This explained the existence of .seen and .sub files with “^” and Sieve dirs with ‘^’ too. That didn’t in fact explain, why they had non zero size. The reason of the non zero size (of files with ^ and ending in .sub and .seen and Sieve dirs), is that as we come from Cyrus versions 2.3 (and probably 2.4 perhaps do it) used the ‘^’ in the seen and sub filenames and Sieve names. So if you do a sync_client of a user with ‘^’ , even in 3.0 it copies you that file content of .sub and .seen from the master to the slave (because it exists! because it has copied with sync_client too but from a 2.3/2.4 version), even when they have not been used in a 3.0 version. Apart the own mail account content is always copied and sync_client doesn’t complain about it (whether you use ^ or a dot for the user).
>>>>>>
>>>>>> So mates, thanks a lot again for your time and I honestly think I’ll sleep better today :)
>>>>>>
>>>>>> We have just discovered these, so we wanted to clarify it.
>>>>>>
>>>>>> Thanks again,
>>>>>> Regards,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Egoitz Aurrekoetxea
>>>>>> Dpto. de sistemas
>>>>>> 944 209 470
>>>>>> Parque Tecnológico. Edificio 103
>>>>>> 48170 Zamudio (Bizkaia)
>>>>>> egoitz at sarenet.es <mailto:egoitz at sarenet.es>
>>>>>> www.sarenet.es <http://www.sarenet.es/>
>>>>>>
>>>>>> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
>>>>>>
>>>>>>> El 17 jul 2019, a las 12:51, Egoitz Aurrekoetxea <egoitz at sarenet.es <mailto:egoitz at sarenet.es>> escribió:
>>>>>>>
>>>>>>> Anyway, not being able to reproduce it… working on reproducing….
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Egoitz Aurrekoetxea
>>>>>>> Dpto. de sistemas
>>>>>>> 944 209 470
>>>>>>> Parque Tecnológico. Edificio 103
>>>>>>> 48170 Zamudio (Bizkaia)
>>>>>>> egoitz at sarenet.es <mailto:egoitz at sarenet.es>
>>>>>>> www.sarenet.es <http://www.sarenet.es/>
>>>>>>>
>>>>>>> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
>>>>>>>
>>>>>>>> El 16 jul 2019, a las 20:23, Egoitz Aurrekoetxea <egoitz at sarenet.es <mailto:egoitz at sarenet.es>> escribió:
>>>>>>>>
>>>>>>>> Perhaps mbname_userid it’s not the exact function… perhaps we could just call something like _append_extbuf() in this case but…. That’s what I was trying to explain… could that be?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Egoitz Aurrekoetxea
>>>>>>>> Dpto. de sistemas
>>>>>>>> 944 209 470
>>>>>>>> Parque Tecnológico. Edificio 103
>>>>>>>> 48170 Zamudio (Bizkaia)
>>>>>>>> egoitz at sarenet.es <mailto:egoitz at sarenet.es>
>>>>>>>> www.sarenet.es <http://www.sarenet.es/>
>>>>>>>>
>>>>>>>> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
>>>>>>>>
>>>>>>>>> El 16 jul 2019, a las 20:20, Egoitz Aurrekoetxea <egoitz at sarenet.es <mailto:egoitz at sarenet.es>> escribió:
>>>>>>>>>
>>>>>>>>> mbname_userid
>>>>>>>>
>>>>>>>>
>>>>>>>> ----
>>>>>>>> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
>>>>>>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
>>>>>>>> To Unsubscribe:
>>>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
>>>>>>>
>>>>>>>
>>>>>>> ----
>>>>>>> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
>>>>>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
>>>>>>> To Unsubscribe:
>>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
>>>>>>
>>>>>> ----
>>>>>> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
>>>>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
>>>>>> To Unsubscribe:
>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
>>>>>
>>>>> --
>>>>> Bron Gondwana, CEO, Fastmail Pty Ltd
>>>>> brong at fastmailteam.com <mailto:brong at fastmailteam.com>
>>>>>
>>>>>
>>>>> ----
>>>>> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
>>>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
>>>>> To Unsubscribe:
>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
>>>>
>>>> ----
>>>> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
>>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
>>>> To Unsubscribe:
>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
>>>
>>> --
>>> Bron Gondwana
>>> brong at fastmail.fm <mailto:brong at fastmail.fm>
>>>
>>>
>>> ----
>>> Cyrus Home Page: http://www.cyrusimap.org/ <http://www.cyrusimap.org/>
>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <http://lists.andrew.cmu.edu/pipermail/info-cyrus/>
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus <https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus>
>
> --
> Bron Gondwana
> brong at fastmail.fm <mailto:brong at fastmail.fm>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20190718/8abce83c/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: saremail-restore.zip
Type: application/zip
Size: 14185 bytes
Desc: not available
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20190718/8abce83c/attachment-0001.zip>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20190718/8abce83c/attachment-0003.html>
More information about the Info-cyrus
mailing list