Object Storage alpha lands on master

Wed Mar 16 13:01:15 EDT 2016

Thanks Mr Francois for the clue. I have compiled openio-common-2.0.0.c1 and
can now compiled cyrus trunk with openio. But i still don't know how to
configure and test, any example configuration ?

i've setup openio 3 node cluster based on openio wiki

On Wed, Mar 16, 2016 at 7:16 AM, Jean-Francois SMIGIELSKI <
jean-francois.smigielski at openio.io> wrote:

> Hi Sayid,
>
> Cyrus has been updated to reflect OpenIO latest version. The solution
> evolved a lot since the 1.0.1 you mention!
> If you prefer using a release number, the *2.0.0.c1* should be your
> current choice: it is the first candidate for a release scheduled for
> 2016/04, destined to be a stable branch on the long term.
>
> Feel free to ask if you need any additional information.
>
> --
> Jean-Francois
>
> ------------------------------
> *De :* Cyrus-devel [cyrus-devel-bounces+jean-francois.smigielski=
> openio.io at lists.andrew.cmu.edu] de la part de Sayid Munawar via
> Cyrus-devel [cyrus-devel at lists.andrew.cmu.edu]
> *Date d'envoi :* mardi 15 mars 2016 21:27
> *À :* Cyrus Devel
> *Objet :* Fwd: Object Storage alpha lands on master
>
> Hi, i've just tried to compile cyrus with openio, and it gave this error:
>
> libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I. -I./lib -I. -I./lib
> -DLIBEXEC_DIR=\"/usr/cyrus/libexec\" -DSBIN_DIR=\"/usr/cyrus/sbin\"
> -DSYSCONF_DIR=\"/etc\" -DHAVE_CONFIG_H -I/usr/include/libxml2
> -I/usr/include/postgresql -fPIC -fvisibility=hidden -g -O2 -I/usr/include
> -MT imap/imap_libcyrus_imap_la-objectstore_openio.lo -MD -MP -MF
> imap/.deps/imap_libcyrus_imap_la-objectstore_openio.Tpo -c
> imap/objectstore_openio.c  -fPIC -DPIC -o
> imap/.libs/imap_libcyrus_imap_la-objectstore_openio.o
> imap/objectstore_openio.c: In function 'objectstore_put':
> imap/objectstore_openio.c:195:42: warning: passing argument 2 of
> 'oio_sds_upload_from_file' from incompatible pointer type
>      err = oio_sds_upload_from_file (sds, &ul_dst, fname, 0, 0);
>                                           ^
> In file included from imap/objectstore_openio.c:49:0:
> /usr/include/oio_sds.h:250:21: note: expected 'struct oio_url_s *' but
> argument is of type 'struct oio_sds_ul_dst_s *'
>  struct oio_error_s* oio_sds_upload_from_file (struct oio_sds_s *sds,
>                      ^
> imap/objectstore_openio.c:195:11: error: too many arguments to function
> 'oio_sds_upload_from_file'
>      err = oio_sds_upload_from_file (sds, &ul_dst, fname, 0, 0);
>            ^
> In file included from imap/objectstore_openio.c:49:0:
> /usr/include/oio_sds.h:250:21: note: declared here
>  struct oio_error_s* oio_sds_upload_from_file (struct oio_sds_s *sds,
>                      ^
> Makefile:4679: recipe for target
> 'imap/imap_libcyrus_imap_la-objectstore_openio.lo' failed
> make[2]: *** [imap/imap_libcyrus_imap_la-objectstore_openio.lo] Error 1
> make[2]: Leaving directory '/opt/cyrus-imapd'
> Makefile:5800: recipe for target 'all-recursive' failed
> make[1]: *** [all-recursive] Error 1
> make[1]: Leaving directory '/opt/cyrus-imapd'
> Makefile:2511: recipe for target 'all' failed
> make: *** [all] Error 2
> root at jessie:/opt/cyrus-imapd#
>
>
> any clue how to fix this ?
>
> OS: debian jessie amd64
> openio-sds installed version 1.0.1-1
>
>
>
> On Mon, Mar 14, 2016 at 4:14 PM, Bron Gondwana via Cyrus-devel <
> cyrus-devel at lists.andrew.cmu.edu> wrote:
>
>> There were a couple of nits around style:
>>
>> +    int remove_temp_spool_file = 0 ;
>> +    int object_storage_enabled = 0 ;
>>
>> (we don't put a space before the ; anywhere in the rest of the codebase)
>>
>> And some
>> }
>> else
>> {
>>
>> or
>>
>> if (condition)
>> {
>>
>> We cuddle the brace like this:
>>
>> }
>> else {
>>
>> if (condition) {
>>
>> Pretty much exclusively.  But those can be fixed easily enough.  The
>> important thing for me was to check that the non-object storage paths are
>> the same, and they read identically to me.
>>
>> I've done a build and Cassandane test run, and I'm happy with it. Pushing
>> a merge of that branch to master now.  Sorry about taking so long to get to
>> this.
>>
>> Meeting in just under 2 hours, might see you there :)
>>
>> Bron.
>>
>> On Wed, Feb 24, 2016, at 01:39, Raymond Poitras wrote:
>>
>> Hi Bron
>>
>> Sorry for the delay and thank you for pushing this work to master.
>>
>> A) While doing this, some of the logic was broken.  (This is now fix.)
>> B) The new solution works very well with a mix of archive (in object
>> storage) and non archive (on regular spool) emails.
>> C) Unarchive have been fix.  (was used previously to remove email from
>> object storage.  Now a new function is used for that).
>> D) Include reconstruct support.  Works very well if message.db file is
>> reliable.
>>
>> Please let me know if you can update master again from here
>> https://github.com/RaymondPo/cyrus-imapd/tree/Cyrus-OpenIO
>>
>> When is the next meeting? Jean-Francois and I would like to attend
>>
>> Regards
>> Raymond
>>
>>
>> 2015-10-13 8:03 GMT-04:00 Bron Gondwana <brong at fastmail.fm>:
>>
>> I've pushed the objectstorage code based on Raymond and Jean-Francois's
>> work to master.  It's almost exactly what was on Raymond's branch with just
>> a couple of modifications.
>>
>> 1) updated for some API changes I'd made to archiving in append.c
>> 2) fixed compilation under warn-unused-variables
>> 3) normalised logic between append_fromstage and append_copy for handling
>> objectstorage errors by creating non-ARCHIVED files.
>> 4) fixed file copy in mailbox_archive to work the same way as the file
>> moves, where it correctly passes the source/destination file to
>> objectstorage and gets the file back, so unarchive works.
>>
>> WARNING WARNING WARNING: while I've tested that normal operation and
>> normal archive file operation (non-objectstorage) works correctly with
>> Cassandane and FastMail experimental branch, I haven't done more than
>> rudimentary testing with the dummy objectstorage.
>>
>> I'm almost positive that reconstruct won't work correctly with object
>> storage turned on right now.
>>
>> This is still very much work in progress - but putting it on master means
>> that the code will be maintained and updated as other API changes are made.
>>
>> Thanks Raymond and Jean-Francois for all your work on this.  I hope it
>> gets more testing and use.
>>
>> Cheers,
>>
>> Bron.
>>
>> --
>>   Bron Gondwana
>>   brong at fastmail.fm
>>
>>
>> --
>>   Bron Gondwana
>>   brong at fastmail.fm
>>
>>
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20160317/1d220ff8/attachment.html>