OpenIO integration in Cyrus3

Conrad Kleinespel conradk at conradk.com
Thu Jun 4 07:43:11 EDT 2015


Hi Jean-François,

Thanks for taking the time to iterate on this.

Regarding configuration, we discussed this at the last meeting and what
would be good is something like this:
- an option to compile Cyrus with OpenIO support, eg: ./configure
--with-openio=yes
- for things like timeouts, namespace names, etc, adding options to
imapd.conf would be the way to go, eg "openio-namespace: OPENIO" or
"archive-openio: yes | no", etc.

I can't answer your other questions at this time (maybe someone else
will), but I mention them at today's meeting at 12pm UTC (2pm French
time). As a reminder in case you can be there yourself, here's the
Hangout URL:
https://plus.google.com/hangouts/_/g4xnqjjb5zvomzeb4kqvja3fz4a

Regarding tests, do you get any specific errors messages in the logs
(syslogs, SMTP server logs, etc) ?

Best regards,

Conrad Kleinespel
conradk at conradk.com
+33 6 23 82 42 79

On Thu, Jun 4, 2015, at 12:22 PM, Jean-Francois SMIGIELSKI wrote:
> Hi!
> 
> Yesterday I woked on integrating OpenIO as a Blob store for Cyrus. I
> temporarily pushed my code in
> https://github.com/jfsmig/oio-cyrus/tree/JFS-openio-integration (the
> repository is a fork for your reference, on github for simplicity
> purposes). Because I sometimes fix and upgrade OpenIO in the same time,
> that code currently requires my own fork of OpenIO at
> https://github.com/jfsmig/oio-sds<https://github.com/jfsmig/oio-sds.>
> 
> This is just a first iteration, managing the download from the blob store
> (in "mailbox_map_record") and the upload (in "mailbox_archive").
> Sorry, this is really a work in progress, not really clean (hardcoded
> configuration, etc).
> 
> At this point, I raised a few questions. I didn't investigated yet around
> them, and I will do in further iterations. Anyway, any useful information
> will be appreciated :)
> 
> * What is the preferred way to manage the configuration pour such a blob
> store module ? I typically need to provide a "namespace name", and maybe
> timeouts, etc. At present, this is hardcoded according to my test
> environment.
> * What is the preferred way to keep a structured in cache? For each
> operation, I need a structure representing the OpenIO client. This
> structure has an internal cache (that takes time to load but greatly
> helps laters). At present, I create a new client for each operation.
> * I currently do poor error management, and later I will maybe need some
> tips about this. (e.g. what is the best behavior when the file is also
> missing on the blob store).
> 
> Last but not least, I currently meet a problem, and I cannot run a single
> test successfully.
> When used in cyrus, the libgridclient.so (from OpenIO) does not behave
> the same as when it is used in another standalone application.
> E.g. When the OIO client receives and parses a reply for an internal RPC,
> the reply contains unexpected fields. This is a clue for bad memory
> management, and my best track for the moment.
> I also experience troubles when trying to debug this. Is there some
> function overloading by cyrus ? (e.g. syslog, fprintf, etc)
> 
> Best regards,
> 
> Jean-François SMIGIELSKI<mailto:jean-francois.smigielski at openio.io>
> OpenIO, Co-founder + R&D Manager
> +33.625135563


More information about the Cyrus-devel mailing list