OpenIO integration in Cyrus3

Jean-Francois SMIGIELSKI jean-francois.smigielski at openio.io
Thu Jun 4 06:22:45 EDT 2015


Hi!

Yesterday I woked on integrating OpenIO as a Blob store for Cyrus. I temporarily pushed my code in https://github.com/jfsmig/oio-cyrus/tree/JFS-openio-integration (the repository is a fork for your reference, on github for simplicity purposes). Because I sometimes fix and upgrade OpenIO in the same time, that code currently requires my own fork of OpenIO at https://github.com/jfsmig/oio-sds<https://github.com/jfsmig/oio-sds.>

This is just a first iteration, managing the download from the blob store (in "mailbox_map_record") and the upload (in "mailbox_archive").
Sorry, this is really a work in progress, not really clean (hardcoded configuration, etc).

At this point, I raised a few questions. I didn't investigated yet around them, and I will do in further iterations. Anyway, any useful information will be appreciated :)

* What is the preferred way to manage the configuration pour such a blob store module ? I typically need to provide a "namespace name", and maybe timeouts, etc. At present, this is hardcoded according to my test environment.
* What is the preferred way to keep a structured in cache? For each operation, I need a structure representing the OpenIO client. This structure has an internal cache (that takes time to load but greatly helps laters). At present, I create a new client for each operation.
* I currently do poor error management, and later I will maybe need some tips about this. (e.g. what is the best behavior when the file is also missing on the blob store).

Last but not least, I currently meet a problem, and I cannot run a single test successfully.
When used in cyrus, the libgridclient.so (from OpenIO) does not behave the same as when it is used in another standalone application.
E.g. When the OIO client receives and parses a reply for an internal RPC, the reply contains unexpected fields. This is a clue for bad memory management, and my best track for the moment.
I also experience troubles when trying to debug this. Is there some function overloading by cyrus ? (e.g. syslog, fprintf, etc)

Best regards,

Jean-François SMIGIELSKI<mailto:jean-francois.smigielski at openio.io>
OpenIO, Co-founder + R&D Manager
+33.625135563
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20150604/1ee76e98/attachment.html 


More information about the Cyrus-devel mailing list