Notes 24 June

ellie timoney ellie at fastmail.com
Thu Jun 27 23:57:43 EDT 2019


So,

> Robert:
> * Cyrus work:
>  - some of the Xapian stuff would be good to test with older index db formats.
>  - It would be good to run tests with two different binaries, one to create old-style data and the other one to see that things still work!
>  - this would be a good general thing, both for replication and for our own testing!
>  - The Xapian issues in the last little while would have been found if we had legacy version testing.
> 

I did a bunch of work on this ages ago, but I forget about it a lot, and I'm not sure anyone but me really knows it's there because we all treat our Cassandane config as set-and-forget.

Cassandane _does_, _sort of_, have support for multiple-version testing. If you have a look in your cassandane.ini (or cassandane.ini.example), you will see that there are separate sections for [cyrus default], [cyrus replica] and [cyrus murder].

When Cassandane spins up multiple Cyrus instances for the replication/murder/etc tests, it'll usually use the [cyrus default] cyrus installation, but if there are configured [cyrus replica] and/or [cyrus murder] installations, it will use those to start up any replica or murder instances needed by the tests.

So, to do multiple-version testing, you install your desired versions of Cyrus, tell Cassandane which install to use for the default instance and which to use for replica/murder instances, then run testrunner.pl normally. The tests themselves have no awareness that there are multiple versions being run, they just know they have a main instance and a replica (or murder) instance. (The gory details are in Cassandane/Instance.pm iirc.)

What it doesn't have support for, is for an individual test to manage the multiple versions (you are either running multiple versions for all tests, or none), or for a test to test any more than one pair of versions, or for a single instance to run two different versions of some Cyrus component. It needs you to manage the Cyrus installs, and tell it which to use for what purpose, and then it delivers those instances when a test asks for a replica or murder instance.

Any tests you write specifically to verify cross-version behaviour will, in a normal situation (i.e. with [cyrus default] only), not be testing cross-version behaviour because both instances will be from the same install. A test _can_ interrogate which Cyrus version is being used for each instance by calling Cassandane::Instance->get_version($instance->{installation}) (installation will be one of "default", "replica", "murder"... i.e. the foo in the [cyrus foo] section name from your config) if you need to assert specific behaviours between specific versions. The :min_version and :max_version magics only look at the default instance, fyi.

I can't remember what the details of the Xapian issue were, but I think to make a test for something like this, you'd make a test that sets up the world on the main instance, then replicates it and runs squatter on the replica, then asserts that the replica's state is what we expect. If you ran this with "the older version" as your default cyrus and "the newer version" as your replica cyrus, then you'd be able to see whether old version -> new version is sane (with the caveat that the test inherently needs to replicate it, so if replicating it would mask the problem, then you can't test for it like this...)

> ellie:
> * not sure if Cassandane is the right vehicle for general multiple version support
> 

The Test::Unit CPAN suite it fundamentally depends on is incomplete and abandonware, so the whole thing is already fairly precarious. I'm worried that a substantial refactor will make it unreliable, and if it's unreliable, it's useless.

Cheers,

ellie
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20190628/ac393937/attachment.html>


More information about the Cyrus-devel mailing list