Fwd: yearly release cycle

[Ken Murchison]

FYI,

This is the release process that the Cyrus IMAPd maintainers are going 
to follow moving forward.  I think this would be a good starting point 
for a discussion for a SASL release process.



-------- Forwarded Message --------
Subject: 	yearly release cycle
Date: 	Fri, 13 Dec 2019 09:59:03 -0500
From: 	Ricardo Signes <rjbs at fastmailteam.com>
To: 	cyrus-devel at lists.andrew.cmu.edu



Hey, remember last month when I asked about releasing Cyrus v3.2 
<https://lists.andrew.cmu.edu/pipermail/cyrus-devel/2019-November/004509.html>?

That thread had some more conversation about what needs to get done 
before v3.2, and I wanted to come back to it and turn some things on 
their head.

Right now, we’re talking about Cyrus releases being feature-bound. 
“We’ll release v3.2 when feature X is done.” I think we’re not being 
well-served by that. As feature X is delayed (for various reasons that 
we can’t easily eliminate), it doesn’t just delay the feature, but also 
all the other minor bugfixes and optimizations that we’ve made in the 
master branch. Also, it sets up the idea that we delay releases for the 
sake of fixes, instead of releasing the fixes that are ready.

That is: every additional criteria for a new release is another doorway 
to delay. Instead of opening those doors, I would rather try to 
eliminate all of them.

I propose that instead of tying releases to milestones, we tie them to 
the calendar. For the sake of full disclosure: I am modeling this 
suggestion on the release cycle of perl 
<https://metacpan.org/pod/perlpolicy>, which I ran for several years. I 
found the process more than satisfactory, then.

 1.

    A new /unstable release/ of Cyrus is made every month. We promise
    only that it compiled and passed the Cassandane test suite on the
    release manager’s computer. It might contain regressions from
    previous unstable releases, it might have crashers or corruptors. We
    try to avoid any of these, but the goal here is a snapshot for easy
    month-to-month testing. These are the odd-middle-digit releases. (3.3.x)

 2.

    A new /major release/ of Cyrus is made every year. We will have
    tested it on as many configurations as we can readily test. We will
    have, some time before the release, frozen the branch for risky
    changes, to reduce churn. In the meantime, new work lives in feature
    branches. (The changelogs from each unstable release provide a good
    basis for the whole-year changelog!) These are the even-middle-digit
    third-digit-zero releases. (3.4.0)

 3.

    A new /maintenance release/ of Cyrus is made for the last two stable
    releases when there are enough fixes to critical bugs to warrant it.
    These are the even-middle-digit third-digit-nonzero releases (3.4.1)

For the above to work, some more properties need to be maintained.

Maintenance releases should be no-brainers to install, so they must only 
fix regressions, crashers, security vulnerabilities, and the like. This 
means that once you’re on 3.4.0, you can always upgrade within the 3.4 
series with a minimum risk. It also means you get no optimizations, 
features, and the like.

Major releases must clearly document any incompatible changes or upgrade 
steps required. Because non-regression bugfixes aren’t backported, we 
want everyone to be able to upgrade from major release to major release, 
so incompatible changes must be kept to a minimum.

In part, this is just “don’t kill off a feature people use just because 
it’s a little annoying.” The more important one is “don’t introduce 
half-baked things that might need to change,” because people will come 
to rely on them before you get the updates finished. For features that 
will require multiple years to get right, they have to go behind a 
default-off configuration option. I’d strongly suggest they all have a 
uniform substring like “unstable”. That way, when a complaint comes in 
that the behavior of JMAP calendaring has changed, we can reply, “well, 
to use it, you had to turn on the unstable_jmap_calendaring” option.

If we go with this policy, we’ll need to…

 1.

    identify what issues are /blockers/ to v3.2.0, meaning they’re
    regressions from v3.0 and would reasonably prevent someone from
    upgrading; this does /not/ include all known bugs, since they may be
    bugs that already exist in the last stable release!

 2.

    pick a release target for v3.2.0; I will arbitrarily suggest March 2
    as “not too far off, but far off enough that we can get things in
    order”; also, if you’re American, March 2 is 3/2 ;-)

 3.

    produce a changleog, and especially identify what changes in master
    need documentation as “incompatible changes”

 4.

    produce a list of changes in master that should be put behind an
    unstable configuration option and then do it

 5.

    decide when to stop merging non-release-related things to master

 6.

    make a plan for who will do monthly snapshot releases

I’ve spoken with ellie and Bron about just a few of these, such that I 
don’t think it’s all crazy. (ellie notes, correctly, I think, that the 
first set of releases like this will be the hard ones, where we work out 
things like “how do we keep track of incompatibilities, upgrade steps, 
and also how do we make snapshots dead easy to release.”) If there’s 
general agreement, I am definitely ready to pitch in and help try to 
make it work!

—
rjbs


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/cyrus-sasl/attachments/20200218/f7585a6f/attachment.html>