Meeting minutes 15 Feb [was Re: Meeting minutes 8 Feb]

[Nicola Nye]

Great summary Ellie. I've been digesting your mail all day.

My gut feeling is that submodules are going to get in our way. We don't
have a big team of regular developers on either the docs or the source
projects and we want to remove process hurdles where we can for those
who contribute. (Regular use of the term "sobmodules" makes me
risk-averse! Much like master documents in Word)

The docs history is extensive and we don't want to lose it.

It would be great to pair source updates with their corresponding doc
changes (encourage developers to help maintain the docs, especially for
man pages), so hiding the docs in a submodule/subtree is potentially
counterproductive.

Being able to keep the docs and source together when branching would be
awesome.

Is there any down side to importing-with-history the cyrus-docs tree
into cyrus-imap somewhere? (subdirectory) Yes, it means docs people have
write-access into the source tree, but the only people likely to be
writing cyrus documentation without needing arcanist approval are those
who know what they're doing.

FWIW, I work on the FastMail codebase committing docs alongside the
source and I haven't experienced any pain and suffering having a
combined repo, from either buildsystem tools or managing documentation
commits.

   Nicola

On Tue, Feb 16, 2016, at 10:16 AM, ellie timoney via Cyrus-devel wrote:
> > Ellie,
> > Could you expand on the "complicated questions"?
> 
> I can try -- to a certain extent they're complicated cause I'm not
> entirely sure how to express them to myself yet either.
> 
> It's mostly around conflicting goals and finding the least bad
> compromise, I guess.
> 
> I think the main useful things we get out of the current segregated
> approach are:
> 
>   * granularity of commit access
>   * distinct build tools for distinct tasks
>   * existing/known workflows
>   * uncluttered commit histories
> (sing out if there's more, I don't work on the cyrus-docs repo heavily)
> 
> And main pain points we get from it are:
> 
>   * lack of relationship between documentation and code (which version
>   of the code does a version of documentation apply to? did this code
>   change have a corresponding documentation change? was this
>   documentation change fixing a bug in documentation, or tracking a
>   change in behaviour? etc)
>   * the need to include documentation (man pages, install docs, release
>   notes, READMEs, etc) in release distributions that is, increasingly,
>   authoritatively managed in the cyrus-docs repository
>   * the need to include documentation (some install docs) on the website
>   that is authoritatively managed in the cyrus-imapd repository
>   * the need to include documentation (some man pages) on the website
>   that is auto-generated from source in the cyrus-imapd repository
> (anything else?)
> 
> And I guess what we want from combining the two repositories is:
> 
>   * resolution of some/all of the pain points
>   * minimal introduction of new pain points
>   * hopefully some way to keep some of the useful aspects
> 
> I did a little research yesterday into some of our options for combining
> the two repositories into one.  There's a few:
> 
> 1) just add the current cyrus-docs files to the cyrus-imapd repository
> 2) use a subtree merge to bring cyrus-docs and its history into
> cyrus-imapd
> 3) keep the cyrus-docs repository separate, but add it to the
> cyrus-imapd repository as a submodule
> 4) something similar to a subtree merge, but manually
> (probably something else?)
> 
> Note that no matter which approach we take here, there'll be a
> non-trivial piece of work in integrating the build systems.  I'm pretty
> comfortable that I can manage this -- I seem to have spent a fair bit of
> time wrangling our autoconf setup so far -- it'll just be fiddly, and
> therefore not quickly finished.
> 
> With option 1 we would lose the commit history of cyrus-docs; I'm
> assuming that's unacceptable.  Otherwise, it's the least complicated
> option.
> 
> With option 2 we keep the commit history of cyrus-docs, with some
> caveats: as I understand it, tracing the history is difficult/fiddly
> (but not impossible)
> 
> With option 3, we keep most of the benefits of the segregated
> repositories.  I'm led to believe submodules are
> kind of a pain to work with, but I suspect that can be encapsulated with
> some custom tooling + maintainer-mode-only autoconf rules, and will
> hopefully not bleed over into normal development/documentation
> workflows.  We wouldn't really gain a clear relationship between doc
> revisions/code revisions, but we also don't have that now.
> 
> With option 4, we might be able to get a clearer/more coherent history
> than with a subtree merge, but we also might not.  I picture this
> involving an index-filter to move the cyrus-docs files into a unique
> subdirectory, plus some remote juggling and a big merge.  I'm not
> entirely clear how this differs from a subtree merge, but some stack
> overflow discussions sort of implied that it does and that under some
> circumstances this was preferable.
> 
> There's more research required (like, actually trying out the different
> merge types to see what happens in practice) before I could conclusively
> recommend one way or another.  I haven't done this sort of thing before,
> has anyone else?
> 
> Part of me is leaning towards option 3, except that like 80% of what
> I've ever heard about submodules could be summarised as "considered
> harmful", and I have no experience with them myself to counterbalance
> that.
> 
> But the really fiddly thing here is always going to be the fact that we
> need to build most of the man pages from rst files, but we need to build
> some of them (and thus their corresponding web pages...) from imapd
> source files.  Taken broadly, i.e. treating "man pages" as a homogenous
> collection, that's a cyclical dependency.  Squishing it all together
> (options 1, 2, 4) makes that relatively easy to resolve just with
> autoconf, but option 3 will need tooling/process work.
> 
> Cheers,
> 
> ellie
> 
> On Tue, Feb 16, 2016, at 01:15 AM, Nic Bernstein via Cyrus-devel wrote:
> > On 02/15/2016 05:34 AM, Nicola Nye via Cyrus-devel wrote:
> > > Nicola
> > > <...>- Next task: look at integrating the docs repo into the source 
> > > repo so that we can single-source things like man pages into man files 
> > > for release packaging as well as html for the website.
> > >
> > > Ellie
> > > - Been thinking about implications of merging docs/source repos, Has 
> > > uncovered some complicated questions with no clear answers yet.
> > 
> > Ellie,
> > Could you expand on the "complicated questions"?
> > 
> > I spent some time last week working my way through the mailing list 
> > traffic from the past several months, and getting my head back into 
> > Cyrus space.  The new stuff on cyrusimap.org looks good.  Now I'm trying 
> > to figure out how and where I can jump back in to this whole squirrelly 
> > docs project.
> > 
> > Yours,
> >      -nic
> > 
> > -- 
> > Nic Bernstein                             nic at onlight.com
> > Onlight Inc.                              www.onlight.com
> > 6525 W Bluemound Rd., Ste 24              v. 414.272.4477
> > Milwaukee, Wisconsin  53213-4073          f. 414.290.0335
> >