From rjbs at fastmailteam.com Mon May 11 10:48:40 2020 From: rjbs at fastmailteam.com (Ricardo Signes) Date: Mon, 11 May 2020 10:48:40 -0400 Subject: minutes, Cyrus call 2020-05-11 Message-ID: <8a2f1da5-bd2a-4748-8615-bd21d9bf7e4a@dogfood.fastmail.com> I think it's been since July that we've not posted any minutes from the regular developer's Zoom call. This was basically fallout from some rescheduling and tool changes, but it's still not great. Sorry about that! Let's get this going again. Today (2020-05-11 ??) in attendance: Bron, Ellie, Ken, Rik, Robert * ellie * Travis has been testing the wrong branches ? generally always master instead of version or feature branches * Also, Cassandane make failures were unreported. * Partha did a ton of work and this should be fixed. * Travis is mostly just failing on Sieve.snooze_tzid - might add that to the disabled tests list. * Getting lots of bug reports for 3.2: all the things not tested during the RC/beta period * Marco found that syslog interception doesn?t work on Redhat system. Would be good to fix it for future Cassandane users. * Oh yeah, 3.2 is out! Only 2 months late. 3.4 for March 2021. Feature freeze end of 2020. * Robert * Big features like attachment indexing - always pushed on master. e.g. if 3.2 users do attachment indexing then we have to tell them to reindex everything when they get to 3.4. * The plan going forward will be to keep big features on long-lived feature branches while they're developed, and only in master when proven out. (This will often mean "has run on Fastmail for a long while under real user load".) * The new build and deploy systems will make this much easier to manage. * Basic plan is to include branches with particular labels in the final build. This will be similar to the current Fastmail branch building stuff. The Fastmail builder will also be able to look at our private gitlab if we want to build anything in secret. * Was looking into a possible split-conversations bug, has been transferred to Bron. * Hoping to get JSCalendar done by June, so have been spending time on that. * Went down a rabbit hole of looking at how languages are stored in Xapian. Now much simpler. * Implemented a way to recalculate xapian language counts exactly. Can always recalculate language counts. * Close to getting to code review for attachment indexing. * Now API can set ?index level? on Xapian document - can allow for partial indexing. Bron suggests adding headers to the level data too. * Ken * Hopefully this week subscriptions for mailbox storage by UUID get finished * Added JMAP Backup tweaks that Fastmail requested - adding options to restore or not restore drafts. * Sieve work! * Added timezone argument to Sieve snooze - not sure if it?s been tested other than in Cassandane. [ed.: not yet! Fastmail Sieve generator needs an update first] * Did more hacking around Sieve! Rewrote bytecode parser to use a lookup table. * Broke and then fixed vacation fcc. * Started deprecating old non-standard sieve stuff. (mark/unmark, etc) * We?re still using legacy notify - hopefully we can switch to enotify and start deprecating it. Rik will talk to Ken about transition plan for Fastmail. * Been working on making a sieve decompiler that can convert bytecode into scripts again. * Might be hard to tell the difference between ?elseif? and ?else { if {} }?. * But it doesn?t need to be exact, just readable to see what was there. * Bron: * Wondering if we can ?autodetect? that a user wants header indexing by seeing the queries they run and turn it on / flag them for reindex. * Talking to CMU about getting DNS transferred to Fastmail server where we can do better website hosting setup. * Rik * ideally, this week we will finish and begin using new-style Cyrus builds, which will mean some changes to the policy about "what goes on master" ? will email when we get there * other than that, all Cyrus work has just been doing routine deploys -- Ricardo Signes (rjbs) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rjbs at fastmailteam.com Mon May 18 20:25:35 2020 From: rjbs at fastmailteam.com (Ricardo Signes) Date: Mon, 18 May 2020 20:25:35 -0400 Subject: minutes, Cyrus call 2020-05-18 Message-ID: <760fd4dd-313c-4eb9-a71c-7fbf157c5b7f@dogfood.fastmail.com> Today (2020-05-18 ??) in attendance: Bron, Ellie, Ken, Rik, Robert Ken * Doing some work on the Backup/restore* APIs * Draft restore performance has been poor, because we?re tracking by message-id * We think we can get a reasonable approximation by looking in the same thread. * Ken still needs to do restoration of parent mailboxes. Mailboxes don't currently have the mailboxId of their parent, so when restored can't detect a rename of their former parent. We could make mailboxes record retain parentid in UUID mailboxes, but it?s not there now. Maybe after the new layout code lands? Robert * The last week was spent mostly on Xapian work * Priority moved from working on finishing work on attachment indexing work toward making it possible to perform a reindex of all users * Removed experimental Xapian features from the main branch. * Fixed small bugs with Quota. * Issue with Xapian - right now the in-memory backend is slow - better solution is index to disk at temp_path and make that a tmpfs. Need to instruct people that it should be on a tmpfs. * We discussed whether to do a separate config option or to just use temp_path. * [ rjbs: I believe the conclusion was that one option is *probably* fine, but I'm not sure it was firmly concluded ] ellie * Mostly been working through fixing the bugs from 3.2.0. Hoping to do 3.2.1 later this week or early next week. Bron * not a whole lot this week * made http_ws do session_new_id() between calls * Might be adding some pushes Rik * also pretty light; hoping to send out notes on proposed new merge policy for master branch soon -- rjbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From rjbs at fastmailteam.com Thu May 28 09:35:58 2020 From: rjbs at fastmailteam.com (Ricardo Signes) Date: Thu, 28 May 2020 09:35:58 -0400 Subject: master branch: merging less Message-ID: <9f69fefa-5f6f-4a26-880b-693901b7e7ca@dogfood.fastmail.com> Cyrus developers, Right now, the policy on what goes in master is "when a committer thinks it's ready for master, it goes in master." This hasn't been a serious problem, but we can do better. What I'd like to do is have feature branches live longer *as* branches, and then be merged when we think they're done. This will make it easier to declare that a feature is really in Cyrus (even if it's only in a 3.${odd} release), and to produce builds with features turned off and on. It will also make it easy to drop an experiment without scouring history much. Finally, it should make code review better, as it can be part of the "can we merge this?" process and when it happens, the whole feature changeset can be seen at once. There isn't much actual policy change to talk about. Something like this: > Changes to Cyrus are made in branches. Branches aren't merged until the feature seems plausibly done. When a feature is still undergoing testing, it's left in a branch. Pull requests are opened on GitHub, and before the branch is merged, code review is completed by another committer to the one who wrote the change. Branches should, whenever practical, be rebased before merging. Trivial bugfixes and changes to documentation may be applied directly to master, but when in doubt, favor making a branch! This will go somewhere in our "Contribute code and tests" section, but right now there seems to be no discussion of policy after the creation of a PR. At present, Fastmail tends to run very close to master, and we're often testing features before they're in a state we'd consider mergeable under this policy. To make it easy to build a Cyrus that includes all the latest bugfixes and approved experimental branches, we built a tool to merge all the pull requests we've flagged for inclusion . You might also find this tool useful for building your own test Cyruses. -- rjbs -------------- next part -------------- An HTML attachment was scrubbed... URL: