From rjbs at fastmailteam.com Tue Sep 1 15:56:01 2020 From: rjbs at fastmailteam.com (Ricardo Signes) Date: Tue, 01 Sep 2020 15:56:01 -0400 Subject: meeting minutes, 2020-08-31 Message-ID: Here are the minutes of our (fairly short) call from this week. * rsto * fixed sieve/jmapquery issue with stray CR characters * ran both afl and hongfuzz fuzzers for approx. * 10 CPU days for * jansson * libical * mime parser in Cyrus (message.c) * no crashers found, so it looks like no super-low hanging fruits (good!) * issue is with getting fuzzers find relevant edges: * either brute-force with lots of CPUs and long time horizon (>=weeks) * and/or see if we can nudge edge detection further * rjbs to contact a successful fuzzer of Perl about AFL fuzzing tips * rsto: We likely need more human labor. * updated Mailbox/set to handle role updates: * all new and regression tests pass. * will start review next week after code cleanup * brong * Distracted by IETF politics * sync replication: haven?t touched it * xapian reindex: grr. Going to write to temp_path for a bunch of records and then compact to temp path again. * skip_domains: reviewed, will merge soon * added bespoke method UserCounters/get; this lets us avoid IMAP connections from middleware when sending startup event on EventSource connection * ellie * rjbs *does* owe her a list of error messages at the top of to-reformat-to-new-format * new 3.2 release! * working through some GitHub issues * qsort_r BSD compat may soon be better-solved * all the time, we?ve had a macro adjusting for argument ordering plus a function for platforms without qsort_r ? but we didn?t compile out the fallback fn, so our macros always called *that! *and then on BSD, the macros changed the order of args, but then called our other-arg-ordered internal implementation * this means all our builds will start using platform-supplied qsort_r, which could expose some surprises * ?It?s been fun.? * murch * bulk of last week: writing/rewriting code to transform quota db to use id instead of name ? and deciding it wasn?t really worth it, given the amount of code that would then also be rewritten * murch: ?I think I convinced Bron that the right thing is to leave it as-is.? * brong: ?Yup.? * some new bug in uuidmailboxes has turned up; maybe an uninitialized return(?) * JMAP Sieve draft should get an update this week * rjbs to provide some feedback [*rjbs: hey I just did that!*] * rjbs * will soon be moving Cyrus mailing lists * *gotta* get build of jmap-calendar cyrus this week! * CalDAVTalk and jscalendar: should not be a conflict, right? * brong: Right. * [ some discussion about how this affects Cassandane because of multivalue rrules ] -- Ricardo Signes (rjbs) CTO, Fastmail -------------- next part -------------- An HTML attachment was scrubbed... URL: From quanah at symas.com Tue Sep 1 16:03:44 2020 From: quanah at symas.com (Quanah Gibson-Mount) Date: Tue, 01 Sep 2020 13:03:44 -0700 Subject: meeting minutes, 2020-08-31 In-Reply-To: References: Message-ID: --On Tuesday, September 1, 2020 4:56 PM -0400 Ricardo Signes wrote: Is there any progress on the relicensing for cyrus-sasl? There are a number of pending fixes, some are important, and certainly behoove a new release, but without a working contribution process and license, we're stuck. Thanks, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: From murch at fastmail.com Wed Sep 2 07:55:26 2020 From: murch at fastmail.com (Ken Murchison) Date: Wed, 2 Sep 2020 07:55:26 -0400 Subject: meeting minutes, 2020-08-31 In-Reply-To: References: Message-ID: On 9/1/20 4:03 PM, Quanah Gibson-Mount wrote: > > > --On Tuesday, September 1, 2020 4:56 PM -0400 Ricardo Signes > wrote: > > Is there any progress on the relicensing for cyrus-sasl? There are a > number of pending fixes, some are important, and certainly behoove a > new release, but without a working contribution process and license, > we're stuck. Unfortunately, CMU is unwilling to re-license at this time.? I don't recall the reason, if any, but this is what I was told by a former colleague. So, any new contributions will have to be governed by the current license for the foreseeable future. -- Kenneth Murchison Senior Software Developer Fastmail US LLC From marc at marclaporte.com Wed Sep 2 09:17:00 2020 From: marc at marclaporte.com (marc at marclaporte.com) Date: Wed, 02 Sep 2020 16:17:00 +0300 Subject: meeting minutes, 2020-08-31 In-Reply-To: References: Message-ID: Dear Ken and Ricardo, One option is to ask new contributions to be dual licensed (current and desired). So if CMU changes their policy, it won't be necessary to ask permission to all the recent contributors. This is what we did to change the Bootstrap license (needing to get approval from hundreds of contributors) https://github.com/twbs/bootstrap/issues/2054 Best regards, Marc On 2020-09-02 14:55, Ken Murchison wrote: > On 9/1/20 4:03 PM, Quanah Gibson-Mount wrote: >> >> >> --On Tuesday, September 1, 2020 4:56 PM -0400 Ricardo Signes >> wrote: >> >> Is there any progress on the relicensing for cyrus-sasl? There are a >> number of pending fixes, some are important, and certainly behoove a >> new release, but without a working contribution process and license, >> we're stuck. > > > Unfortunately, CMU is unwilling to re-license at this time.? I don't > recall the reason, if any, but this is what I was told by a former > colleague. > > So, any new contributions will have to be governed by the current > license for the foreseeable future. From quanah at symas.com Fri Sep 4 19:01:14 2020 From: quanah at symas.com (Quanah Gibson-Mount) Date: Fri, 04 Sep 2020 16:01:14 -0700 Subject: meeting minutes, 2020-08-31 In-Reply-To: References: Message-ID: --On Wednesday, September 2, 2020 8:55 AM -0400 Ken Murchison wrote: > So, any new contributions will have to be governed by the current license > for the foreseeable future. Ok, thanks Ken. So the second part that would be useful to know is, what sort of rights assignment, etc, may need to be signed off on for a change to be accepted? There are a number of pending MRs that are rather large in scope IIRC. Also, is there a general rule of thumb to when a rights assignment may be necessary? For example, with the OpenLDAP project we have this guideline: Thanks! --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: From quanah at symas.com Wed Sep 9 21:46:27 2020 From: quanah at symas.com (Quanah Gibson-Mount) Date: Wed, 09 Sep 2020 18:46:27 -0700 Subject: meeting minutes, 2020-08-31 In-Reply-To: References: Message-ID: --On Friday, September 4, 2020 5:01 PM -0700 Quanah Gibson-Mount wrote: > > > --On Wednesday, September 2, 2020 8:55 AM -0400 Ken Murchison > wrote: > >> So, any new contributions will have to be governed by the current license >> for the foreseeable future. > > Ok, thanks Ken. > > So the second part that would be useful to know is, what sort of rights > assignment, etc, may need to be signed off on for a change to be > accepted? There are a number of pending MRs that are rather large in > scope IIRC. > > Also, is there a general rule of thumb to when a rights assignment may be > necessary? > > For example, with the OpenLDAP project we have this guideline: > Is someone able to answer as to what the process should be for accepting merges? I'd like to be able to move forward on cyrus-sasl development. :) Thanks, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: From ellie at fastmail.com Thu Sep 10 20:07:10 2020 From: ellie at fastmail.com (ellie timoney) Date: Fri, 11 Sep 2020 10:07:10 +1000 Subject: meeting minutes, 2020-08-31 In-Reply-To: References: Message-ID: Hi Quanah, > Is someone able to answer as to what the process should be for accepting > merges? I'd like to be able to move forward on cyrus-sasl development. :) I'm not really sure, but maybe this is something we can all chat about and figure something out. Are you able to join one of our weekly conference calls? The zoom link is here: https://www.cyrusimap.org/imap/support/feedback-meetings.html Not sure what timezone you're in, but we usually flip to the other end of the day around mid-late October (once AU/US switch their respective DST's). I'm not currently sure what date we'll be switching over, but it's not far away now. So maybe if the current meeting time is unsuitable for you, the AU summer/US winter time might be more workable? Cheers, ellie From rjbs at fastmailteam.com Mon Sep 14 09:44:30 2020 From: rjbs at fastmailteam.com (Ricardo Signes) Date: Mon, 14 Sep 2020 09:44:30 -0400 Subject: meeting minutes, 2020-09-14 Message-ID: <140e3d6b-4b99-401c-b76b-7c73235a9b80@dogfood.fastmail.com> 2020-09-14 * ellie * xsyslog is now merged! * please get into a habit of using it in new code. there?s some examples in b9191eeb09 * if fm ops notices specific log outputs they want fixed they can throw them at ellie * will continue to chip away at converting existing ones as time/motivation permits * mbname_from_foo() constructor APIs (esp mbname_from_userid()) should work harder to only produce valid mbname objects? mbname_from_intname() and mbname_from_extname() look fairly thorough, but mbname_from_userid() is especially naive, causing #3169. i haven?t looked at the others (if there are others). * Australia will be back in DST again in a few weeks, so we should start thinking about when to flip the meeting time to the other end of the day * added a new makefile target to run each cunit test individually (instead of along with all others) and found 55 tests that don?t run on their own, mostly due to the same leaky abstraction * imapd.conf contents were being generated, but not every test that relied on those did the generation, so they passed because an earlier test did * some mbox name tests remain to fix, though (?) * custom annotation definitions: are specified case-insensitive, but we load them case-sensitively ? would be easy to fix, but a comment suggests that our DAV annotations are case-sensitive. Q: Do we need *custom* DAV annotations? * brong * working on compaction stuff * first pass of synchronous replication is done! needs testing, but maybe this week can go onto unstable testing stores * MR called ?debug syslog lock ordering?, keeps ptr atr of open locks; the bookkeeping has been a mess because a bunch of code just closes the fd, not an explicit lock destroy * [ some discussion of how we can improve this and track locks better ] * fix search code path to better eliminate items in non-mail folders * rsto * AFL has detected an httpd crasher! \o/ * ran fuzzer for 11d against the httpd, didn?t find any other crashers * may turn fuzzer back to other components, or maybe update its dictionary for better-targeted attack * JMAP calendars work still happening on the side * ?I?d like to return to the coverage report.? * have found a few places that are clearly under-tested * the rest of us should give it a look to see what else looks insufficiently tested * also: can we run an instrumented-for-code-path-entry Cyrus on a used-but-not-for-customers branch to gather more real-world usage? Answer: yes, probably, but there are some owner/mode shenanigans to make it all work * murch * spent a bunch of time on JMAP Sieve script code, mostly the testing code * there have been some memory management issues with ownership of strings -- Ricardo Signes (rjbs) CTO, Fastmail -------------- next part -------------- An HTML attachment was scrubbed... URL: From quanah at symas.com Mon Sep 14 11:10:04 2020 From: quanah at symas.com (Quanah Gibson-Mount) Date: Mon, 14 Sep 2020 08:10:04 -0700 Subject: meeting minutes, 2020-08-31 In-Reply-To: References: Message-ID: <91CA1513CEE24019FB29AD0C@[192.168.1.156]> --On Friday, September 11, 2020 11:07 AM +1000 ellie timoney wrote: > Hi Quanah, > >> Is someone able to answer as to what the process should be for accepting >> merges? I'd like to be able to move forward on cyrus-sasl development. >> :) > > I'm not really sure, but maybe this is something we can all chat about > and figure something out. Are you able to join one of our weekly > conference calls? The zoom link is here: > https://www.cyrusimap.org/imap/support/feedback-meetings.html > > Not sure what timezone you're in, but we usually flip to the other end of > the day around mid-late October (once AU/US switch their respective > DST's). I'm not currently sure what date we'll be switching over, but > it's not far away now. So maybe if the current meeting time is > unsuitable for you, the AU summer/US winter time might be more workable? Hi Ellie, I'm on the US west coast, so Pacific time. It looks like 11 UTC is 4 AM, which isn't a time I can easily make. ;) So the other time may be workable, depending on what it is. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: From rjbs at fastmailteam.com Mon Sep 14 11:19:08 2020 From: rjbs at fastmailteam.com (Ricardo Signes) Date: Mon, 14 Sep 2020 11:19:08 -0400 Subject: meeting minutes, 2020-09-14 In-Reply-To: <140e3d6b-4b99-401c-b76b-7c73235a9b80@dogfood.fastmail.com> References: <140e3d6b-4b99-401c-b76b-7c73235a9b80@dogfood.fastmail.com> Message-ID: On Mon, Sep 14, 2020, at 09:44, Ricardo Signes wrote: > 2020-09-14 > * rsto > * AFL has detected an httpd crasher! \o/ My mistake, this was found by honggfuzz. -- rjbs -------------- next part -------------- An HTML attachment was scrubbed... URL: From rjbs at fastmailteam.com Mon Sep 28 07:17:08 2020 From: rjbs at fastmailteam.com (Ricardo Signes) Date: Mon, 28 Sep 2020 07:17:08 -0400 Subject: meeting minutes, 2020-09-28 Message-ID: <4f7a0667-2451-4b3f-9f81-c1e25486ab98@dogfood.fastmail.com> Very short call today! 2020-09-28 * bron * in the process of opening a PR to always run squatter in batch mode whether rolling or not * previously, without doing indexing in chunks, users could get stuck on indexing all mail and back up replication! * possibly we?re seeing squatter failing to index some messages; have had >0 reports of ?search has no results?; one possibility, is that we?ve got a bug in the http-based append code, so it?s affecting dav and jmap moves? (no synclog append/user?) * got a PR in flight to reduce how often we do a crc32, to avoid CPU load * murch * last week mostly jmap sieve (implemented query!) * Thu/Fri: memory leak squashing! * handling of STATUS:CANCELLED itip was busted and has been fixed; this affected both itip generation and itip reply processing [*rjbs: *this was a fun one] * have added a new argument to Backup/restore* to crank up log level per-call to see why a restore might be failing * ellie * been doing housekeeping work * next v3.2 release notes are ready, just pending a last run-through of low-hanging pending issues to fix * rjbs * not much Cyrus to report! * expect some planning of upcoming 2021 projects soon -- rjbs -------------- next part -------------- An HTML attachment was scrubbed... URL: