2020-12-21 BTR Koa Retrospective

Attendees: @Régis B. @Ned Batchelder (Deactivated) @Sofiane Bebert @Pierre Mailhot @Former user (Deleted) @Maksim Sokolskiy @Demid @Peter Pinch @Tobias Macey

(Started from threads here: https://discuss.openedx.org/t/dec-21-koa-release-retrospective-meetup/3852)

Successes and achievements

• We shipped on time

  • Felipe: "It was our target, it was predictable and it was not easy, but we said we would do it and we did."

  • Ned: "We had active participation at face-to-face meetings and online."

  • Peter: "The community chose a date that worked for them, and we worked together to finish the work by that date."

  • Sofiane: "I liked that we were able to deliver on time."

• Community engagement

  • Tobias: "I liked the fact that there was broad community engagement in identifying and working through the issues that were of concern to operators of the Open edX platform. That provided a helpful amount of visibility into what to expect going into the Koa release and working through issues before they were made permanent."

  • Pierre: "Collaboration is key."

  • Ned:

    • "We had active participation at face-to-face meetings and online."

    • "More decisions were handled by the group in Koa than in Juniper."

  • Sofiane: "I liked that members of the group were available to share their knowledge and experience."

  • Régis: "Community involvement was very impressive, even more so than for Juniper"

• We fixed bugs and shipped features

  • Peter: "identifying issues that needed to be fixed, identifying people to fix them and also identifying issues that would not be fixed."

  • Sofiane: "Personally, I was able to contribute to a couple tickets and submit my first PRs."

  • MFEs are now a reality

    • Felipe: "It was announced as being part of juniper but in practice it was still lacking a lot of work that was only merged by koa."

    • Ned: "MFE deployment is much further along."

  • Régis: "I was very happy to resolve some very thorny bugs, in collaboration with the rest of the group"

Why did you participate in the release?

• Keeping up with Open edX releases is part of my professional requirements

  • Tobias: "I participated in the release because at MIT Open Learning we are responsible for managing multiple installations of the platform. We also work to stay up to date with releases as they come available."

  • Pierre: "We like to keep up with current releases. We also do not want to be left behind and play catch up with new releases."

  • Sofiane: "Allow ourselves to be as close as possible to the latest release to benefit from the latest features of the platform."

  • Régis: "It's important for Tutor users to know that the project keeps up closely with the upstream releases"

• I enjoy contributing to the Open edX community

  • Pierre: "It’s mostly our contribution to the Open edX community.  It also allows us to help people in Slack and Discourse when the new version is released and people have questions or problems."

• To release Koa on time

  • Felipe: "I wanted to help make the release cadence something we can trust."

  • Peter: "MIT Open Learning has a relatively narrow window for updates twice a year. We really wanted to see this update get released in this timeframe."

• To make Open edX releases better

  • Ned: "To help produce the best community release we could."

• To learn more about Open edX

  • Sofiane: "Learn from the community the ins and outs of running and developing open edX."

• Because Ned asked me

  • Régis: "Ned asked me to create a working group and I like Ned so I said OK"

What didn’t we do so well, and what could we have done to do better?

• The working group does not have clear guidelines for its members

  • Tobias: "I think that one thing that we can do going forward which will help as more people get involved is to draft up a set of expectations or guidelines for involvement in the working group."

  • Sofiane: "There was a period when testing the upgrade to ubuntu where it was hard to contribute until progress was shared in the configuration repo. For initiatives that require the input from a few people, we should start with a common way of sharing work/progress/fixes."

  • Max Sokolski: "Have a feeling that newcomers can be a bit confused about the process (it’s not bad but maybe should be documented)"

  • Discussion:

    • "Only native install" this needs to be clearer up-front.

    • Where is task tracking?

      • Board isn't public, needs an account

      • How to coordinate multiple efforts

    • How to join the group

      • Who is in the group

    • What are the goals of the groups, and the boundaries of those concerns

    • Action: add to the wiki page: Build-Test-Release (BTR) Working Group

      • How to subscribe to the BTR discourse category

    • Action: decide on the process to become a member of the WG (this might not be a binary process)

    • Action: describe how to take on a ticket

• The communication loop is too slow

  • Ned:

    • "I felt like activity happened right after the face-to-face meetings, but it was harder to make things happen at other times. "

    • "Especially near the end of the cycle, I felt like I needed to do things quickly and so couldn’t be as collaborative as I would prefer. "

  • Peter: "Meeting face to face is hard, and it’s disappointing that we needed to meet face to face to keep work on track."

  • Sofiane: "When someone takes on a ticket, they should commit to a date to provide an update on the ticket (personally something that I need to do better). Otherwise, other priorities can always creep up."

  • Discussion:

    • We work in forums and Jira

    • Regis pinged people when needed, and got good results

      • Set a norm that pinging is OK?

    • Explicit dates on tickets would make clear when something is late vs when someone is just being impatient

    • We fixed all the bugs, but there was a big rush at the end

    • Ticket assignment happens face-to-face

      • People were great about stepping up to take tickets

      • But it could sit for two weeks before getting assigned

      • Can we assign tickets without being face-to-face

      • Sofiane liked getting the details of the ticket ("this is good for a first ticket")

        • More use of labels could help

      • Use Discourse more?

        • A thread for Jira tickets needing attention (Pin the thread)

      • How to highlight contributions?

        • Mozilla's newletter: https://mozillagfx.wordpress.com/

        • Blog or discourse?

          • What about Slack? Good way to celebrate achievements

• We did not do enough testing

  • Felipe: "From the moment we cut the beta branch to the moment we released I think not enough testing happened. I know at least we did not do enough testing in those dates and we need to get better at it."

  • Sofiane: "Testing the platform as a whole felt a bit too “exploratory”. For someone that’s new, it would be useful to have a starting point or some type of guidelines."

  • Discussion:

    • It would be good to know if something is part of the release (ORA changes)

      • Security change hours after the Koa release

        • "Much of edX is oblivious to the release cycle"

    • CI/CD for the future

    • Testing strategy:

      • Categories of what should be tested

      • Maybe platform map could help?

      • New people can be assigned a subsystem to test

      • Give people sandboxes for testing

• There are too few people in the working group

  • Pierre: "There are still a lot of people / organizations who have been part of the Open edX community for a long time that do not delegate anyone or participate in the BTR meetings."

  • Discussion:

    • Hypothesis: only bleeding-edge adopters will join; others are on older releases.

    • How to get more people?

      • Make it easier for people to understand how to be a part (guidelines)

      • Regis is working on badges

• We are not able to make large impactful changes as a working group (yet)

  • Régis: "There is no incentive to make large contributions to Open edX"