Elasticsearch 7.10

What is this about?

• Briefly describe service and its role and impact on our systems.

• Who owns it within edX?

• What do we plan to do with it in a nutshell?

• Is this a small, medium or large project (ballpark)? 

• Any other relevant context to help understand importance, size or impact.

Why are we doing this?

• Why does this need to be done? What is the impact of not doing this?

• Could we be better off not doing it, postponing, retiring the service, or replacing it with another?

• Why did we pick this version to upgrade to?

• Why does this need to be done now?

• Can someone else do this for us?

Have we done this before?

• Have we done this before for in edX.org instance or for any other Open edX instance?

• Did the last time go well?

• Do we want to do things differently this time, and how?

• Provide links to wiki pages of similar previous projects

What do we plan to do?

• What is the scope of this project?

• On a high level, what is our plan? Identify big milestones.

• Who do we plan to involve (groups of stakeholders)?

Are there risks and dependencies?

• Are there any known risks?

• How can we mitigate them?

• Have we reviewed system requirements (e.g. AWS, Python, etc.) for this upgrade?

• Are there dependencies with other projects?

• Are there any third-party integrations that might be affected by this upgrade? What other systems or services rely on this service?

What is the impact of this upgrade on various users of the system?

• Which user groups or personas will this upgrade impact? E.g. developers, learners, educators, site operators, etc.

• Are there any new features?

• Are there any features or functionality that are being removed as a result of the upgrade?

How will we do it?

• Path to production

  • upgrade devstack and sandboxes

  • upgrade stage (terraform)

  • upgrade prod/edge (terraform)

• Rollout plan

  • see Elasticsearch 7.10 Upgrade Tracking

• Will there be any downtime?

  • no

• What are 

  • ??

• Do end users of the edX website need to be notified? Which users of the service need to be notified at edX or externally?

  • end users? no (and hopefully they don’t notice)

  • teams? yes. for stage/edge/prod we’ll need to coordinate with owning teams. see Elasticsearch 7.10 Upgrade Tracking

How will we communicate about it?

• How will communication take place (internally and externally)?

• What mode of communication will be used? E.g. regular or private Slack channel, will it be internal to edX or shared with Open edX space, email, regular meetings.

• How often will routine communication take place? Is there a need for regularly touching base?

edX wants to upgrade Elasticsearch to version 7.10 in time for Maple Open edX release.

Diane has a lot of knowledge about the previous Elasticsearch upgrade - Elasticsearch Ownership and Rollout. It was a Blended project with Racoon Gang.

This time we expect owning squads to do the upgrade, after which SRE will flip the switch.

Scope

Upgrade all IDAs that are on 7.8, see list at Elasticsearch 7.10 Upgrade Tracking.

All of these IDAs are part of the Open edX named release.

All tickets for this project will have ElasticSearch-7-10-upgrade label in JIRA.

SRE work will also be tracked in https://openedx.atlassian.net/browse/ISRE-1006.

Environments and path: devstack -> sandbox -> stage -> prod/edge (we only have one mongo cluster per env).

Dependencies

Elasticsearch upgrade can be done independently from other upgrade projects, e.g. MongDB, Django, etc.

Each upgraded IDA can be release independently, as soon as it is done.

Timeline

Though, we need to complete this anytime before October 15, we aim at finishing and upgrade prod by September 7.

However, since more services are used by Elasticsearch than by MongoDB, we want to get MongoDB upgrade out of the way first, then do Elasticsearch.

More detailed timeline to follow.

Project Team

Next steps

DONE