Elasticsearch Ownership and Rollout

Context

See https://openedx.atlassian.net/wiki/spaces/COMM/pages/1413810945

Runbook

https://openedx.atlassian.net/wiki/spaces/COMM/pages/1741455546

Effort Coordination

This blended project upgrades our outdated and unsupported Elasticsearch from version 1.5 to version 7 in each of our services. We will need a resource from each owning squad to do a final review and deploy of the Elasticsearch upgrade for their owning service. The expected effort is:

  • Squad engineer’s review of PR: 2-5 hours depending on the complexity of the service’s use of Elasticsearch (PR will already be reviewed carefully by @Diana Huang or @Michael Terry (Deactivated))

  • Squad engineer’s deploy and monitoring of the migration: 2-3 days depending on production rollout

Service/Repo

Owning Team

PRs

Current Blocker

Status

Service/Repo

Owning Team

PRs

Current Blocker

Status

terraform

Arch-BOM

 

@Tim McCormack working on updating ES clusters to ES 7.8.

devstack

SRE / Arch-BOM

https://github.com/edx/devstack/pull/611

https://github.com/edx/devstack/pull/607

 

  • notes

  • comments-service

  • edx-platform

  • course-discovery

  • analytics

edx-notes-api

SRE
#site-reliability-engineering

Contact:@Fred Smith (Deactivated)
ARCH-BOM: @Diana Huang

https://github.com/edx/edx-notes-api/pull/181

https://github.com/edx/edx-notes-api/pull/200

indexing on prod caused errors due to malformed tags - checking to see if we can/should fix this.

  • PR review

  • PR merge

  • Deploy to Stage

  • Deploy to Edge

  • Deploy to Prod (target week: Sep 21st)

course-discovery

Engage (Always Available)
#discovery

Contact: @Michael Terry (Deactivated) ARCH-BOM: @Feanil Patel (Do Not Use) (Deactivated)

Waiting for new PR with original code + fixes.

Fixes code: https://github.com/raccoongang/course-discovery/pull/13

  • RG working with @Feanil Patel (Do Not Use) (Deactivated) and @Michael Terry (Deactivated) to resolve search query discrepancies.

  • PR review

  • Manual test

  • PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (week: Jan 4th)

edx-search

TNL
#tnl-squad

Contact: @sburch
ARCH-BOM: @Tim McCormack

https://github.com/edx/edx-search/pull/104

https://github.com/edx/edx-platform/pull/25081

 

 

  • PR review

  • Test confidence

  • edx-search PR merge

  • edx-platform PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (Actual week: Nov 16)

cs_comments_service

TNL
#tnl-squad

Contact: @David Joy
ARCH-BOM: @Robert Raposa

https://github.com/edx/cs_comments_service/pull/327

 

  • PR review

  • Test confidence

  • PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (target week: Oct 27th)

edx-analytics-data-api
edx-analytics-pipeline

Data Engineering
#data-engineering

Contact: @Stuart Young (Deactivated)
ARCH-BOM: @Diana Huang

https://github.com/edx/edx-analytics-data-api/pull/389

https://github.com/edx/edx-analytics-pipeline/pull/835

@Stuart Young (Deactivated) review and testing

  • PR review

  • Manual test

  • PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (target week: Nov)