/
Elasticsearch Ownership and Rollout

Elasticsearch Ownership and Rollout

Context

See [BD 19-20] Technical Plan (Haystack Replacement, Elasticsearch Upgrade)

Runbook

ES7 deployment strategy

Effort Coordination

This blended project upgrades our outdated and unsupported Elasticsearch from version 1.5 to version 7 in each of our services. We will need a resource from each owning squad to do a final review and deploy of the Elasticsearch upgrade for their owning service. The expected effort is:

  • Squad engineer’s review of PR: 2-5 hours depending on the complexity of the service’s use of Elasticsearch (PR will already be reviewed carefully by @Diana Huang or @Michael Terry (Deactivated))

  • Squad engineer’s deploy and monitoring of the migration: 2-3 days depending on production rollout

Service/Repo

Owning Team

PRs

Current Blocker

Status

Service/Repo

Owning Team

PRs

Current Blocker

Status

terraform

Arch-BOM

 

@Tim McCormack working on updating ES clusters to ES 7.8.

devstack

SRE / Arch-BOM

https://github.com/edx/devstack/pull/611

https://github.com/edx/devstack/pull/607

 

  • notes

  • comments-service

  • edx-platform

  • course-discovery

  • analytics

edx-notes-api

SRE
#site-reliability-engineering

Contact:@Fred Smith (Deactivated)
ARCH-BOM: @Diana Huang

[BD-20] Haystack Replacement and Elasticsearch upgrade for edx notes api by NikolayBorovenskiy · Pull Request #181 · openedx/edx-notes-api

https://github.com/edx/edx-notes-api/pull/200

indexing on prod caused errors due to malformed tags - checking to see if we can/should fix this.

  • PR review

  • PR merge

  • Deploy to Stage

  • Deploy to Edge

  • Deploy to Prod (target week: Sep 21st)

course-discovery

Engage (Always Available)
#discovery

Contact: @Michael Terry (Deactivated) ARCH-BOM: @Feanil Patel (Deactivated)

Waiting for new PR with original code + fixes.

Fixes code: https://github.com/raccoongang/course-discovery/pull/13

  • RG working with @Feanil Patel (Deactivated) and @Michael Terry (Deactivated) to resolve search query discrepancies.

  • PR review

  • Manual test

  • PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (week: Jan 4th)

edx-search

TNL
#tnl-squad

Contact: @sburch (Deactivated)
ARCH-BOM: @Tim McCormack

https://github.com/edx/edx-search/pull/104

https://github.com/edx/edx-platform/pull/25081

 

 

  • PR review

  • Test confidence

  • edx-search PR merge

  • edx-platform PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (Actual week: Nov 16)

cs_comments_service

TNL
#tnl-squad

Contact: @David Joy (Deactivated)
ARCH-BOM: @Robert Raposa

https://github.com/edx/cs_comments_service/pull/327

 

  • PR review

  • Test confidence

  • PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (target week: Oct 27th)

edx-analytics-data-api
edx-analytics-pipeline

Data Engineering
#data-engineering

Contact: @Stuart Young (Deactivated)
ARCH-BOM: @Diana Huang

https://github.com/edx/edx-analytics-data-api/pull/389

https://github.com/edx/edx-analytics-pipeline/pull/835

@Stuart Young (Deactivated) review and testing

  • PR review

  • Manual test

  • PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (target week: Nov)

 

Related content

Working Groups
Working Groups
Read with this
Elasticsearch 7.10 Upgrade Tracking
Elasticsearch 7.10 Upgrade Tracking
More like this
ES7 deployment strategy
ES7 deployment strategy
More like this
Elasticsearch Upgrade Task Force
Elasticsearch Upgrade Task Force
More like this
[BD-19, BD-20] Elasticsearch & Haystack Upgrades
[BD-19, BD-20] Elasticsearch & Haystack Upgrades
More like this
Elasticsearch 7.10
More like this