/
Elasticsearch Ownership and Rollout

Elasticsearch Ownership and Rollout

Context

See [BD 19-20] Technical Plan (Haystack Replacement, Elasticsearch Upgrade)

Runbook

ES7 deployment strategy

Effort Coordination

This blended project upgrades our outdated and unsupported Elasticsearch from version 1.5 to version 7 in each of our services. We will need a resource from each owning squad to do a final review and deploy of the Elasticsearch upgrade for their owning service. The expected effort is:

  • Squad engineer’s review of PR: 2-5 hours depending on the complexity of the service’s use of Elasticsearch (PR will already be reviewed carefully by @Diana Huang or @Michael Terry (Deactivated))

  • Squad engineer’s deploy and monitoring of the migration: 2-3 days depending on production rollout

Service/Repo

Owning Team

PRs

Current Blocker

Status

Service/Repo

Owning Team

PRs

Current Blocker

Status

terraform

Arch-BOM

 

@Tim McCormack working on updating ES clusters to ES 7.8.

devstack

SRE / Arch-BOM

https://github.com/edx/devstack/pull/611

https://github.com/edx/devstack/pull/607

 

  • notes

  • comments-service

  • edx-platform

  • course-discovery

  • analytics

edx-notes-api

SRE
#site-reliability-engineering

Contact:@Fred Smith (Deactivated)
ARCH-BOM: @Diana Huang

[BD-20] Haystack Replacement and Elasticsearch upgrade for edx notes api by NikolayBorovenskiy · Pull Request #181 · openedx/edx-notes-api

[BD-20] Elasticsearch fix: return course-specific notes by NikolayBorovenskiy · Pull Request #200 · openedx/edx-notes-api

indexing on prod caused errors due to malformed tags - checking to see if we can/should fix this.

  • PR review

  • PR merge

  • Deploy to Stage

  • Deploy to Edge

  • Deploy to Prod (target week: Sep 21st)

course-discovery

Engage (Always Available)
#discovery

Contact: @Michael Terry (Deactivated) ARCH-BOM: @Feanil Patel (Deactivated)

Waiting for new PR with original code + fixes.

Fixes code: [BUGS][NBorovenskiy]/OeX_ES=57 problems that were discovered after the first testing of the es7 course discovery service on live data on edx stage env by NikolayBorovenskiy · Pull Request #13 · raccoongang/course-discovery

  • RG working with @Feanil Patel (Deactivated) and @Michael Terry (Deactivated) to resolve search query discrepancies.

  • PR review

  • Manual test

  • PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (week: Jan 4th)

edx-search

TNL
#tnl-squad

Contact: @sburch (Deactivated)
ARCH-BOM: @Tim McCormack

[BD-19] Transition to the Elasticsearch 7.8.0 version by Golub-Sergey · Pull Request #104 · openedx/edx-search

https://github.com/edx/edx-platform/pull/25081

 

 

  • PR review

  • Test confidence

  • edx-search PR merge

  • edx-platform PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (Actual week: Nov 16)

cs_comments_service

TNL
#tnl-squad

Contact: @David Joy (Deactivated)
ARCH-BOM: @Robert Raposa

https://github.com/edx/cs_comments_service/pull/327

 

  • PR review

  • Test confidence

  • PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (target week: Oct 27th)

edx-analytics-data-api
edx-analytics-pipeline

Data Engineering
#data-engineering

Contact: @Stuart Young (Deactivated)
ARCH-BOM: @Diana Huang

https://github.com/edx/edx-analytics-data-api/pull/389

https://github.com/edx/edx-analytics-pipeline/pull/835

@Stuart Young (Deactivated) review and testing

  • PR review

  • Manual test

  • PR merge

  • Devstack PR merge

  • Deploy and Test on Stage

  • Deploy to Prod (target week: Nov)

 

Related content

Working Groups
Working Groups
Read with this
Elasticsearch 7.10 Upgrade Tracking
Elasticsearch 7.10 Upgrade Tracking
More like this
ES7 deployment strategy
ES7 deployment strategy
More like this
Elasticsearch Upgrade Task Force
Elasticsearch Upgrade Task Force
More like this
Elasticsearch 7.10
More like this
Large Instances Meeting Notes 2024-08-06
Large Instances Meeting Notes 2024-08-06
More like this