Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Land Cliff's PR: https://github.com/edx/edx-platform/pull/14925
  2. Ops – Upgrade rabbit servers. 
    Jira Legacy
    serverJIRA (openedx.atlassian.net)
    serverId13fd1930-5608-3aac-a5dd-21b934d3a4b4
    keyOPS-1837
     
    Jira Legacy
    serverJIRA (openedx.atlassian.net)
    serverId13fd1930-5608-3aac-a5dd-21b934d3a4b4
    keyOPS-1838
     
    Jira Legacy
    serverJIRA (openedx.atlassian.net)
    serverId13fd1930-5608-3aac-a5dd-21b934d3a4b4
    keyOPS-1835
  3. Create new celery queue / routing key in edx-internal (Eric Fischer (Deactivated)) - 
    Jira Legacy
    serverJIRA (openedx.atlassian.net)
    serverId13fd1930-5608-3aac-a5dd-21b934d3a4b4
    keyTNL-6875

    1. https://github.com/edx/edx-internal/pull/50
    2. Update jenkins job to reference --routing_key
    from #3 above.
  4. Enable persistent grades on edge. (Nimisha Asthagiri (Deactivated)?) - find ticket
    Jira Legacy
    serverJIRA (openedx.atlassian.net)
    serverId13fd1930-5608-3aac-a5dd-21b934d3a4b4
    keyTNL-6626

Backfill Grades

  1. Enable write_only_if_engaged waffle switch on all environments (prod, stage, edge, load test). - ticket
  2. Slow delete persistent grades tables (34 million rows in prod). ~5-8 hours expected - Devops ticket
  3. Export list of courses (per environment) from course overviews table on read replica or call Courses API.  (Note: Do not start this until tables have been deleted. - ticket

Stage
Jira Legacy
serverJIRA (openedx.atlassian.net)
serverId13fd1930-5608-3aac-a5dd-21b934d3a4b4
keyTNL-6874

  1. Delete persistent grades tables.
  2. Run against demo course, with 100 users per batch.
  3. Delete persistent grades tables again.
  4. Run against demo course, with 1000 users per batch.
  5. Run against all other courses on stage, using 1000 users per batch.
  6. Decide if we're happy with batching and worker count.

Edge 
Jira Legacy
serverJIRA (openedx.

...

atlassian.net)
serverId13fd1930-5608-3aac-a5dd-21b934d3a4b4
keyTNL-6876

  1. Redo batch size testing, if stage didn't have enough data for useful metrics.
  2. Using batch size from batch size test here or or stage, run against 10 courses
  3. Run against 100 courses
  4. Run against all courses.

Production
Jira Legacy
serverJIRA (openedx.atlassian.net)
serverId13fd1930-5608-3aac-a5dd-21b934d3a4b4
keyTNL-6877

  1. Use batch size from edge
  2. Run against demo course
  3. Run against 10 courses
  4. Add extra celery worker
  5. Run against 10 courses
  6. Run against 100 courses
  7. Run against 200 more courses (optional)
  8. Run against all courses

...