...

Transformer Version Updates

Two-phase Deployment

When deploying a Transformer with a new version:

• Release Phase 1. Release  Release the code with the new Transformer schema in a prior release of the code that actually requires using it.  The schema change should be backward compatible so that older Transformer code doesn't break.
• Run Update Tasks. Run  Run a management command to enqueue CBS updates for all courses in the system. 

  Jira Legacy
  server JIRA (openedx.atlassian.net) serverId 13fd1930-5608-3aac-a5dd-21b934d3a4b4 key TNL-6325

• Release Phase 2. Release  Release the new Transformer code that requires the new schema after 24-hours (need to monitor to get a better estimate) from running the management command.

Versioning Scheme: read-version versus write-version

Right now, Transformers only have a single version number. But with a multi-phase version upgrade, we'll need to distinguish which version is used for writing versus which version is used for reading.

The only way a single version would work is if the management command runs on an isolated machine with the updated code (writing data with the new version), while remaining workers execute old code (expecting the previous version or higher).  However, in a world of continuous deployment we don't want to halt version releases and wait for a single machine to complete its work before upgrading other workers.

So, the version increment process would be:

• In Release Phase 1, increment the write-version.  
  • Updates store this new write-version as the schema version of the Transformer's data.  
  • Reads continue to compare the old read-version (being less than or equal) to the version of the Transformer's data.
• In Release Phase 2, increment the read-version. 
  • Both versions are now equal.
  • Reads now compare the new read-version to the version of the Transformer's data.

Archive: Considered Solutions (from Brainstorms)

...