Versions Compared

Old Version 14

changes.mady.by.user Edward Zarecor (Do Not Use) (Deactivated)

Saved on

compared with

New Version Current

changes.mady.by.user Feanil Patel

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Brian Beggs discussed the progress toward milestone 1 of Docker Devstack.  At M1, developers working at edX will be transitioning away from Vagrant to develop on Docker Devstack exclusively.  The most important outstanding work is around getting docker images built and pushed to dockerhub.io from CI.  We are hopeful that this work will merge in the next couple of days.
  • Edward Zarecor (Do Not Use) (Deactivated) discussed the current state of Split Mongo Modulestore.  Specifically, that implementation does copy on write of course structures for the purpose of versioning.  Use of versions is currently limited to rolling back to the prior version, but all structures are stored.  The fact that we make a complete copy on each change, couple with the fact that content has bled into structures, causes split to chew through storage very rapidly.  We are currently in discovery for the change, but are looking at strategies for pruning the structures back.
    • e (Unlicensed) mentioned that his team was actively working on a solution to this problem.  He will reach out to Feanil Patel and Edward Zarecor (Do Not Use) (Deactivated) to collaborate.

    • This rough and ready JavaScript has been kicking around for a while, but is too slash-and-burn for real use.  It's provided for context only.

      Code Block
      languagejs
      var sleep_time=5000

db.modulestore.active_versions.find().forEach(function(doc) {
    db.modulestore.structures.update({_id:doc.versions['draft-branch']},{$set: {"do_not_delete":1}})
})

db.modulestore.active_versions.find().forEach(function(doc) {
    db.modulestore.structures.update({_id:doc.versions['published-branch']},{$set: {"do_not_delete":1}})
})

db.modulestore.structures.find({'do_not_delete': { $ne: 1 }}).count()

// Danger, danger.  Make sure you have a backup and really know what you're doing before uncommenting.
// db.modulestore.structures.find({'do_not_delete': { $ne: 1 }}).forEach(function(doc) { sleep(sleep_time); db.modulestore.structures.remove({_id:doc._id});})

db.modulestore.structures.find({'do_not_delete': { $ne: 1 }}).forEach(
    function(doc) {
	sleep(sleep_time);
	db.modulestore.structures.remove({_id:doc._id});
	
    })


db.modulestore.structures.update({},{$unset: { 'do_not_delete':1}}, false, true)


  • Kevin Falcone (Deactivated) let a discussion queuing up our thoughts about a redesign of CSMH the supports low cost storage alternatives and configurable limits to storing history of student submissions.
    • We asked that the community please submit their research and auditing use cases so we can take them into consideration
    • Discussions are in early phases now
    • Moving history out of the database should:
      • reduce cost
      • speed deployments and tests, obviating repeated migrations
      • avoid exhausting 32bit unsigned int primary keys in the CSMH tables
  • DevOps Team update: We are happy to announce that Joe Mulloy (Deactivated) has joined our team
  • Feanil Patel discussed the status of the ElasticSearch upgrade.
    • We are using 1.5 everywhere, which allows us to ship additional services on devstack and fullstack that could not use 0.9, which the forums required
    • Sven pointed out that 1.5 is already end of life'd
    • We acknowledge this and plan to upgrade, but that work is blocked by efforts to remove our dependency on Haystack, which is no longer an active project.
    • The following projects rely on Haystack
      • notes
      • discovery
  • Ned Batchelder (Deactivated) led a discussion of the status of planning for Ginko!
    • Things to know
      • There will be no upgrade.sh
      • Will not include docker
      • This will be the first release that includes the catalog service
      • Will ship with ElasticSearch 1.5
      • Will ship with Ansible 2.3.1
      • Please help with testing!
  • Julia Eskew (Deactivated) led a discussion of our plans for the Django 1.11 upgrade.
    • We believe we have a solid idea of the scope of the project
    • We'll be introducing tox to allow testings with both 1.11 and 1.8
    • 1.11 will be release with Hawthorn and is an LTS release
    • 1.11 is the last release of Django that will support python 2x
    • We have decided that upgrading to python 3 is out of scope for this effort
    • John is doctoryes on github: https://github.com/doctoryes
    • We are happy for community help with upgrades both PRs and testing, please reach out
    • This upgrade is driven by staying current with security patches rather than specific new features
    • We're starting now to make sure that no named release will be in support while it's version of django is out of support
  • Edward Zarecor (Do Not Use) (Deactivated) presents on EdwardF (Deactivated)'s behalf, our effort to deprecate and retire the edX OpenID provider functionality.  
    • This was originally built for a single course, but has been used in other ways. 
    • We have an alternative for folks that are currently using the provider, please reach out for details
    • We've wanted to do this for a while, but were motivated by compatibility issues that would block the 1.11 upgrade
  • Edward Zarecor (Do Not Use) (Deactivated) we're aiming to merge Stanford's changes in their fork of edx-certificates back upstream
  • Amy asked