2017-09-26 Hangout Agenda and Notes
The Hangout agenda is open to any topics related to deploying edX and running it in production. Add your agenda topics directly to ths document.
Joining the hangout does require an invitation. If you need one, please send an email to openedx-operations-hangout@edx.org from the account that you would like us to invite.
Agenda:
- Future of full stack
- Update on Mongo structure pruning
- Django 1.11 Upgrade
Notes:
- Devstack in Hawthorne will be in docker
- Our plan is to drop the vagrant image of full stack. Full stack was just the native play run on Vagrant and maintaining.
- MITx is currently building
- Structure pruning
- Correct, but not performant version: https://github.com/edx/tubular/pull/184
- Correct, but more performant version: https://github.com/edx/tubular/pull/185
- PR 185 runs revealed a couple of edge case bugs
- Version histories overalaped partially in reruns, essentially an off by n type of bug
- Cases where courses had one and only one version
- Tobias' questions
- What are some general things that you recommend for performance
- Optimize workers to CPU units
- Use instance types with reasonable network throughput
- Cache document structures in memcache – validate you have the right config for this
- Moving to a shared cluster as, MITx is doing, is our best practice recommendation
- Can code jail/cappa be run on a separate set of hosts from the main LMS process
- Code jail can and is today
- Capa is theoretically possible
- What tools are you using to solve performance problems
- New Relic
- APM
- custom instrumentation that we've added
- Meliae, send a PROF signal to a specific process to get a dump of current process memory
- New Relic
- What are some general things that you recommend for performance
- Django 1.11 update
- Tox merged to the platform today to allow testing with both 1.8 and 1.11
- The plan is to run both in parallel avoid regressions
- We are expecting logs of failures in 1.11 at first
- We will black list the broken tests initially.
- When they are fixed, we'll move them to whitelist
- Eventually we'll converge on 1.8/1.11 dual compatibility
- We plan to have this work done by the end of the year
- We are targeting Django 1.11 for Hawthorne
- Tox can also give you different versions of python
- After 1.11 comes Django 2.0, which will only support python 3.0
- A good example tox.ini file can be found here: https://github.com/edx/edx-ora2
- We also landed a change migrate from nose to pytest