Page Comparison

...

Technologies

Haystack

• current version: 2.8.1

• used versions: 2.8.1 (course-discovery), 2.8.0 (edx-notes-api)

• supported ES versions: 1.x, 2.x

...

• Forums - still not working in devstack 😞

  • Alan Zarembok (Deactivated) - maybe ask someone on Lahore to see if they can do something about this.

• Stuart Young (Deactivated) - still working on failures on analytics api

• Diana Huang - edx-notes, still running into issues with make upgrade

• Fred Smith (Deactivated) - still working on sandboxes

• Michael Terry (Deactivated) - discovery - managed to get things to ES5 and tests fail

  • Lack of good documentation

• Dave Ormsbee - switched to edx-search

Agenda 4/30/20 -

• Discuss Blended Brief

  • Discuss the possibility of moving off Haystack as a Blended Brief

    • Diana Huang to come up with a draft

  • Scope down the work to having forums, edx-search, and analytics-data-api to latest version of ES blended brief

    • got buy-in from owners on reviewing/scoping work

• Made decision to target ES7 because there are still major limitations to using ES5 with our haystack stack (see notes)

Useful notes/resources

• Elasticsearch-py 5.0.0+ requires changing from using the create to index API call (see: https://github.com/elastic/elasticsearch-py/issues/474#issuecomment-256903012 )

• The docs for elasticsearch-py indicate that you must use a matching major version. And in my testing, that’s true – or at least, that the surrounding changes in our own code (renamed parameters, etc) don’t work with both server versions.

• Old (server) release notes, each with breaking changes listed: https://www.elastic.co/guide/en/elasticsearch/reference/index.html

• Migration helper plugin for server for 1->2 and 2->5: https://github.com/elastic/elasticsearch-migration

  • Only warning for 1->2 upgrade locally: “Boolean fields will return 1/0 instead of T/F in scripts, aggregations, or sort values, in fields: modelresult:has_enrollable_paid_seats, modelresult:has_enrollable_seats, modelresult:hidden, modelresult:hidden_exact, modelresult:is_current_and_still_upgradeable, modelresult:is_program_eligible_for_one_click_purchase, modelresult:mobile_available, modelresult:mobile_available_exact, modelresult:published, modelresult:published_exact.” – we need to see if we rely on old behavior anywhere

  • Only warning for 2->5 upgrade locally: “Indices created before v2.0.0 must be reindexed” – which means we must upgrade to 2.x, reindex, then upgrade to 5.x.

• While beta versions of django-haystack 3.0 do support ES 5.x, drf-haystack (used by Discovery at least) requires the 2.8.x version of django-haystack. So we’d also need to get them to release a beta (and they are light on maintainers).

...