Non-tutor deployment discussion

Brian: Every MFE is a SPA. running every single one of those without some tool to do it for you (like tutor) is painful. But with frontend-base, this issue goes away, b/c there will be one “thing” to run.

Feanil: We’ve been moving towards making the platform less tutor-required in order to handle complexity. We’re simplifying complexity – this is something that tutor has served as a layer to help with. We think that we’re aligned that this is useful in itself: upstream complexity reduction.

Dave: to what extent do we take a stance on uwsgi, granian, etc? for example, dave would really like to use X_ACCEL_REDIRECT, which is available on Caddy and some others, but not on everything. would be nice to be able to assume operational features

• Feanil: Which kind of worker process/load balancer we use, there’s a point at which it’s definitely an operator choice (e.g. load balancers are definitely outside). Something like uwsgi process is something we can have opinions on, e.g. “we expect these things, these system dependencies” (in README). We can be more explicit about what we need on a feature basis, e.g. “anything that supports X-Accel-Redirect”. Means that we are implicitly accepting bug reports from a wider range of things, but it’s a conversation we should have, e.g. we support things that have these features, but if you go outside, we’re not supporting it (we’ll accept PRs though).

• Adolfo: Another example – webpack. But would we actually be agnostic to that? Probably not. We will probably need to do this on a Case-by-case basis.

Part of the discussion here is that we could just rely on tutor, we could move on to more useful work in the platform and hand off the managing of the packages to tutor.

Ty: we ought to have a non-tutor reference deployment. A source of truth that isn’t just tutor

• Specifically known good service versions and configuration options that match Tutor

• Also for each release

• Feanil: EVen without a full reference deployment , we can document our operational constraints today using READMEs. For example: GitHub - openedx/openedx-platform: The Open edX LMS & Studio, powering education sites around the world! . We can even say things like “We support any proxy which supports X_ACCEL_REDIRECT” as a way of helping operators choose their infra without just using tutor wholesale.

Brian: We shouldn’t assume tutor, but we should develop tutor-first. Metaphor: in game development, you don’t developing assuming a single console, but you do start by targeting on console and then port it to others. Tutor is something we can rely on in the meantime as long as we don’t have a non-tutor reference (“Open edX from Scratch”).

• Feanil: Agree. But, there are things that work with tutor, but don’t work with pure upstream. Perennial example: Tutor sets toggles to True, and it gives the appearance of things being on by default, but they are not on by default upstream, which creates confusion.

• Kyle: We need a way of enforcing this, like a zero-config override policy

• Feanil: yes but even without enforcing it, we can do a lot of cleanup now

• Brian: How would this fit in without plugins being installed by default?

Adolfo: Agree with the above, but would +1 the idea of actually creating and maintaining a “from scratch” / “bare metal” install method. There is someone on the forums who is very keen to deploy without tutor - we should connect with him. (This person.)

• Feanil: GitHub - feanil/minimal-edx-platform: The minimal stuff needed to spin up default edx-platform for dev/testing.

• Kyle: agree, but don’t make another tutor by accident:GitHub - regisb/openedx-install: Open edX: The "Install From Scratch" Manual

• Brian: Does zero config overrides handle plugins being installed in the default install?

  • Kyle: The problem are feature toggles that Tutor core has, like ENABLE_COMPLETION. That should be done upstream. Not objecting to having tutor plugins enabled by default.

    • Feanil: Rollout story is better on tutor, but that’s something we can use to our strength. Eg.. for new catalog page (plugin enabled by default). At some point we’ll decide that it is defacto and should be installed by default in edx-platform. Having the tutor plugin on by default before doing it in platform gives us a 70% rollout.

      • Kyle: Could we have plugin to opt-out?

        • Feanil: That’s fine but part of the reason it’s a plugin in the first place is because we’re not sure if we want it in the platform.

        • Brian: makes more sense for things like filters, not MFEs. In MFEs, the default is it being added to the MFE plugin. What gets upstreamed and moved from plugin to upstream is bigger for some plugins than it would be for an MFE itself.

        • Feanil: We have these words that mean different things in different context depending on what we want them to mean (“on by default”, “optional”, etc.)

        • Kyle: We align that using the plugin system is a good way to rollout, if the core product relies on tutor in particular then tutor becomes part of the core product.

Dave: There’s a fair amount of deployment simplification that happens in tutor because tutor pushes and accelerates some of this work which is valuable.

Dave: if this is a priority, then we should make some time at the conference to speak about it. there have been proposals like “here’s how we run that platform without tutor” but they haven’t made it in

• Feanil: Maybe a technical town-hall or BOF would be useful for this.

Adolfo: We can’t be fully independent of tutor and have a backend+frontend combined plugin system. This is a gap outside of tutor, then we have to do things manually.

• Kyle: A path to this by improving sample plugin to show step by step how you would tie things together, and how Tutor would implement this mapping.

• Brian: One of the reasons why Tutor works well is because it’s opinionated about where the frontend is. It is the build pipeline. But outside of Tutor, it’s much looser – you might have deployed your MFEs to a CDN, and if you change a setting in the plugin, that doesn’t guarantee that there is a way to rebuild which pushes those changes to the CDN.

• Adolfo: ArchWiki is a good example of an english-based plugin system. Imagine a page which says “How to install Aspects” – it would walk an operator through the installation process, acknowledging that we can’t know exactly what their deployment looks like.

• Feanil: “Do you want it really easy? We have tutor. Do you have constraints? Here are all of the details and the guts of it, have fun.”

Dave: It’s great to say “we should document it better” and it’s great to have a tutor-agnostic deployment method (though it will be hard to maintain over time). But, does this solve the root issue that tutor maintainers, rather than product WG, have the final say over what features are on and off? If we want to shift that final say, then we need to have some way of choosing which plugins/config go into tutor as part of the default install.

• Feanil: Yes, agreed. We seem aligned on the value of supporting non-tutor. But there is still the question of what our Core Product is. The working model had been that “Core Product == Tutor Default”, but that has caused discord sometimes, notably with Indigo, where the tutor maintainers saw it as the right thing for the community, but axim saw it as a layer of complexity that we needed to peel off

• Ty: likes a Core Product plugin which uninstalls the stuff we want, installs the stuff we do. Would need some one-line way for operators to do this, for the purpose of setting up a test env or for getting a reference installation

• e.g.

  pip install tutor-openedx # tutor + our openedx overrides

• Brian: What about working to do a tutor[vanilla] so that it’s upstream and a bit more collaborative.

• Dave: Not worried about turning of things like indigo, but more worried about lower level infrastructure things might be harder, eg. granian/uwsgi.

• Ty: One thing that prompted this was making the new notifications as a default plugin.

  • Will the new plugin be able to do this sort of thing without breaking other things?

    • Brian: For MFEs we can put things in as the default content of a slot, but that means that frontend code lives in the MFE instead of in a plugin. We don’t have a way to install frontend plugins by default without something external (like Tutor).

• Kyle: Which would go on the testing site?

  • Feanil: We currently do the default theme, not Indigo on the test site, because that’s how we develop.

Action Items