Versions Compared

Version Old Version 38 New Version 39
Changes made by

Moisés González

Kyle McCormick

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Moises: https://docs.docker.com/reference/dockerfile/#run---mounttypessh I wanted to use the RUN --mount=type=ssh instruction to install private packages. This command needs an additional flag to the build command that I can add via the DOCKER_BUILD_COMMAND filter. The problem is that the --ssh=default=$SSH_AUTH_SOCK flag gets escaped by shlex. As a workaround is possible to use os.environ.get("SSH_AUTH_SOCK") but some people may not want to print those values to the stdout.

    • what type of packages?

    • Zameel: Installing npm packages from a private github repo

      • cloned with personal access token, installed locally

      • didn’t need to use --mount=type=ssh

    • Dawoud: is it --ssh=default=$SSH_AUTH_SOCK or --ssh default=$SSH_AUTH_SOCK ?

    • Kyle: --ssh=default=$SSH_AUTH_SOCK – what is getting escaped?

    • Max: we are also interested in this answer

      • Moisés González do you have a POC to try?

      • RG is interested in helping to implement or test the proposal

      • In our non-tutor deployments we extensively used SSH key for private repos cloning

  • Max’s verdaccio plugin: https://discuss.openedx.org/t/local-npm-registry-for-mfe-builds/13526

    • Issues

      • doesn’t work without host.docker.internal hostname

    • Important insight- timing

      • very good time for low parallelism (max 2) – 12min

      • worst time for high parallellism (no max) – 27min

    • Regis

      • Caching npm deps – good.

      • Regis runs verdaccio on a remote server.

        • Getting issues with timeouts/connection.

        • not sure if connection issue is b/w verdaccio and tutor or verdaccio and npm

      • Running a container before a build – not supported in tutor naturally

      • Doesn’t make sense to have separate verdaccios for nightly vs master

      • Decouple launching verdaccio instance vs configuring NPM_REGISTRY

        • starting verdaccio could be managed by docker, not docker-compose

        • tutor local do verdaccio-start ?

      • Max - similarly, don’t like idea of having a Tutor service that’s not truly a tutor service

      • Max - why a separate machine for verdaccio?

        • regis - didn’t know about host.docker.internal

    • Max - it doesn’t seem to be resource intensive for his machine

      • Max’s coworker disagrees

      • Regis - it uses …. MB memory and … MB disk space. Not very intensive in either case

    • What does the api look like?

      • Regis: tutor local do verdaccio start

        • would start verdaccio. would print an instruction to run tutor config save --set NPM_REGISTRY=host.docker.internal:$PORT

      • Kyle: tutor verdaccio start, same as above

      • Max: tutor npm-mirror start, same as above

      • Max: plugin or core?

      • Regis: try out a couple weeks , and see whether to add it to core?

      • Kyle: worried about resource usage depending on different machines, want it to be optional

    • Max: parallellism, why does unbounded parallelism slow it down?

  • Zameel - non-latin languages and mysql 5.7

  • Nihad: https://discuss.openedx.org/t/unable-to-import-open-edx-demo-course-in-redwood-release/13455 ,

2024-07-15

  • Glib: Tutor Plugins -

    • Discussing with marketing wg the idea of having a marketplace catalog

      • Making plugins available for operators

      • Particularly plugins with one-click installation process

      • ^ … that’s tutor plugins!

        • particularly plugins in the tutor index

      • Question: Many plugins require post-actiosn

        • Rebuild image

        • Run init

        • Configure something

      • Idea: a flag that runs all those necessary extra steps automatically, to enable the one-click installation

      • Idea: always rebuild and run init

        • Max mentioned that he wouldn’t want Tutor to automatically run any actions, since many operators have customizations that make it so they wouldn’t want to do every step necessarily

        • in other words – some operators will want to be able to run steps manually, others will want it all to happen automagically

      • Dave: Could Tutor have a GUI interface ?

      • Kyle: What is the UI for the marketplace to be?

        • Glib: First phase, just a catalog

          • Description, installation instructions

          • Manually maintained

        • Glib: Integration into tutor would be a second phase

      • Dawoud: We are already re-building and re-initting when we run tutor ... launch, so not sure if we want a flag on the plugin command

        • Ghassan: we also have tutor ... do init --limit=pluginname

        • Glib: We’re thinking about command that must be run after plugin is installed and enabled

        • Idea:

          • Code Block
            tutor plugins install jupyter --enable --rebuild --run-init --configure JUPYTER_VERSION=5

        • Glib: Currently it requires several commands

          • Code Block
            tutor plugins install blah
tutor plugins enable blah
tutor config save --set ...
tutor images rebuild openedx
tutor local do init --limit=blah

        • One option:

          • Code Block
            tutor plugins install blah && \
tutor plugins enable blah && \
tutor config save --set ... && \
tutor images rebuild openedx && \
tutor local do init --limit=blah

        • A compromise:

          • Code Block
            tutor plugins install blah --enable --configure ...
tutor local launch --limit-init=blah

      • Moises: If launch had a flag to rebuild, would that cut it?

      • Dawoud: install can take multiple plugins

      • Kyle/Glib: --enable would enable all provided commands

      • Glib: When you’re installing a plugin from outside the indexes, it’s pip install -e ., right?

        • `tutor plugins install ./path/to/mylocal/plugin`

          • this works

        • $(tutor plugins printroot)/myplugin

          • putting plugins here will install them automatically

      • Three changes, Kyle will make issues, looking for volunteers to implement

        1. An --enable flag on tutor plugins install

        2. A --configure flag on tutor plugins install , just like tutor config save --set ...

        3. A --limit-init flag on tutor ... launch

  • Moises: Build time for installing Python packages: https://github.com/overhangio/tutor/pull/1088

    • Glib: Brian M says he was experimenting with pip-tools replacement in rust, but hit an issue where it doesn’t support egg files

    • Dawoud:

  • Michael: being able to limit init for launch might solve another problem: Shared mysql server, don’t want to provide admin user or password, so mysql init fails

...