Tutor Users' Group

Owned by Kyle McCormick
Last updated: Nov 18, 2024
9 min read

Charter

It’s become clear that there is an acute need to support new Tutor users and plugin developers. It’s also clear that Tutor maintainers benefit from hearing feedback from people who are using the tool. 

Please join our users’ group for sharing help, tips, and feedback on using Tutor and developing Tutor plugins. Bring your questions, and get ready for some live debugging! Expert or newbie, all are welcome.

Announcements will be posted to Slack in #tutor-maintenance

Recordings and Transcripts

All public Working Group meetings follow the Recording Policy for Open edX Meetings

https://drive.google.com/drive/folders/1hdlItbQ6RK8iHeOZXahpo_vIo5offr_h

Agenda and Notes

We meet every other week. Our schedule and video conferencing link is on the Working Group Calendar. The meeting is usually hosted by @Kyle McCormick .

You can add topics below ahead of time, or add them during the meeting.

2024-11-18

2024-11-04

2024-10-21

2024-09-23

2024-08-26

2024-08-12

2024-07-29

  • Moises: Dockerfile reference I wanted to use the RUN --mount=type=ssh instruction to install private packages. This command needs an additional flag to the build command that I can add via the DOCKER_BUILD_COMMAND filter. The problem is that the --ssh=default=$SSH_AUTH_SOCK flag gets escaped by shlex. As a workaround is possible to use os.environ.get("SSH_AUTH_SOCK") but some people may not want to print those values to the stdout.

    • what type of packages?

    • Zameel: Installing npm packages from a private github repo

      • cloned with personal access token, installed locally

      • didn’t need to use --mount=type=ssh

    • Dawoud: is it --ssh=default=$SSH_AUTH_SOCK or --ssh default=$SSH_AUTH_SOCK ?

    • Kyle: --ssh=default=$SSH_AUTH_SOCK – what is getting escaped?

    • Max: we are also interested in this answer

      • @Moisés González do you have a POC to try?

      • RG is interested in helping to implement or test the proposal

      • In our non-tutor deployments we extensively used SSH key for private repos cloning

  • Max’s verdaccio plugin: Local NPM registry for MFE builds

    • Issues

      • doesn’t work without host.docker.internal hostname

    • Important insight- timing

      • very good time for low parallelism (max 2) – 12min

      • worst time for high parallellism (no max) – 27min

    • Regis

      • Caching npm deps – good.

      • Regis runs verdaccio on a remote server.

        • Getting issues with timeouts/connection.

        • not sure if connection issue is b/w verdaccio and tutor or verdaccio and npm

      • Running a container before a build – not supported in tutor naturally

      • Doesn’t make sense to have separate verdaccios for nightly vs master

      • Decouple launching verdaccio instance vs configuring NPM_REGISTRY

        • starting verdaccio could be managed by docker, not docker-compose

        • tutor local do verdaccio-start ?

      • Max - similarly, don’t like idea of having a Tutor service that’s not truly a tutor service

      • Max - why a separate machine for verdaccio?

        • regis - didn’t know about host.docker.internal

    • Max - it doesn’t seem to be resource intensive for his machine

      • Max’s coworker disagrees

      • Regis - it uses …. MB memory and … MB disk space. Not very intensive in either case

    • What does the api look like?

      • Regis: tutor local do verdaccio start

        • would start verdaccio. would print an instruction to run tutor config save --set NPM_REGISTRY=host.docker.internal:$PORT

      • Kyle: tutor verdaccio start, same as above

      • Max: tutor npm-mirror start, same as above

      • Max: plugin or core?

      • Regis: try out a couple weeks , and see whether to add it to core?

      • Kyle: worried about resource usage depending on different machines, want it to be optional

    • Max: parallellism, why does unbounded parallelism slow it down?

      •  

  • Zameel - non-latin languages and mysql 5.7

  • Nihad: https://discuss.openedx.org/t/unable-to-import-open-edx-demo-course-in-redwood-release/13455 ,

2024-07-15

  • Glib: Tutor Plugins -

    • Discussing with marketing wg the idea of having a marketplace catalog

      • Making plugins available for operators

      • Particularly plugins with one-click installation process

      • ^ … that’s tutor plugins!

        • particularly plugins in the tutor index

      • Question: Many plugins require post-actiosn

        • Rebuild image

        • Run init

        • Configure something

      • Idea: a flag that runs all those necessary extra steps automatically, to enable the one-click installation

      • Idea: always rebuild and run init

        • Max mentioned that he wouldn’t want Tutor to automatically run any actions, since many operators have customizations that make it so they wouldn’t want to do every step necessarily

        • in other words – some operators will want to be able to run steps manually, others will want it all to happen automagically

      • Dave: Could Tutor have a GUI interface ?

      • Kyle: What is the UI for the marketplace to be?

        • Glib: First phase, just a catalog

          • Description, installation instructions

          • Manually maintained

        • Glib: Integration into tutor would be a second phase

      • Dawoud: We are already re-building and re-initting when we run tutor ... launch, so not sure if we want a flag on the plugin command

        • Ghassan: we also have tutor ... do init --limit=pluginname

        • Glib: We’re thinking about command that must be run after plugin is installed and enabled

        • Idea:

          • tutor plugins install jupyter --enable --rebuild --run-init --configure JUPYTER_VERSION=5

        • Glib: Currently it requires several commands

          • tutor plugins install blah tutor plugins enable blah tutor config save --set ... tutor images rebuild openedx tutor local do init --limit=blah

        • One option:

        • A compromise:

      • Moises: If launch had a flag to rebuild, would that cut it?

      • Dawoud: install can take multiple plugins

      • Kyle/Glib: --enable would enable all provided commands

      • Glib: When you’re installing a plugin from outside the indexes, it’s pip install -e ., right?

        • `tutor plugins install ./path/to/mylocal/plugin`

          • this works

        • $(tutor plugins printroot)/myplugin

          • putting plugins here will install them automatically

      • Three changes, Kyle will make issues, looking for volunteers to implement

        1. An --enable flag on tutor plugins install

        2. A --configure flag on tutor plugins install , just like tutor config save --set ...

        3. A --limit-init flag on tutor ... launch

  • Moises: Build time for installing Python packages: https://github.com/overhangio/tutor/pull/1088

    • Glib: Brian M says he was experimenting with pip-tools replacement in rust, but hit an issue where it doesn’t support egg files

    • Dawoud:

  • Michael: being able to limit init for launch might solve another problem: Shared mysql server, don’t want to provide admin user or password, so mysql init fails

    •  

2024-06-03

  • MFE build time (below)

  • Configuring open edx services with tutor

    • Moises and Maksim work on large instances group

    • Some of tutor’s default were meant to simplify setup

    • For large instances, they need to undo tutor’s defaults

    • example: celery queues

      • tutor uses one celery queue

      • at scale, the queue can fill up

        • using patches, large instances has been undoing tutor’s configuration changes

    • edx-platform is complex to follow, with all the indirection

      • related: OEP-45 simplification

    • could be better to have a blank slate

    • edunext had to replicate ansible installation settings, using tutor

    • they are using settings patches

      • yaml settings patch

      • common settings

      • production patches

      • some files don’t have patches

  • Having plugins expose configuration interfaces

    • tutor-contrib-pod-autoscaling

    • rather than add more config settings, the plugin

  •  

2024-05-20

 

 

2024-05-06

…

2024-04-22

Present: several people

  •  

2024-04-08

Meeting link: https://meet.google.com/rhs-veey-rrp

Present: @Chintan Joshi , @Danyal Faheem, @Dave Ormsbee (Axim) , @Dawoud Sheraz , @Deimer Morales ,@Maksim Sokolski, @Maria Grimaldi, Qasim Gulzar , @Régis Behmo

  • Integrate Tutor with IntelliJ IDEA/Pycharm (by Qasim Gulzar@edly)

  • How to track the eclipse with Tutor (by @Régis Behmo if time permits) (it did not)

Notes:

  • MongoDb 7 upgrade: scheduled for Redwood

  • utf8mb3 upgrade. We reached the following decisions for Redwood:

    • upgrade the connection string and the database default charset to utf8mb4 for all platforms

    • For existing platforms, upgrade the default database charset (with ALTER DATABASE databasename CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;).

    • For existing platforms, convert tables in a conservative manner, using instructions similar to:

      ALTER TABLE tablename CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;. The list of candidate tables is available here.

    • Implementation of these decisions can be followed in this GitHub issue: https://github.com/overhangio/tutor/issues/938

  • MySQL high memory issue on Arch Linux: https://github.com/docker-library/mysql/issues/579#issuecomment-1075119349

    • It was suggested that Arch users manually create a docker-compose.override.yml file.

    • We should update the troubleshooting instructions for Arch Linux users. @Deimer Morales can you please open a pull request to update the Tutor docs?

2024-03-25

  • Auto-mounted directories

    • Can we add mount keywords in our own plugin? → yes, or open a PR to upstream plugin

    • Do we list the keywords anywhere? example, platform-plugin-

      • The exact list is not documented anywhere

  • MFE mounts

  • Let’s talk about MFE image building… why we (usually) don’t need to re-build, and figuring out this Docker build cache madness. (@Régis Behmo )

    • Theoretically, rebuild should only happen if (1) we change an MFE and (2) we’re pushing it to production

    • Errors when installing node_modules

      • npm error, socket reset, registry hangup

    • Docker build cache size needs to be set large enough so that it doesn’t get garbage collected between mfe builds

  • Talked about https://github.com/kdmccormick/tutor/pull/34 - kyle will continue to work on

  • Production vs dev flow in docs - docs nudge you towards production flow

  • Remote image building - anyone?

    • edunext has - they build tutor-nightly images on an ec2 instance

      • 20-30 minutes for uncustomized mfe image

    • custom multiarch builder - regis builds macos images even though he doesn’t have a mac

Older Notes

Paste old notes here.