Page Comparison

...

• Follow-up on tutor-contrib-verdaccio.

  • Official plugin? Core? Not ready? Need more time to evaluate?

• We tutor maintainers should keep an eye on DEPR tickets and make sure plugins get deprecated if the upstream code gets deprecated

2024-07-29

• Moises: https://docs.docker.com/reference/dockerfile/#run---mounttypessh I wanted to use the RUN --mount=type=ssh instruction to install private packages. This command needs an additional flag to the build command that I can add via the DOCKER_BUILD_COMMAND filter. The problem is that the --ssh=default=$SSH_AUTH_SOCK flag gets escaped by shlex. As a workaround is possible to use os.environ.get("SSH_AUTH_SOCK") but some people may not want to print those values to the stdout.

  • what type of packages?

  • Zameel: Installing npm packages from a private github repo

    • cloned with personal access token, installed locally

    • didn’t need to use --mount=type=ssh

  • Dawoud: is it --ssh=default=$SSH_AUTH_SOCK or --ssh default=$SSH_AUTH_SOCK ?

  • Kyle: --ssh=default=$SSH_AUTH_SOCK – what is getting escaped?

  • Max: we are also interested in this answer

    • Moisés González do you have a POC to try?

    • RG is interested in helping to implement or test the proposal

    • In our non-tutor deployments we extensively used SSH key for private repos cloning

• Max’s verdaccio plugin: https://discuss.openedx.org/t/local-npm-registry-for-mfe-builds/13526

  • Issues

    • doesn’t work without host.docker.internal hostname

  • Important insight- timing

    • very good time for low parallelism (max 2) – 12min

    • worst time for high parallellism (no max) – 27min

  • Regis

    • Caching npm deps – good.

    • Regis runs verdaccio on a remote server.

      • Getting issues with timeouts/connection.

      • not sure if connection issue is b/w verdaccio and tutor or verdaccio and npm

    • Running a container before a build – not supported in tutor naturally

    • Doesn’t make sense to have separate verdaccios for nightly vs master

    • Decouple launching verdaccio instance vs configuring NPM_REGISTRY

      • starting verdaccio could be managed by docker, not docker-compose

      • tutor local do verdaccio-start ?

    • Max - similarly, don’t like idea of having a Tutor service that’s not truly a tutor service

    • Max - why a separate machine for verdaccio?

      • regis - didn’t know about host.docker.internal

  • Max - it doesn’t seem to be resource intensive for his machine

    • Max’s coworker disagrees

    • Regis - it uses …. MB memory and … MB disk space. Not very intensive in either case

  • What does the api look like?

    • Regis: tutor local do verdaccio start

      • would start verdaccio. would print an instruction to run tutor config save --set NPM_REGISTRY=host.docker.internal:$PORT

    • Kyle: tutor verdaccio start, same as above

    • Max: tutor npm-mirror start, same as above

    • Max: plugin or core?

    • Regis: try out a couple weeks , and see whether to add it to core?

    • Kyle: worried about resource usage depending on different machines, want it to be optional

  • Max: parallellism, why does unbounded parallelism slow it down?

• Zameel - non-latin languages and mysql 5.7

  • course outline was not getting saved to mysql

    • mysql was using latin-1 instead of utf-8

      • Kyle: is https://discuss.openedx.org/t/palm-upgrade-running-rdbms-in-tutor-mysql-vs-mariadb-utf8-vs-utf8mb4/11030 relevant?

    • Zameel: https://discuss.openedx.org/t/load-missing-lesson-information-in-mysql-table/10034/11

  • Regis: In Redwood (v18), default collation changed from utf8 to utf8mb4

  • Zameel: could we have tutor docs warning about this?

  • Dawoud: https://github.com/overhangio/tutor/pull/1079

  • Zameel: to fix, we made a mysql backup, changed encoding from latin-1 to utf-8. encoding, not character set.

  • Dawoud: when tutor configures mysql, we pass in args to ensure it’s utf8: https://github.com/overhangio/tutor/blob/palm/tutor/templates/local/docker-compose.yml#L43 . But these are not passed to the managed service.

  • Zameel: mysql 5.7 by default uses

  • Kyle: “bare metal” edx-platform docs: https://github.com/openedx/edx-platform?tab=readme-ov-file#bare-metal-advanced

    • a note there that flags must be passed in to configure mysql to use utf-8 and utf8mb4 , rather than latin-1

    • not neccessary for mysql 8, though.

• Nihad: https://discuss.openedx.org/t/unable-to-import-open-edx-demo-course-in-redwood-release/13455 ,

...