Large Instances Meeting Notes 2023-11-28
Video recording: https://us02web.zoom.us/rec/share/TAEEzM5djRppbKtaP_ZSs6ZQ_BwXfycdja_h-HgPHCUhxxVBgWVnN4hvZSEpLsQ.Lmi-J20W6Wvv4C38?startTime=1701190788000 (Passcode: %Gh06CDe )
Notes
Updates from the Community:
Jeremy (2U):
Experimenting with the setup of Devspace. For now, Harmony does not have all the required dependencies, so the Devspace Basic helm charts are being used. There are challenges in using custom images. The development workflow is still WIP
Felipe (Edunext):
Working on OpendX K8S operation cost reduction, and prioritizing the next few months' projects.
Codejail is still an issue in terms of resource consumption. Solutions are frozen for now.
Working also on a post for the harmony roadmap for the next 6 months.
A discussion about HPA implementation for OpenedX IDAs was opened at https://discuss.openedx.org/t/hpa-support-for-extra-idas-and-resources-in-the-openedx-ecosystem/11662. Feedback from the community is appreciated.
Working on the Aspects environment implementation.
Emad (Edspirit):
Working on Quince migration
Working on the PR here: https://github.com/openedx/openedx-k8s-harmony/pull/49 (@Moisés González will review it)
Opencraft
Working on the PR here: (monitoring PR which @Jhony Avella will review)
Integrating harmony on Grove
@Felipe Montoya discussed the potential of integrating the monitoring tools with Aspects; even when Aspects is not yet stable, it could simplify operations in the long run. @Braden MacDonald suggested testing a prototype once Aspects is stable. There’s no clear path to do so for now, however, the question is open for further exploration.
Raccongang
Working on a K3S single instance installation. This is WIP
Aspects: diving into the installation of the tool. Trying to create a helm chart for it.Testing in a development environment.
@Felipe Montoya mentioned that Aspects’s technology is there but making it usable with meaningful dashboards is quite a challenge. The expectation is that dashboards are going to be pluggable.
Open pull requests:
@Emad Rad PR: (@Moisés González as reviewer).
Monitoring PR by @Gábor Boros: (@Jhony Avella as reviewer).
Karpenter PR by @Jhony Avella : (it will get merged once @Gábor Boros confirm this issue is solved).
AI-Generated Summary (Likely Contains Errors)
Meeting summary for Kubernetes & Large/Multi Instances DevOps collaboration (11/28/2023) | |
|
|
Quick recap | |
|
|
The team discussed potential trial updates and the development workflow. Felipe shared updates from the Admin side, focusing on cost reduction and prioritization of the next few months' projects. The team also touched on a problem related to the co-jail. Updates on various projects were shared, including the progress of Emad's deployment and the integration of Harmony into Groove. The meeting concluded with a discussion on integrating monitoring aspects into their platform. | |
| |
| |
Summary | |
|
|
Trial Updates and Dev Space Exploration | |
Felipe initiated the meeting. The team discussed potential trial updates, with Felipe taking the lead. Jeremy presented updates from Eden's side, sharing that they had been experimenting with the setup of dev spaces and helm charts for clusters. He mentioned the possibility of using their internal home charts for development, as well as the Helm charts. However, he also noted some challenges with using custom images. The team agreed to continue exploring these options. | |
|
|
Workflow, Processing, Cost Reduction, Co-Jail Issue | |
The team discussed about the development workflow and the delivering processing space, with Moisés inquiring about the process and Jeremy acknowledging that they were still working on it. Felipe also shared updates from the Admin side, focusing on cost reduction and prioritization of the next few months' projects. They also touched on a problem related to the co-jail, with Moisés mentioning that the fixes were currently frozen. Towards the end, Felipe further explained the issue with the co-jail and the potential risks associated with it. | |
|
|
Progress and Harmony Integration in Projects | |
The team discussed the progress of various projects. Moisés shared updates on aspects of Emad's deployment, with plans to keep the team informed as progress is made. The team also discussed the Hpa implementations in Kubernetis, with jhony highlighting the need for further discussion. Emad raised a PR about a Harwon project and requested a reviewer, which Moisés agreed to do. Gábor also shared updates on a PR he opened to address issues 37 and 38, introducing Prometheus and Grafana to the project, and plans to integrate Harmony into Groove. The team concluded with a discussion on moving forward with the plan to transition growth to using Harmony. | |
|
|
Integrating Monitoring Aspects Into Platform | |
Felipe discussed the potential of integrating monitoring aspects into their platform, noting that while aspects is not yet stable, it could simplify operations in the long run. Braden suggested testing a prototype once aspects is stable. Max shared updates on their experiments with creating a production-ready Kubernetes environment and their ongoing work with aspects. Felipe also mentioned that they are working on publishing information for users using analytics, but converting logs into user-friendly dashboards has been challenging. The team also discussed several open pull requests and upcoming priorities. | |
|
|
Next steps | |
|
|
| |
| |
| |