...
\uD83D\uDC65 Participants
...
⏮️ Previous TODOs
| Task report | ||||||
|---|---|---|---|---|---|---|
|
\uD83D\uDDE3 Discussion topics
...
Item | Presenter | Notes |
|---|---|---|
Frontend-app-authoring issues and libraries |
| |
Django | ||
Setup.py related maintenance | ||
maintainer-at-large status | ||
(Time permitting) Devstack settings deprecation | https://github.com/openedx/public-engineering/issues/247 Tutor's dev settings are a thin layer on top of its common settings overrides, which are always used with the yaml-based production.py. That means that "rebasing" Tutor's dev settings on to yaml-free common.py actually adds a ton of complex differences between Tutor's prod and dev, not to mention that it would break any tutor plugins that used yaml overrides.a It seems that if we're going to rebase tutor's dev settings onto common.py, then we should do the same tutor's prod settings, and couple it together with a broader yaml deprecation. That does majorly increase the scope of this effort, probably turns it into a next-year project. Curious if you agree or if there's any small-bite work that you think I'm missing. | |
Discussion about a potential new ADR |
✅ Action items
- Next: Maintenance at large
- Next: Other upcoming maintenance - Django, Setup.py
- Kyle McCormick create an ADR about the approach to settings files in edx-platform and how we want to orient them eg. common.py → production.py → development.py or testing.py (desired but not true right now)
⤴ Decisions
- FC work tracking happens in the
public-engineeringrepo.
⏺️ Recording and Transcript
Recording: https://drive.google.com/file/d/1uocF1AeuzOQpr1IbqwQpE9BMgFHzWk3_/view?usp=sharing
| Expand | ||
|---|---|---|
| ||
Maintenance Working Group Meeting - 2024/11/21 09:00 EST - TranscriptAttendeesAdolfo Brandes, Awais Qureshi, Feanil Patel, Feanil Patel's Presentation, Jeremy Ristau, Kyle McCormick, Kyle McCormick's Presentation, Michelle Philbrick, Robert Raposa, Sarina Canelake TranscriptFeanil Patel: Hello. Jeremy Ristau: Go Feanil Patel: refresh that and share my screen. meeting notes are here. I think so far I only have one thing to check in on. I don't know if there's any other stuff we want to talk through. think if there's any maintenance tasks that we should Feanil Patel: First thing on the list is there was a conversation in Slack around the quality of the front end app authoring repo. I think I'm not sure where it was left, but essentially the question was, is this repository actually having more issues or is it just that there's a lot more activity because of all the new content libraries related stuff in there? And so I don't know, I think that conversation was unclear, so people wanted to discuss it here as well. Feanil Patel: What? Thank you. Kyle McCormick: My understanding is that part of the issue is with the issue tracking itself. The fact that we're using repos issues both for task tracking for the content libraries project and for bugs and new feature in new features in the library project alongside bugs and feature requests for existing studio features. So the stuff that the maintainers at 2U would be interested in are a subset of the bigger pile of issues that exist in the repo. and that's kind of like I think broken the team's ability a little bit to figure out what they need to pay attention to. Is that fair Jeremy? Jeremy Ristau: So that's the issue side. Yep. And then of course more activity means more potential issues that arise. Feanil Patel: Jeremy Ristau: And yeah, the root cause is the fact that there is an active work stream going on inside of that repo or I would argue more than one. Feanil Patel: Right. Right. Feanil Patel: Changes happening. Yeah. Yeah. Jeremy Ristau: There's the unit page work as well as the libraries work. And I think the conversation here is sort of appropriate for is that how we want streams to work? it might be Adolfo Brandes: Adolfo Brandes: Would labels or tags help? Feanil Patel: Adolfo Brandes: Do you think there any I'm not sure in that repo… Feanil Patel: Do we use any labels right now in that Yeah. Adolfo Brandes: but I've seen in the past in other repos I do like tag bugs as actual bugs with the red little thing and use that to organize for example the front-end working group board. So bugs are always at the top. Just using an example, right? Feanil Patel: Yeah. Yeah. Looks like there's Go ahead. Jeremy Ristau: I would probably say it would at least from my perspective,… Jeremy Ristau: I think labels would work better in the other direction, which would be leveraging filtering out things that have a label like content libraries v2 and new unit page or something like that. So that the list of what we would need to triage is only the things that aren't tied to an active work stream. 00:05:00Adolfo Brandes: Jeremy Ristau: Yeah. Yeah. Adolfo Brandes: Yeah, that could be done. Adolfo Brandes: It's always going to be some little additional work is going to have to happen on one end or the other, so I guess what's the golden path or… Jeremy Ristau: Yeah. Exactly. Adolfo Brandes: the simplest path like a new naked issue is what usually is created, somebody that comes in, they don't know about the conventions, they just create an issue, right? Okay,… Jeremy Ristau: And then triage happens related to that issue. and… Adolfo Brandes: I see what you're saying. Yeah. Jeremy Ristau: if we have one person triage just to add a label and then another person triage for some other reason, it might be easier when a project team is creating their items as that scope of people is limited. Feanil Patel: Right. Jeremy Ristau: It might be more effective. I don't know about easier, but it might be more effective to have labels applied to that set of work. Adolfo Brandes: Adolfo Brandes: Yeah, if you're working a project, it's fine to add a little rule whenever you add something to that project, right? yeah. Kyle McCormick: We ask our funded contributors to do this for PR titles. Feanil Patel: Right. We also,… Kyle McCormick: So it's not a crazy ask. Feanil Patel: it looks like they did add the ability to filter by project. So, if all of the content library stuff is already on a content libraries board,… Adolfo Brandes: Really? Adolfo Brandes: That would be ideal. Feanil Patel: it might be possible to simply reject all of those, but I don't know if that's the case. Feanil Patel: Feanil Patel: How do you know if there's a right because labels that track projects that might already have a project board might be redundant and… Adolfo Brandes: Yes. Yeah. Feanil Patel: kind of annoying but do you have a board? Kyle McCormick: Right. Yeah,… Kyle McCormick: there is a board. I can link it. I don't know of a built-in GitHub feature for filter by not in projects, but unless I've added it in the meantime. Feanil Patel: I think they have because Let me see. They certainly added a in project. So, we can see if we can do a negative. Kyle McCormick: We can use the minus sign on it. I bet. Okay. Feanil Patel: Yeah, let me see. Yeah, you can do a minus sign project. So, I think this might be a way of doing what we want and it just might be a matter of teaching Adolfo Brandes: just so we discuss it. Feanil Patel: Adolfo Brandes: What's the worst case nuclear alternative create issues in a separate repository entirely? yep. Feanil Patel: for yeah having the per project stuff live in public engineering wouldn't be crazy I think for any active projects and… Feanil Patel: then as a part of closing down an active project you would move it to the relevant library relevant repo if there's unfinished work and it's useful Feanil Patel: Feanil Patel: to keep those tickets or you would close them out and saying this project is not going to finish this way. it's slightly further away from the code… Feanil Patel: but it's also close enough to the people who are actively working on it without bothering the maintainers with aspirational tickets. Yeah. Adolfo Brandes: Yeah. I mean,… Adolfo Brandes: we already have a one big aspirational board and repository which is OpenX proposals, So, it's not like we aren't already doing that in one step removed. Feanil Patel: Feanil Patel: So if we Yeah. Kyle McCormick: Feanil Patel: If we remove the libraries overhaul I assume as the project Kyle. Kyle McCormick: Mhm. Yep. Feanil Patel: Then it goes from 116 to Yeah. Jeremy Ristau: Yeah, it's about half. Yeah. Feanil Patel: It just drops it by half. Feanil Patel: So it's like 55. Adolfo Brandes: From my point of view as a Yeah. Feanil Patel: So that is much easier to work with. most of which are bugs and some release related testing. Kyle McCormick: And some of those are just things that aren't in the project. Kyle McCormick: Sorry, Alpha. Go ahead. Adolfo Brandes: No, I was just going to say from my point of view as an sort of on the ground PR reviewer most of the time as long as there's a link between the PR… Feanil Patel: Mhm. Yeah. Adolfo Brandes: which is an issue in the repo. There's no way around that. with the other wherever it is like the main issue it doesn't matter. Feanil Patel: Feanil Patel: Right. Right. 00:10:00Kyle McCormick: Yeah, going back to why did we come up with that issues live close to the code rule. the people it was so that maintainers could have easier access to the things that are going on with their code and so that devs who are interested in the code could see… Feanil Patel: What's coming? Kyle McCormick: what the problems were. And if this is cutting in the opposite direction,… Feanil Patel: Yeah. Yeah. Kyle McCormick: then I think that's maybe a reason to use something like public engineering for active projects. Jeremy Ristau: Another thing that's popping into my head, you can filter for just no project. and there are things that are popping up from the new unit page work. Feanil Patel: All right. Jeremy Ristau: Right. Adolfo Brandes: Yeah,… Adolfo Brandes: That doesn't have a board that I know. But we could have one. it's fine. we could put it in the front end working group report even. It doesn't like Jeremy Ristau: So that's kind of what I was wondering is would putting all project work in public engineering sort of push toward more conformity in the way that the projects are made visible and managed. you might expect a board perfunded contribution or something like that where you could link from the funded contribution confluence page and… Adolfo Brandes: Yeah, I would be totally fine with that from my point of view. Jeremy Ristau: sort of tie the work to the description. it seems like that might enable a little more traceability. This is certainly a weak opinion that I have, though. Feanil Patel: Feanil Patel: Yeah. Project still even exist. yeah, I think a lot of the OSPRs are still also ending up on The CC coordination board. Adolfo Brandes: What about just let's think edge cases a little bit. Feanil Patel: I guess that's less of them. Adolfo Brandes: Drive by wishlist contributors. Feanil Patel: Adolfo Brandes: Somebody wants something out of a repo and they just go in and create an issue there. That's always going to happen anyway,… Feanil Patel: I think Yeah. Adolfo Brandes: right? Yeah. Feanil Patel: And that's a thing that the maintainer should be dealing with. Adolfo Brandes: Okay, cool. Jeremy Ristau: Yep. Bugs. Feanil Patel: I think it's more I had if somebody has a resource to be working on a project and is going to be both creating issues and bugs. I'm almost wondering if that highle epic should always exist in the repo. for example, if content libraries has an epic currently, probably it exists on the platform road map,… Feanil Patel: I'm guessing. and… Kyle McCormick: Yeah. Yeah. Adolfo Brandes: Yeah, the epic is usually the Feanil Patel: I wonder if it should. And it's a great question. Yeah. Yeah. Kyle McCormick: But does it belong in the back end repo? And if we're working in packages then Yeah. Feanil Patel: Feanil Patel: I don't think it's an easy answer, but that's the thing you would want. if you went to front-end app authoring and you're like, " there's a big overhaul of the unit page." That's the thing I would want to know as somebody walking into that codebase. I would a lot more about that than I care about there is a bug in the teams on the new content libraries. Jeremy Ristau: I think that is… Jeremy Ristau: what the bottleneck of the maintainer should be though, if I walk into frontend app authoring instead of knowing that there's an epic label and going and trying to find that one epic out of all the issues is to just … Feanil Patel: Yeah. Yeah. Feanil Patel: Yeah. Talk to the maintainer. Okay. Jeremy Ristau: if I'm interested in starting to contribute or understanding what the activity is, just now I know who the maintainer is. I'm going to go talk to them. I think we have other channels that help everyone sort of stay in touch about the road map items. Feanil Patel: Yeah. Yeah. Yeah. Fair enough. Jeremy Ristau: Yeah, there could easily also be a link in readmes as well that says for visibility into the active epics go to the platform road mapap and… Feanil Patel: Just thinking through what we're trying to solve for related. Michelle Philbrick: Feanil Patel: Yeah. Yeah. Jeremy Ristau: just that in all the readmes. Feanil Patel: Yeah. Yeah. Yeah. Yeah. Yeah. That's not a crazy idea. Michelle Philbrick: Does somebody have a link to you mentioned another board for the coordination board. Feanil Patel: This is the one that I assume you and Tim are using. I don't know what it's called anymore. Michelle Philbrick: Sorry. The contributions board. Feanil Patel: Contributions board. Yeah. Yeah. Michelle Philbrick: Yeah. Okay. Gotcha. Feanil Patel: But This one doesn't have any sort of fully third party contributions. it just has funded contributions and all of the contributors are on the inside of something. So I was curious… 00:15:00Michelle Philbrick: Feanil Patel: if there were those things because that's also removing anything that has a board idea is what I was trying to figure out if that would work or not. And that I think wouldn't because of that problem but in general filtering out. Michelle Philbrick: Yeah, a lot of I don't think I've seen too many of them… Feanil Patel: Go ahead. Yeah. Yeah. Michelle Philbrick: where because usually all the boards show up under projects. I've seen a few times maybe the coordination board, but not a whole lot. so I don't know how many of them are actually on both Feanil Patel: So I think it makes so based on what you guys are saying, it makes sense to have the highle epics stay on the platform road map. It would be and then Jeremy Ristau: So I think if we go that route and if we keep work tracking tickets FC tickets in repos, I think what we would have to do is we have a runbook for maintenance or… Feanil Patel: Jeremy Ristau: maintainership I guess is the right word. and it has a link to the issues tab, Go here, look at any new ones. I wouldn't want to have to keep changing the link over and… Feanil Patel: Right. Right. Jeremy Ristau: over again. Don't look at that project. Feanil Patel: Yeah. Yeah. Yeah. Yeah. Jeremy Ristau: We would have to say no projects at all. that would remove coordination. Feanil Patel: Right. Right. Jeremy Ristau: That would remove all of the things that are tied to it. Feanil Patel: It's like this is all the rest of the stuff that nobody else is looking at for any reason. Jeremy Ristau: Right. Exactly. Feanil Patel: Which is this stuff here. Jeremy Ristau: So, if we're saying that's what the maintainer should be looking at, I'm fine with that. If there's a concern that we're also going to be putting these things on projects just to make them more visible somewhere else as well and… Feanil Patel: Jeremy Ristau: and then we have to filter them out in our view. there will be some misses, I think. Feanil Patel: Yeah, I think in my mind the maintainer should be looking at this plus any open source contributions that don't have a like that came in sort of off the street as it were,… Feanil Patel: anything where it was a public contribution that isn't part of another project but somebody had provided a bug fixer and those would go on the contributions board. So it would be like this plus the contributions board is I think what I would want maintainers to look at. Jeremy Ristau: Is there a way to write that as a linkable filter? Feanil Patel: Yeah, that's a great question. Let's see. Jeremy Ristau: I don't know if you can do ors inside of the filter bar like sub ors. Feanil Patel: Feanil Patel: Yeah, I forget. Jeremy Ristau: Maybe you're talking Kyle,… Feanil Patel: Let's see. Jeremy Ristau: but I can't hear you. Kyle McCormick: Kyle McCormick: I also don't think that R is for fun and… Kyle McCormick: contributions would be going on. Yeah. Feanil Patel: And issues. Yeah. Yeah. Yeah, that's true. It wouldn't be an issue. So I think this would be fine for looking at issues and then I don't know but looking at PRs you'd kind of want yeah I think so… Jeremy Ristau: presumably it's the same Unless they're like,… Feanil Patel: although I think most PRs are not going to be in projects so that doesn't help us because the PRs are probably linked to issues that are in projects at best but I don't think people so the open source contributions those are all pulled Jeremy Ristau: are they pulled into a triage board or something? And board is a project. Feanil Patel: into the contributions So if we did that, that would be all of the Right. Jeremy Ristau: So yeah, Okay. Feanil Patel: So this is everything that is a public contribution. Some of these have FC labels. So we could do a minus FC So, this is showing things that have no project or are on the contributions board and don't have an FC label in theory. 00:20:00Jeremy Ristau: I'll toss it out there that I understand that now that we went through a 20-minute conversation about it. as the person who maintains one repo, but there's a lot of other repos with a lot of disperate maintainers. this is going to be a tricky thing to get right… Feanil Patel: Feanil Patel: Right. Yeah. Jeremy Ristau: if you really have activity all over the place. Feanil Patel: Yeah. Yeah. Jeremy Ristau: You're again muted, Kyle. Kyle McCormick: Why can't it just know? Kyle McCormick: I feel like we're moving away from simplicity in this. We're putting complexity on complexity. I think there's GitHub projects are cool, but let's be honest, they're complicated and confusing. GitHub issues, just like that list of issues on a repo is pretty straightforward. It's a list. They're open or closed. You can put labels on them, right? They're very accessible to maintainers. I like the idea of that being the backlog for maintainers. Feanil Patel: Kyle McCormick: Kyle McCormick: I think that's a really simple thing to say. Your issue tab is your responsibility. So for that reason, I'm leaning towards using a separate repo for issues that are not the maintainer's responsibility. Feanil Patel: Yeah, that makes sense. Kyle McCormick: Pull requests are another story because we have to create the pull requests in the repository. Feanil Patel: Yeah. Yeah. Kyle McCormick: We have a complicated… Kyle McCormick: but working system in place for that with all the open source contribution and funding contribution labels. Feanil Patel: Yeah. Yeah. Feanil Patel: Yeah. That set of labels I think would work for filtering out pull requests. but yeah, I'm not opposed to that as a decision, which is that projects should not create issues in the repos… Feanil Patel: if they're going to be actively worked on. Feanil Patel: Feels weird to say. Kyle McCormick: Project should not create issues in repose. Kyle McCormick: Can you say that again? Feanil Patel: I think what we're saying is if active development projects not run by the maintainer should manage their issues elsewhere. Adolfo Brandes: Why exclude the maintainer like… Adolfo Brandes: if they're working on a new feature? Feanil Patel: That's a fair question. Feanil Patel: Maybe it doesn't exclude the FC work happens in public engineering. That's not Yeah. Kyle McCormick: It's a nice rule. Kyle McCormick: Yeah, I'd say if the maintainer wants to use issues in their own repo, that sounds like their prerogative. If you're saying that the issue tab is their backlog and they want to put items in their backlog, great. I'll probably do that in the repos I maintain Feanil Patel: Sarina Canelake: Yeah, this seems like I mean I'm just sort of listening,… Sarina Canelake: but it seems like the confusion is maintainers not necessarily understanding what issues represent or being able to filter them. And if maintainers are making their own issues in their own repos, they presumably know how to manage those issues. Feanil Patel: Yeah. Right. Feanil Patel: Yeah. Yeah,… Jeremy Ristau: Yeah, the burden that or… Jeremy Ristau: least the burden that we're suffering from is every week the maintainer on call changes, but on a random Tuesday there might be 12 new content library tickets created because the team discovered identified a whole bunch of things that they wanted to do,… Feanil Patel: they did some grimming, right? Jeremy Ristau: right? And so now you've got 13 new items that you just need to figure out what to do with. If that happens twice in a week and you only triage issues once, you might have 20 or 30 new issues that you have to figure out are all related to this project or that project or when there's more than one happening at a time,… Feanil Patel: Jeremy Ristau: it really starts to get confusing. But yeah,… Feanil Patel: All right. Jeremy Ristau: if 2 UTNL is the maintainer of authoring and 2UTNL decides that instead of Jira, we're going to use GitHub, they should be able to put the backlog of known issues and known work inside of the issue backlog. That does make sense to me. Yeah. Feanil Patel: Yeah, I mean I think if we just move the FC work out that should sort of simplify a lot of this and we can sort of iterate on it further if we need to. 00:25:00Feanil Patel: So this I think probably needs to be communicated to product working group at least anywhere else. Michelle Philbrick: Michelle Philbrick: I sorry can I ask one clarifying question and I don't know Jeremy maybe this is a question for you but for FC projects there's a ton in authoring right now with open craft and… Kyle McCormick: All right. Michelle Philbrick: everything like that. Do those all require to you TNL signoffs for FC projects or no? and is that the blanket rule for all FC projects? Okay. Jeremy Ristau: No. So, core contributors have the same, write or merge access that maintainers would. So, as long as a funded contribution contains a core contributor, that's generally how all the work would funnel through. Yeah. Michelle Philbrick: Michelle Philbrick: Okay, that's great… Feanil Patel: Yeah. Yeah. Michelle Philbrick: because then I don't have to bug you guys. Feanil Patel: Yeah. Yeah. Yeah. Yeah. You don't. Michelle Philbrick: Yeah, All right. Thank Feanil Patel: Yeah. Yeah. Yeah. Okay. Serena, as a person who's often in both the product meeting and this one, could I ask you to transfer this information over to that working group? Sarina Canelake: Yeah, I think this probably also needs to be said during an AXM meeting… Sarina Canelake: since we're probably the ones doing that. but yes,… Feanil Patel: Yeah. Yeah. Feanil Patel: Yeah. Yeah. Sarina Canelake: I can bring this to that. Feanil Patel: I can put it in our parking lot today for the ax and stand up and that'll cover us. Sarina Canelake: Yeah, I think that makes sense. Kyle McCormick: Yeah, I can retroactively fix this for the content libraries project. Jeremy Ristau: Thank you. Feanil Patel: Nice. Sarina Canelake: Yeah, I'll fix this for my projects. Kyle McCormick: Sorry, one follow-up question. we talked about tickets that are like we need to do X work for our funding contribution. What about bugs that come in that are particularly related to the testing of funded contribution? Can those be repo bugs or do those need to be public engineering bugs? Jeremy Ristau: Jeremy Ristau: This strikes me as very similar to when we have epics and we tie them all to an epic ticket and then at some point we want to close the epic but we have some lingering tickets around. generally what I would do is I would keep all the tickets that are found during an epic life cycle tied to the epic until we're ready to close it. Kyle McCormick: Jeremy Ristau: and then when we're ready to close it, I would remove the epic label from the open tickets and they would just become backlog items. So I guess I'm proposing that same approach of they live with the funded contribution work until the funded contribution is done then they can migrate to the repo. Kyle McCormick: Mhm. Yeah. Kyle McCormick: Okay, that sounds good. I think there's going to be some amount of scenarios like this where a test release tester finds a bug, creates it in the repo that they have the best knowledge of. They're like, "This is a front end app authoring issue. I'm going to make it there." And then someone either you or someone on the contribution team says, " this belongs in public engineering. I'm going to move it now." I think that's going to be hard to get away from with our current testing process where the testers don't know, don't have 100% of the context. Feanil Patel: Feanil Patel: Yeah. Yeah. Jeremy Ristau: Yeah, I think that,… Jeremy Ristau: I think that's okay personally. Yeah. Feanil Patel: Yeah. we're always going to be triaging and rearranging and everybody's going to have to contribute a little bit to that based on their knowledge. so Yeah. Sarina Canelake: Yeah, I can imagine a number of edge conditions like the FC team finds a bug,… Sarina Canelake: but it's not directly related to the FC and they're not sure they're going to get to it in the FC, so they file it as a repo bug, but then eventually the FC project decides to pick it up. communication on the tickets is always going to be the best bet. Feanil Patel: But if we can keep all of the active workstream issues out, that will hopefully reduce noise enough that on call person can do a much better job triing of incoming bugs. Okay, we're at 9:30. So that is the end of the general maintenance meeting. I had some conversations I wanted to have about maintenance stuff that maybe we can put for next time. 00:30:00Sarina Canelake: So, I think everyone on this call is in the US and Adulo is already taking next week off, but we'll meet in two weeks. We're not meeting next week. Feanil Patel: Feanil Patel: Yeah. Nobody's meeting next week. If anybody's watching this recording, so that let's talk about that in two weeks. and then we're going to switch over to Edex platform specific conversations. If you want to stick around for that, feel Otherwise, if you want to disappear, feel free to do that as well. Feanil Patel: Thank you, for removing the meeting on the calendar so it's not confusing. all right. Sarina Canelake: Yep. Sarina Canelake: You guys. Happy Thanksgiving. Feanil Patel: Everything's saving,… Robert Raposa: team as well. Feanil Patel: All right,… Michelle Philbrick: Bye. Feanil Patel: Kyle, start us off. Kyle McCormick: Yeah. … Kyle McCormick: this is what we were going to talk about later today, Fel, but I figured may as well do it while we're here. Feanil Patel: Dead. Kyle McCormick: Yeah. So, the dev stack deprecation. I have been trying to make a base settings file in X platform that would be usable by both tutor and devstack. One which does not make the assumptions it makes could be used by any development environment. That's the guiding principle. as I've done that, it seems more and more like making just that file and making it not depend on YAML configuration would in addition to being a breaking change for 2U,… Feanil Patel: Thank you. Mhm. Kyle McCormick: it would be a huge breaking change for tutor and one that would I think make tutor more complex rather than less complex. The reason being that the way tutor currently structures settings is it uses production.py which has a YAML file. So all tutor settings are based use a YAML file and then it builds on top of that production settings and dev settings. So if we get rid of the YAML necessity for dev settings then you have this entirely separate configuration path for dev but it also still has to use the ammo file for prod. Kyle McCormick: And now all the plug-in points, the patches that people could put into their YAML files and into those different tiers of the settings files, everything related to dev is broken in a way that's kind of hard to understand and explain to people. The upshot of that is I think we should either deprecate all YAMLbased config in one go or… Feanil Patel: Kyle McCormick: just put this on ice for now. Feanil Patel: Can you say a little bit more about… Feanil Patel: how tutor plugins rely on overriding YAML specifically? Feanil Patel: Is there not an interface somewhere in tutor for update setting that handles that and… Kyle McCormick: Right. Right. Kyle McCormick: So, tutor, let me find the plugin points. Feanil Patel: feel free to take over the screen if you want to share Kyle McCormick: Sure, we'll do that. Can't wait. Feanil Patel: Kyle McCormick: So tutor exposes these end patches for patching the YAML file and… Feanil Patel: Okay. Yeah. Kyle McCormick: it also exposes these settings patches and… Feanil Patel: The settings. Yeah. I see. So, you'd have to deprecate those end patches essentially. Kyle McCormick: you have to deprecate them and I guess this is assumption I need to check, but I think they're there for a reason. I think they're there because there's some things that you have to do in YAML config. Otherwise, I don't know… 00:35:00Feanil Patel: Yeah. Right. Kyle McCormick: why Reus would have introduced those patches because he doesn't like YAML config. right,… Feanil Patel: That would be good to understand better because if there is a limitation that we would be experiencing by dropping this, we would want to know about that sooner rather than later. Feanil Patel: Feanil Patel: I'd be surprised that there would be a limitation. It might be that certain things are harder to render because there's a bunch of code in production.py that reads those settings and then generates new settings. Kyle McCormick: that's almost definitely it. Feanil Patel: I know. Kyle McCormick: Yeah. right? Feanil Patel: Yeah. the celery stuff is in particular super egregious in that it fully renames the celery cues based on a convention that is fully made up and inside of production. Feanil Patel: Feanil Patel: and nowhere else. But there might be other things in that vein. Feanil Patel: Yeah, there's a lot of updates that happen and then the name used to change. I forget if it's still in there. there's some complex logic around the Q names. Kyle McCormick: Yeah. … Kyle McCormick: there's definitely some logic going on here. Code jail. Ever anywhere where these code blocks in production. are indicative of things that depend on the YAML file in a way that… Feanil Patel: Right. Right. Kyle McCormick: if you were to get rid of the AML file, you'd have to replicate this logic or some version of it in your downstream settings file. Feanil Patel: Okay. Yeah. Feanil Patel: Feanil Patel: So perhaps the things to do is continue to allow for YAML and then we can iterate on moving things into essentially the cleanup we've talked about so the previously… Feanil Patel: which is cleaning up production.py so it only contains that logic and not like a bunch of the redundant overrides can still be done and… Kyle McCormick: Mhm. Yeah. Feanil Patel: that's independent of all the work you're doing right now. So what would that mean is a development.py that still allows for YAML importing is the upshot of if we decided to keep everything not break everything today. Kyle McCormick: I think it would be very similar to devsack.py. Feanil Patel: Kyle McCormick: The things that tutor does on top of devsack.py is pretty simple. Feanil Patel: Yeah. … Feanil Patel: is it just a matter of leaving devstack.py Right. Kyle McCormick: Let's see this is the extent of tutor's changes to devsack.pai Y that it's not already changing for production settings. Yep. Feanil Patel: Yeah, it's pretty small. Feanil Patel: A lot of that looks very similar to what's in the minimal config.yamel honestly it's a lot of making sure that the root paths are correct and then Okay. so the question on the table is dropping YAML worth us taking essentially a year-long deprecation embarkment right now should we embark on this year-long deprecation to get rid of YAML is that worth it or do we just let Feanil Patel: Feanil Patel: Do we want to just make improvements and simplifications that are not breaking but continue to allow YAML overrides which in spite of my purest desires feels like the more correct route of action? Robert Raposa: And… Robert Raposa: when you say is it worth it, can you just remind me what the benefits are to the community given that I'm guessing to you we'll continue to use it, and… 00:40:00Feanil Patel: Yeah. Yeah. Robert Raposa: we'll find a way to do this. Feanil Patel: Yeah. Yeah. Feanil Patel: Robert Raposa: Feanil Patel: Right. The benefits are that … Feanil Patel: when we get to the point where everything is inside of a y file and operators are essentially maintaining their own settings.py file that imports the development one or the production one and then adds their overrides on top of it. The story for how to explain how things work and the interpretation of how things get set is much simpler. so it simplifies operations but it also allows for more powerful configuration because for example right now there are certain things that are not possible without adding complexity to the edex platform code around if you needed to reference objects in your y file which certain parts of the system expect you would need to pass in essentially a class path and then have the production.py Feanil Patel: PI file know that setting is currently a string needs to be converted into an object that reference that is mapped by that string. and so things like that mean that when you introduce those kinds of settings or when you want to make use of those kinds of settings you need to add new code to the core platform. So it sort of reduces our pluggability story over a long period of time. I think not enough that it's worth the level of lifting we're talking about here, at least but in the sort of long-term goal that it aligns with is fewer people have to touch EdX platform to make the changes they want to make. This is one of the places where people need to touch to be able to introduce new settings of certain kinds right now. Feanil Patel: Feanil Patel: But that might be a small enough subset that maybe it's okay for us to pay that cost for a couple of years… Kyle McCormick: Feanil Patel: until this becomes a different priority. Jeremy Ristau: Yeah. Good. Kyle McCormick: My agreement in favor of slightly different our configuration system is just egregiously complicated. Kyle McCormick: I encourage anyone here to dig through all the layers. choose a setting and figure out what are all the ways this can be configured and just follow the path. It's 10 layers for the simplest setting. It's like really bonkers. I think it makes this really high barrier to entry for new developers. It's a pain for maintainers. I remember at Edex a lot of bugs were the result of it. As someone in the open source community, I see just hours of discussion lost to disagreeing about… Feanil Patel: Yeah. Suspended. Kyle McCormick: what the default configuration is because tutor is this configuration on top of the edex default configuration because the openex default configuration is not viable. Just things like that where it's I know I'm saying a lot of superlative words. I'm saying really and egregious and all that, but if it's just me then it's probably not worth it. But I have this sense that we are dealing with this every day. Robert Raposa: I'm going to go ahead Kyle McCormick: We are dealing with this crushing amount of configuration complexity and the only way to chip away the only way to make it better is to remove layers rather than add documentation and shims on top of what we have. Jeremy Ristau: Jeremy Ristau: I was just going to say this feels a lot like the call for proposals that the product working group did recently and then sort of weighing them against each other for what might get put into a roadmap. there might be a need to put this out into the community to see if people agree enough to warrant prioritizing it even though it's going to be a large effort. if it is a barrier to entry or if everyone is just absorbing hours of pain every couple of months or every couple of weeks or if people wish they could spin up the open edex project using default configuration and they can't like things like that like finding out through the whole user community whether or… Feanil Patel: Yeah, it's a good point. Jeremy Ristau: not this can't be pushed down the road would be helpful I Robert Raposa: Kyle McCormick: Yeah. Robert Raposa: And… Robert Raposa: the one thing I was also curious about, you don't have to get into the details here, but of you mentioned it would be great to have one less layer, and help simplify things. And I'm just curious if it will turn out to be one less layer or if it will turn out to be one more layer. 00:45:00Robert Raposa: in terms of if everyone still needs to plug in the L in some other way is will things get less complex or… Feanil Patel: Robert Raposa: will things get more complex because it's like here's this whole complex thing that is very difficult to understand and you're still using that plus you also have this other method as well and I don't Feanil Patel: I think it pushes the complexity onto the people… Feanil Patel: who are choosing it as opposed to defining it as a requirement to use the platform. the ideal would be that edex platform would have a settings file that you could run for development without needing to understand any of this stuff. and that you could quickly modify. And then if you had production systems more complexity, that is complexity you would be taking on as an operator. Robert Raposa: Are you imagining that this would only really go forward… Feanil Patel: Robert Raposa: if tutor could remove its reliance on YAML? Kyle McCormick: Yeah. Yeah. Robert Raposa: Okay. Yeah. Kyle McCormick: If no one can viably run the platform without YAML, I wouldn't remove Y. Feanil Patel: Yeah. Then yeah,… Kyle McCormick: That would just be a bad way to spend time. Feanil Patel: right. Yeah. the other thing I'll say, Kyle, is that even if we don't drop BML support, we can still currently make a huge amount of progress by getting rid of we currently have one,… Feanil Patel: two, three, four devstack prefix settings files and a Docker-Prouction PI file. a lot of things can be removed and cleaned up that simplify our settings today, even if we continue to have YAML support. Kyle McCormick: You're right. Kyle McCormick: Yeah, totally. Feanil Patel: Kyle McCormick: I would like to do those things. I have a couple PRs to remove some of the noops from production.pay. I've been back burnering them in favor of the change we agreed to in the deber. Feanil Patel: Yeah. Yeah. Kyle McCormick: But if we're not doing that,… Feanil Patel: Yeah. Yeah. Kyle McCormick: then yeah, do the small stuff. Feanil Patel: And rather than I think one of the goals of completing this work there should be one it may still be worth it to rename it just to have sort of clarity and step-wise change and we can find where people are relying on it and update it of moving from devstack… Kyle McCormick: Okay,… Feanil Patel: which is a loaded term to development.py I which is a clear sort of generic in comparison. Kyle McCormick: So, I think what I'm hearing is a path forward is incremental improvements, turn pay into which hopefully for you folks to you be pretty easy to base your devstack settings on if you would like to remain in sync with development.py I'm happy to work with USA on that or whoever and then put up a call for discussion goal for proposals on I guess validating this validating or… Feanil Patel: Kyle McCormick: invalidating this feeling that it's worth a long effort to radically simplify our settings. Feanil Patel: I guess the other question I have Kyle is currently devstack.py builds on top of production.pay… Feanil Patel: which sort of adds to the level of complexity. Kyle McCormick: Yeah. Right. Feanil Patel: Is that a thing you would retain or is that a thing you would change Right. Kyle McCormick: I think there are some different philosophies here. one philosophy is you have your common settings which are basically like you do your best to make them good and then your production settings make those into a real production system and then your development settings are the dev version and you don't Yeah, that's a long way of saying a v you common production development another philosophy is your production settings are your defaults and… Feanil Patel: Yeah. Yeah. Yeah. Yeah. Yeah. Both depend on common Feanil Patel: Kyle McCormick: Then your development settings are a refinement of those that remove all the production stuff. And the production settings would break unless you apply run them with a set of secrets. Feanil Patel: Right. Yeah. Feanil Patel: And that's a philosophical question. and… Kyle McCormick: I tend to like the production base. Feanil Patel: also common production dev as a sequence or Kyle McCormick: No, I tend to prefer production are the base settings and… Feanil Patel: Feanil Patel: Got it. Kyle McCormick: development are a refinement of that kind of forces you to be like this code is meant to run in production and… Feanil Patel: All right. 00:50:00Kyle McCormick: we will tweak it so it runs on our local machines. And I think that's a good philosophy and… Feanil Patel: Got it. Yeah, that makes sense. Kyle McCormick: also aligns with the fact that production of pi is actually the base of all the settings right now. Feanil Patel: I Yeah. Yeah. Yeah, production and common probably need to be more combined instead of trying to common. Kyle McCormick: Yes, except that testpi testing.py is based on common.py… Feanil Patel: Right. Yeah,… Kyle McCormick: which is something we could change. Feanil Patel: we could make testing also be based on production. Kyle McCormick: That's internal. Mhm. Feanil Patel: And then switch to optimizer testing which might be useful. cool. Feanil Patel: I think not everything is my favorite, but that is not a crazy set of things to do. And then yeah, I think honestly if we have six fewer settings files, even if they behave like they currently do, it sort of simplifies looking at which one is getting used. and that can sort of help with a lot of the delayering because once you're in a Python file, it is a lot easier to continue to navigate up and down Python files. Feanil Patel: Feanil Patel: But knowing which settings files is the one to start with is actually kind of hard today… Kyle McCormick: Right. Feanil Patel: because devstack with workers devstack docker I think… Robert Raposa: And the discussion that you just had about testing Feanil Patel: if people want to make some of those changes those should live in their personal YAML files and not in the platform and getting those out of the platform will be useful. Robert Raposa: should testing be off production or not? and whether dev stack or development is off of production and why just having a clear description of… Feanil Patel: Robert Raposa: how things work and if they work more consistently. I think that it also help people navigate few things and it just makes what we have more versus Feanil Patel: Yeah. Yeah. Feanil Patel: Do you have an ADR already about this, Kyle, that you were going to land? Kyle McCormick: No, I don't have one. Feanil Patel: It may be worth it as you this is a good decision that's worth recording I think that we just discussed and… Kyle McCormick: Yeah. Yeah,… Feanil Patel: it may be worth adding an ADR as a part of this change. Sorry Robert I cut you off. Kyle McCormick: do that. Yeah. Robert Raposa: You're sort of seconding… Robert Raposa: what I'm saying within Night. Feanil Patel: Yeah. Yeah. Feanil Patel: Yeah, because I think we're making a bunch of nice decision we're both making decisions and vocalizing the decisions that have already been made and it would be good to write them down. Kyle McCormick: Can you give me some action items in the doc now? Feanil Patel: Yeah, I can do that. Kyle McCormick: Thanks for bearing with us as the plans have changed in the dev techer. Jeremy and Robert Robert Raposa: We are running low on time… Jeremy Ristau: Same. Robert Raposa: but two potential topics that could be quick or could be deferred. one is that plug-in conversation we were having in the ADR around I guess as far as edex platform is concerned. Robert Raposa: I created a new issue that I find. Yes. Kyle McCormick: I think I can summarize while you find it. Kyle McCormick: We have two ways of configuring signals. One is the Django plugins way where it's like a string dictionary in your P kind of like a thing we invented several years ago. the other way is the Django way but using open edex events as the definitions of the events and Robert realized that we have these two ways now and the Django events the open events ways looks a lot nicer and a lot less weird. Robert Raposa: Kyle McCormick: So could we get rid of one of them? And I fully support Robert Raposa: Yeah. and… Robert Raposa: but that sprung something highly related but that I wasn't aware of that Kyle had mentioned around this was that at one point in time it sounds like according to Kyle so I'm going to repeat what I think you said and then you can correct me is that there was the idea that we would be dog fooding in capabilities by using those in capabilities inside edex platform where we don't actually have plugins yet because they're already in edex platform and that they would in theory make it easier to extract them as plugins when we want them to actually be plugins. 00:55:00Robert Raposa: Robert Raposa: Yes yes yes. Kyle McCormick: particularly Django plug-in framework stuff,… Kyle McCormick: not openics filters and events. Robert Raposa: And to me that just looks and is confusing and happens to use what you just described, in terms of this weird signal interface for plugins. that is what's going on here? I was trying to figure out if it was somehow defining an API for other plugins. it didn't make sense to me because it's not a plugin. so I created this separate ADR which I just posted in an issue in edex platform for potentially creating an ADR. We all agree about this. Robert Raposa: So this is just an form partially we've got a separate edex platform road map that includes things for making things more clear this… Feanil Patel: Yeah. Yeah. Robert Raposa: since I feel like I generally know a decent amount and this confused me it probably Yeah. Kyle McCormick: It's a good lit test. Kyle McCormick: If it confuses you, it's probably confusing. Robert Raposa: So this might be something. Does it make sense for me to drop this issue on that road map issue? If I can find the road map issue. Kyle McCormick: Go for it. Robert Raposa: And then I can figure that out later. the only other thing which we only have one minute left. So this is just quick inform is there's some open issue around edex platform logging format and… Feanil Patel: Robert Raposa: I guess it's like the log formatting is all in code so you can't actually do anything with it. and we may want to do stuff with it but I'm just punting on it… Feanil Patel: Great. Yeah. Robert Raposa: because you can't. and so there's a question of when and if someone will want to actually invest in changing that, but it seems like a smaller scary version of your whole settings conversation changing the way things But I think there is an existing issue already from MIT about it. Feanil Patel: Yeah. this madness. Kyle McCormick: Wait, hold on. Feanil Patel: It's Yeah. Kyle McCormick: Yeah. Right. Yeah,… Feanil Patel: Feanil Patel: Yeah. no. It's gotten bigger since I was here last. Feanil Patel: Cool. Yeah. Robert Raposa: … Robert Raposa: so I might actually pull that issue just onto the road map issue just so it can come up as a later topic when we look at things. Awesome. Kyle McCormick: definitely do that. Thanks, Good. Feanil Patel: Yeah. The login config stuff should not be in code and there should be reasonable defaults that are not what we have. Kyle McCormick: I'll add this to my corpus of complaints about configuration. Feanil Patel: Yeah. Yeah. Kyle McCormick: That's good. Feanil Patel: Because this is defined here and then loaded into the end files if I remember correctly. Feanil Patel: Feanil Patel: And then I think munched in certain parts of production.py for fun. Kyle McCormick: You Jeremy. Feanil Patel: right. yeah. Robert Raposa: Can you just remind me… Robert Raposa: how to find the edex platform road map issue? Kyle McCormick: I'm gonna jump over to all hands. Yeah. Feanil Patel: Platform mapap is the repo and there should be a project with the same name. Robert Raposa: Got it. Robert Raposa: So it's not in edex platform. Feanil Patel: Yeah. Yeah. Robert Raposa: Okay, great. Thank you. Congratulations. Feanil Patel: Yeah. No problem. All right. Bye. Meeting ended after 01:00:07 👋This editable transcript was computer generated and might contain errors. People can also change the text after it was created. |