Maintenance Working Group Meeting (2024-02-08 08:35 GMT-5) - Transcript

Attendees

Adolfo Brandes, Feanil Patel, Feanil Patel's Presentation, Felipe Montoya, Jeremy Ristau, Kyle McCormick, Maksim Sokolskiy, Max Frank, Michelle Philbrick, Navin Karkera, Robert Raposa, Sarina Canelake, Tim Krones, Tobias Macey, Xavier Antoviaque, Yagnesh nayi

Transcript

This editable transcript was computer generated and might contain errors. People can also change the text after it was created.

Feanil Patel: Okay.

Feanil Patel: Hi everybody. Welcome to the second meeting of the maintenance working group.

Feanil Patel: We've got a couple of different things to cover. I'm going to start to do from last time. Jeremy do you have new information on u up the Ubuntu upgrade. and the edx platform tests infrastructure

Jeremy Ristau: I asked about it. I'm not totally sure. I would call it an update per se. I have confirmation that previous you want to upgrades or dealt with as a collaboration between internal teams inside of 2u so there's infrastructure upgrades. We got to go to SRE for that but then there are code updates those can be done and have been done by the bomb teams historically…

Feanil Patel: Yeah.

Jeremy Ristau: if tests or test infrastructure needed to be changed that would be done waiting. Obviously. All those teams are heavily Consolidated now, so I get any information about what would need to change. We don't know that because we haven't looked into it, but I also don't have confirmation that anyone to upgrade would actually be prioritized by to you right now. So.

Feanil Patel: okay.

Feanil Patel: right

Jeremy Ristau: I think I can say I know what we've done historically, but I don't have any Clarity on what we'll do going forward.

Feanil Patel: there's sort of two places where I think that. We sort of need help is I don't think we need necessarily to you to do an Ubuntu upgrade but the infrastructure that runs the tests is still I think maintained by to you. So either helping us get that updated so we can test two things that are more which is the mango upgrade in the Ubuntu upgrade itself the mango upgrade I think being the more urgent of the two and hopefully the smaller lift.

Feanil Patel: Maybe we can Target that and understand sort of what it would take to get that over the line.

Jeremy Ristau: And are you referring specifically to the Cyprus test infrastructure or something else?

Feanil Patel: No, no, so the GitHub actions Runners for the edx platform tests are created and managed by to you at the moment.

Jeremy Ristau: Okay, and

Feanil Patel: and the version of Mongo on there is controlled by building that container somewhere Onto You infrastructure. So in order for that version of Mongo to change currently we need to use help.

Jeremy Ristau: Okay, so I can definitely confirm that the Mongo 6.0 upgrade is in our list of upgrades to do. It hasn't been fully prioritized yet…

Feanil Patel: Okay.

Jeremy Ristau: because I think we just decided recently to skip Mongo 5 and…

Feanil Patel: Yeah.

Jeremy Ristau: go straight to Mongo 6 instead rolling upgrades.

Feanil Patel: Yeah.

Jeremy Ristau: So I think we're sort of in the phase of planning for it, but if Help is needed to upgrade Mongo. That's something we definitely will offer.

Feanil Patel: Yeah, I mean in particular the thing I think we need help with is the current test Runners. I think only have Mongo 4 on them. And so if they had both installed or both.

Feanil Patel: that's a couple more people if they had both installed or if there were two set of Runners so that we could test with both that would I think identify a lot of issues and let us give us a lot more confidence. So I don't know how big a lift that is, but that I think is the thing to prioritize.

Robert Raposa: and finale I'm just curious I don't know the config for the runners that you're discussing is that private is that okay?

Feanil Patel: I think yeah, so historically I know to you has been managing that so I don't know the full details, but I think the runners themselves are run on sort of two you containerization infrastructure somewhere and then the config for that. I believe that's part of why arkbomb had admin access on adex platform was to manage that config.

00:05:00

Feanil Patel: that said I think that I looked at that code recently and I think that it can become less complicated by using their GitHub actions to set up Mongo that I used in some of the other tests, but it's hard to set them up on the Ubuntu machines that we're currently using for edx platform for the unit tests. So I think there's a way off of this if that's a thing that's desirable, but we need to sort of think about that. before cup

Feanil Patel: we can't hear you.

Kyle McCormick: Bye Cool, the tests are only run on to infrastructure.

Feanil Patel: Yeah.

Adolfo Brandes: That's good.

Kyle McCormick: If they're against the open edx or edx if they're in the open adx or edx private Forks of that If you open a PR against your personal Fork of X platform. it will run the same tests on GitHub actions like Really donors.

Feanil Patel: Okay,

Kyle McCormick: I know that was done for tests time reasons you could Get the test time down by running the test themselves. I don't know if there are more differences. So that is something we should check out because maybe we just switch to the GitHub action Runner version.

Feanil Patel: Yeah, I looked at that and it looked like there was a bunch of stuff installed by default on the edx platform Runner. So I wonder if there's different I might have not looked at all of the GitHub actions workflow files and maybe they're separate workflows that run depending on which is available. yeah, okay, so

Feanil Patel: Yeah.

Feanil Patel: cool Felipe do you have any updates Jeremy? I'm gonna mark this done for now.

Feanil Patel: unless you want to sort of do more follow-up, but I think that there's maybe a different open question about the Mongo to Mongo six update which we can sort of have follow-up conversations on

Tobias Macey: Just as a side note to the Ubuntu conversation for the deployment that I'm running. We're currently actually using the Upstream python containers running against Debian Buster and that's been stable for us just as a anecdote not necessarily advocating for that but in case that's useful to anyone.

Adolfo Brandes: That's interesting. I ought to bring this up to hajis. So we don't necessarily need. Ubuntu

Tobias Macey: Correct. Yeah,…

Adolfo Brandes: Okay.

Tobias Macey: we were able to build our images using Debian Buster. We're not using every feature of edx app. So, I don't know if there are elements of coverage that we're missing in that but we've been able to successfully build and run our instances and production for a number of months and Debbie and Buster from the Upstream python container image.

Adolfo Brandes: And What does that give you over using Ubuntu longer support? What?

Tobias Macey: It's just more in line with the rest of our infrastructure. So we use Debbie and as the base Ami for all of our other applications, and so it's just consistency. Yeah, just not coupled to the Ubuntu ecosystem and…

Adolfo Brandes: Okay.

Tobias Macey: then also because we're using the Upstream python Docker images. It just reduces the number of steps required to build the image. So we don't have to pull in the dead snakes BPA, etc, etc.

Adolfo Brandes: Yeah, that's valuable info.

Adolfo Brandes: Thank you.

Feanil Patel: It sounds good. Did tell me did you guys get coach jail to work with that setup for edex platform?

Tobias Macey: So we are actually using the external code jail service. And I think we're building that against Debian as well.

Feanil Patel: got a

Tobias Macey: Let me find that.

Tobias Macey: it looks like that we're using the Ubuntu container for I don't know exactly why we didn't go with Debian for that. But I think we just at the time just did a wholesale copy paste of What other folks are doing?

Feanil Patel: and then

00:10:00

Feanil Patel: that catchment and for that you just have a code jail or you have an app armor profile associated with that code jail. image

Tobias Macey: Yep. Looks like we're using sudoers actually.

Tobias Macey: Yeah, so we have a pseudoers file this that we're using inside that image.

Feanil Patel: Okay, awesome. Let's see. Yeah, Felipe. Do you want to check them in with you for a second on the commerce testing? Did you guys get a chance to talk about that?

Felipe Montoya: Yes, so I talked to the teams in an Erics that are using that and one larger team would rather get rid of commerce completely but they are using Enterprise and the time we briefly spoke out if Enterprise was to continue more forward without a Commerce how that look like and they rather like that vision. Try to help on the Enterprise side to remove agencies on the comments.

Feanil Patel: Okay.

Felipe Montoya: The modern game could devote some of our amount of equipment when we're putting it two hours a week or something to just keeping the lights on mode. So maintenance tests that they operates in completely recover and that's it. But at least not without the customer that they work for. Also committing to eCommerce they couldn't take more resources in that. But something's more like towards the big then definitely they can do it and just require.

Feanil Patel: I think we will need that where the moment so maybe you could put me in touch with them or have them post something in the forums. Whoever the right technical contact is over there.

Felipe Montoya: Yeah of those.

Feanil Patel: awesome

Felipe Montoya: or both actually Yeah.

Feanil Patel: Probably both eventually.

Feanil Patel: cool

Feanil Patel: Yeah, and I think definitely Jenna and I would both be interested in talking more about what it would take to. Extract the dependency on commerce from Enterprise if that is. A thing you guys want to help? support and pursue

Feanil Patel: All right, the next two things on here are updating the maintenance spreadsheet with a new column. How did a quick run of the old pilots and put that data in there? I have not yet gone through the catalog info files and started automatic automating that in there yet. So that's still outstanding and then I have posted an update to discourse about the change in requirements we discussed last time but I've not yet hosted a new.

Feanil Patel: A new list of repo seeking maintenance because I've been waiting on some other people to provide me lists, and I think I have most of that now, so hopefully we'll be able to do that soon.

Feanil Patel: So that's Felipe go for it.

Felipe Montoya: Yeah, so if we're ready when we've been discussing the whole weekends what we can do in terms of Maintenance. Should I update you directly on the reposphere to maintain their maintenance industry both or wait for the post? And then right? As a responsible.

Feanil Patel: why don't you send me the list that you guys are interested in so I can cross-reference where there's multiple people and where there's not and then I'll start posting here's a list of things that we're looking for. So it's not the bombard Everything at Once although I'm happy to change my sort of tactic if people feel it would be better to just say here's the full list and please volunteer, so I'm not sure about which to do so I could use some support some. thoughts on that

Felipe Montoya: Sure, I live in you with reset. We are interested in and

Feanil Patel: Okay, awesome. Thank you.

00:15:00

Feanil Patel: Okay, let's see. That's good.

Feanil Patel: Okay, sounds good. Thanks, very

Feanil Patel: That's just a bit.

Feanil Patel: Next so we covered the Ubuntu upgrade. Is there anything else anybody wants to say on the Ubuntu upgrade?

Feanil Patel: So the node 18 to 20 upgrade an Adolfo raise this I believe yesterday or the day before we've been talking about it a couple of times but to summarize I think I'm gonna give it a shot Adolfo and then you tell me if I missed anything to summarize we are. Currently running node 18, which is technically not end of life. But the node ecosystem is basically over it and starting to upgrade things to node 20 and requiring things to have no 20 newer versions of packages. We are depending on are starting to drop support for node 18, which may require us to upgrade to node 20 sooner rather than later. I miss anything Adolfo.

Adolfo Brandes: No, that's about it.

Adolfo Brandes: and as I mentioned there we have a long time quote unquote Until it's end of life. But that really is just sumac right…

Feanil Patel: Okay.

Adolfo Brandes: because the one after that he is already end of life. So this is too soon for Redwood,…

Feanil Patel: right

Adolfo Brandes: I think. maybe we can get in a couple of them if he's that actually supports node 20, but as I mentioned in a slack thread because tutor only supports one version of notes in production. during this transitional phase all mfps are going to have to support node 18 and 20.

Feanil Patel: functions

Adolfo Brandes: It's the same thing that happened during the 16 to 18 upgrade.

Feanil Patel: is

Adolfo Brandes: So the unit tests have to have a test Matrix for both versions. in the MF fees

Feanil Patel: It possible to change something in tutor to just bring up two images temporarily like in the plugin for mfe's. Which I assume already has a list of MFS that it's loading just have a second container.

Adolfo Brandes: it's possible but It's really clunky. and I doubt Jesus is going to accept a PR that does that

Feanil Patel: I mean Okay.

Feanil Patel: That sounds like a conversation worth having.

Adolfo Brandes: Yeah, but as I mean, I'm also maintainer of tutor MFE which is where this would happen. and I actually think This should be on the mfis,…

Feanil Patel: but

Adolfo Brandes: right? it's

Adolfo Brandes: it needs to be done at some point. right

Feanil Patel: Yeah, yeah, I don't think I'm disputing that. I think.

Feanil Patel: If it's a timing issue. I think being able to gracefully upgrade some of the mfes without having to block on all of the mfps is really valuable.

Adolfo Brandes: No, I agree.

Feanil Patel: And so if our development environment can't support anything but a big bang that to me is a weakness of our development environment.

Adolfo Brandes: But the development environment if you run mfe's individually can support the different versions.

Feanil Patel: Sure, our communities.

Adolfo Brandes: but in production But in production,…

Feanil Patel: Yeah, yeah. whatever we say is the official version should be flexible enough to be able to let us upgrade some things and…

Adolfo Brandes: yeah, I know it.

Feanil Patel: not other things I think because

Adolfo Brandes: Yeah. But it's a complicated conversation…

Kyle McCormick: an MFE can support

Adolfo Brandes: because then you lose some of the benefits of the Xavier single image for all them plus when we move on to a pyro world.

Adolfo Brandes: That's just going to be the case. Sometimes all of my fees are gonna have to be upgraded at once at least all the ones that are in supported by the shell. Maybe not. I don't know. But this is a big conversation.

00:20:00

Feanil Patel: Yeah, let's get it started. Yeah.

Kyle McCormick: Adolfo

Adolfo Brandes: Yeah. Go ahead.

Kyle McCormick: 10 an MFE support both note 18 and 20 at the same time. are they?

Adolfo Brandes: yeah, yeah, that's

Kyle McCormick: So could we start testing specific mfes, running tests on? Okay, so whether or…

Adolfo Brandes: That's a great first action item,…

Kyle McCormick: you can do it we should do that because

Adolfo Brandes: but start enabling no 18 tests or…

Feanil Patel: No 25th.

Adolfo Brandes: not 20 times

Feanil Patel: Adolfo do you think for all the mfe's can you just pick it up doing that? It sounds like there's already PRS for that.

Adolfo Brandes: Yeah, sure.

Sarina Canelake: Yeah, something that I don't feel I've heard brought up. But just as another thing to consider about upgrades is we're gonna have to keep doing these upgrades and we might be getting more mfes. And so the more we're stuck in that big bang situation. It's just going to get harder and harder we're gonna have to keep dealing with this over and over so I think it's important that we try and look at all the solutions and even if there's up front cost to one solution if it makes the next few years of upgrades easier, I would say we should go with something like that.

Adolfo Brandes: That's a good point. Yeah.

Feanil Patel: Yeah.

Adolfo Brandes: It is something that it's in the back of my mind with the move to spiral and how that's going to impact things. regarding in particular because right now the gatekeeper is tutor right like having to do all at once sort of…

Feanil Patel: Sure.

Adolfo Brandes: but it will have a higher level actually lower level gatekeeper going forward worth investigating how that's going to

Tobias Macey: So I missed what you said was the upcoming gatekeeper Beyond tutor.

Adolfo Brandes: viral, …

Feanil Patel: payroll

Adolfo Brandes: it's where we're actually Moving forward faster with that now and in that world, it's a single web app a single Spa right? it's not multiple sbas anymore. And that will introduce certain constraints on polymer fees or loaded built on and so forth. We're not sure for example, if we'll be able to build nmfe with nodes, 18 where the rest are built with no 20 and That's still work.

Feanil Patel: right

Adolfo Brandes: I don't know.

Tobias Macey: Okay.

Adolfo Brandes: We'll see.

Tobias Macey: If somebody can drop a link in with any references to this project because it's the first I've come across it, but I also don't spend much much time in the front end world. So

Adolfo Brandes: There's a draft to whip.

Feanil Patel: Yeah.

Adolfo Brandes: I will link them.

Sarina Canelake: Adolfo should just direct people to slack room that we made yesterday.

Tobias Macey: Thank you.

Adolfo Brandes: Also, yeah, if you're interested there's a hash viral room now.

Sarina Canelake: Yeah, I would say if people are interested join the payroll slack group and ask your questions in there so that we can provide the links to everybody and have them saved in one place.

Feanil Patel: Can one of you add the link to the slack room to the meeting notes? Thanks that Adolfo.

Adolfo Brandes: Yeah, I'm on it.

Feanil Patel: I'll add your PR to

Feanil Patel: All right. Let's see. Okay, so that's the node situation. Yeah, I think getting started with enabling node 20 wherever we can so that we're at least testing on both an identifying any blockers is a great idea and we should do that. No matter what Adolfo once you ticket that up, perhaps we can coordinate but I think we can announce it to the community since we're still not in the world where everything has maintenance for any core contributors that are out there. if you have knowledge in this area and can look at these. PRS and help fix tests, if that's necessary or anything or review things that be super helpful.

Feanil Patel: We can post a call to action once the tickets exist for people to pick them off.

Adolfo Brandes: So just not a good put the cart before the horses, but I'm probably gonna focus our prioritize the mfgs that are part of releases going for.

Feanil Patel: Yeah, absolutely. We want to get those out the door because I mean the others aren't a blocker for tutor so we don't have to. life

Adolfo Brandes: Exactly exactly.

00:25:00

Feanil Patel: Thank you next Adolfo. You want to talk about moving npm workspaces?

Adolfo Brandes: Yeah, so the first one is more of all for help than anything. if you go to the issue that's linked there.

Adolfo Brandes: But that's the main issue governing the move of at edx npm namespace to add open. Edx, and we're doing it piecemeal package by package. And it turns out it's a reasonable amount of manual work. Even if it's just tax replacement. But you still have to test all of it. So each PR takes a bit of a bit of work, right? So Brian Smith.

Feanil Patel: yeah.

Adolfo Brandes: Has been tackling this for a couple of weeks. And we had yagnesh submit a PR to learner Dash which we're reviewing and testing. Thanks for that. Yeah, but if anybody else wants to help with this particular effort.

Adolfo Brandes: just reach out to me jump into that issue, whichever you prefer. and if you have an MFE you'd like to take up for this. It will be great. It's not already done.

Feanil Patel: Adolfo I

Adolfo Brandes: basically where we're in the middle of moving the Paragon and part content platform packages and that takes a from work

Feanil Patel: Yeah. Yeah I realized that I wrote up a couple of how-to's while I was doing this initially. I don't know if I shared them with you previously, but has the full list of things that need a scope change, but also how to move a scope and how to update the places where it's reference. And with some scripting that I wrote up,…

Adolfo Brandes: awesome But that I…

Feanil Patel: so maybe that's useful.

Adolfo Brandes: if you show that I forgot it. thank you again.

Feanil Patel: Yeah, it's in the meeting notes now so you can take a look at that and maybe feel free to edit and rework it. However you need to but yeah,…

Adolfo Brandes: All right. No, perfect.

Feanil Patel: but from when I was doing the original research, Those are my notes and my scripts.

Adolfo Brandes: So if nobody has any questions and They don't want to. volunteer off the bat There's a segue.

Feanil Patel: I think not all the volunteers are here. Yeah.

Adolfo Brandes: so in doing this in particular with course authoring We ran into a specific issue which was a lot of pests started failing because of this, it's because you're updating package lock across the board. So things start breaking right? It's a normal thing,…

Feanil Patel: right

Adolfo Brandes: but then One of the tests we were investigating was in the video upload speed. And it's a new page, So we were even considering it for our release going forward. But it turns out that the video uploads feature is not really usable by the community at Large.

Adolfo Brandes: And then the conundrum was what do we do about this test? Because we can't really run this page to figure out what's wrong.

Adolfo Brandes: And it's sort of blocking progress on a separate issue, So in discussing this with vanille and maximum engineering We come up with a proposal Basically make these tests optional when we encounter such a situation.

Feanil Patel: it's

Adolfo Brandes: so they don't fail Ci or anything like that and then of course for the future to decide whether To deprecate this feature or make it usable by the community, which I guess would be the ideal situation. just make it a plug-in, right? So it doesn't block the development of the MF yet. So Of course plugins are separate discussion. That's going forward pretty strongly at the moment, but there's still no. Single plugin API or everything but that'll come soon.

Feanil Patel: it's

Adolfo Brandes: any objections to the thoughts concerns

Feanil Patel: yeah, I mean, I think especially I'd love to hear from to you because I think this is going to happen with you guys more than with anything else, but

00:30:00

Feanil Patel: I think in the short term there might be some tests that get disabled that are relevant to you because nobody else knows how to test them.

Feanil Patel: If they're I know it's hard to sort of commit effort to helping test those things right now as well. So just more of an FYI maybe and you guys can adjust internally you want to make sure you heard all of the things Adolfo just said that we will turn off tests for things that we can't test.

Adolfo Brandes: Yeah.

Feanil Patel: in the community releases And look like their speech features that require sort of internal to you other company infrastructure, but all right see a thumbs up. I'm going to go with that for now. All right.

Robert Raposa: Sorry, I'm just not clear what turning off.

Robert Raposa: Dig into the details here. I'm just curious is it possible that have them be optional in some way that we can run? and others aren't

Adolfo Brandes: I think you can make tests. I'm not sure on this particular set of tests, but I think you can just make them sort of a warning. They don't block CI. from continuing

Adolfo Brandes: but you can see the failure, but of course if you're not looking at the warnings you Which is not something people normally do.

Robert Raposa: Get it. Yeah, everyone's certain same sweet of this. pumpkin

Tobias Macey: Yeah. I'm not sure…

Feanil Patel: Yeah.

Tobias Macey: what the test setup is. But if you using pytest, you can either use the mark feature to be able to mark them as a particular scope and on select those conditionally in your test run or…

Feanil Patel: Okay.

Tobias Macey: you can also mark them as xfail where you say I expect this test to fail and then you can optionally say I want this to be an actual failure even though I say that I wanted to fail

Feanil Patel: but

Adolfo Brandes: at a runtime you mean right you configure the tests are runtime to do that or…

Tobias Macey: Correct. Yep.

Adolfo Brandes: not, because that then

Tobias Macey: It's based on how you invoke the pi test run. It'll say either run these tests and…

Feanil Patel: that Yeah.

Tobias Macey: expect them to fail or run these tests and actually fail it.

Feanil Patel: Yeah. Just I think these are JavaScript tests and just Tobias, but I know Jess I think has a similar setup for sort of running specs and optionally running specs

Robert Raposa: because It …

Adolfo Brandes: Worth looking into it.

Robert Raposa: and I guess before just thumbing it I'm not totally clear on what the problem is because if the problem is and we've got tests that start failing and we don't care about them. So we don't want to know about it, but that's an obvious like hey if this is it to you feature we want to know is that thing is failing and now we're going to disable the test and we're never going to know about that. It feels like there's so

Feanil Patel: No, so yeah, let me attempt to summarize what Adolfo is saying, which is that we in trying to do upgrades certain tests started failing and there was no documentation on how to get to some of the pages that we're failing. Or how to enable them without sort of having a lot to you infrastructure available and…

Robert Raposa: it's

Feanil Patel: like digging deep into the code and it was clear that the feature was specific to you. So it was hard to sort of find that the information they needed to be able to validate that the failure is either real and needs to be fixed or the failure is a false positive and can be ignored.

Jeremy Ristau: So maybe to clarify because I was under the assumption you're talking about the video uploads page but it does not sound like you are talking about that as that's not to you specific thing. My guess is you're talking about the transcript settings feature that was added or

Feanil Patel: So the video uploads page currently only really works with the 2u video encode manager and other to you infrastructure. It's actually not.

Jeremy Ristau: The video dependency.

Feanil Patel: Yeah, yeah. So the video upload page is part of the video Pipeline and so is not usable by the community as it stands today.

Jeremy Ristau: Okay. Clarity helps. Thanks.

Robert Raposa: and so your example is clarifying my concern…

Feanil Patel: stick

Robert Raposa: which is During the upgrade process because we had this test we found something that may or may not be an upgrade problem for you. And if we just got rid of those tests one would know to you wouldn't do thinking that there is any problem with the upgrade we would need to know and…

Feanil Patel: right

00:35:00

Robert Raposa: maybe there's just some separate process of it. We're disabling this but two units to find a way to find to go through the upgrade process on its own and whether it's running these tests locally manually or something so that we actually are not like surprise it broke in production because we aren't running these tests that it would have said

Feanil Patel: Yeah, and I think that's correct. And I would love to you to sort of help us provide with a process That wouldn't be a huge burden on the community.

Robert Raposa: yeah. Yeah. cool, so

Feanil Patel: Do you want me to put it to do in here for you to follow up on that Robert?

Robert Raposa: You sure.

Adolfo Brandes: And then I deal World in my ideal world. This for example would be a plugin right so we don't have to worry about it at all and you have your own pipeline test plugins and…

Feanil Patel: Yeah. Yeah, but I mean even in the meantime,…

Adolfo Brandes: forth. Yeah.

Feanil Patel: it might be helpful to be able to say when you disable a test we Market in this way, so you can go find all the things that need to be plugins. if we had a process for marking them disabled that makes it easy for them to find later.

Adolfo Brandes: Or yeah.

Feanil Patel: I think that's totally worth doing.

Adolfo Brandes: I mean actually in the ideal World we'd have a way to run this pipeline or documentation or package products that ships with open attics, right?

Feanil Patel: we'll get there we'll get there.

Jeremy Ristau: Yes, definitely We're not in an ideal situation though. So We much more tangible paths forward.

Feanil Patel: Yes. Yeah exactly.

Jeremy Ristau: But I've dropped a note to the team that owns that page already.

Feanil Patel: Okay, so That's more.

Jeremy Ristau: So we'll figure it out.

Feanil Patel: Thank All right and Robert I've added it to do for you. Let me feel free to add more words to make it easy for you to action on that.

Robert Raposa: sure, I guess what Jeremy do you want to take that action item if you're going to be working with or do you want me to I'm happy to keep it and be in the discussion but

Jeremy Ristau: I think that T&L can be the example but this is not specific. This is when any tests are disabled, how are we informed about it that feels a little bit broader. So if you want to keep running with that and I'll make sure that T&L solves its very specific them dependency problem.

Robert Raposa: together and I like to know what you had proposed but there's also the second piece that somehow the community needs the notifications that this upgrade process is happening. And here's the time to actually do a thing that you would be thinking about there.

Feanil Patel: Yeah, yeah. I mean, I think so that process already exists. I think there's the time between cutting a release and finalizing Where things will get tagged and can be tested. Before they're fully released. So on the community release side. I think that process already exists. And it just might be internally within to you when that happens. You may want to like.

Feanil Patel: Put some internal actions on that trigger.

Robert Raposa: All I might check in with you that just more details of what that means. Thanks.

Feanil Patel: Yeah, I'm happy to But BTR does a bunch of announcements that you can probably trigger on.

Feanil Patel: cool

Adolfo Brandes: also also not Jeremy maintenance said Product is starting to get way more involved in the release process. Starting with redwoods.

Feanil Patel: Get right.

Adolfo Brandes: So a lot of this stuff is going to surface way earlier than a month or two before release. So I mean whatever I run into something like this,…

Feanil Patel: right

Adolfo Brandes: I'll gladly bring it up here for example.

Adolfo Brandes: or

Feanil Patel: We can start with that and Robert's gonna come back with us with maybe a written process so that you don't have to be at the synchronous meeting to find out about these things. but I think these are good steps forward. We can sort of iterate on them as we learn from them.

Feanil Patel: All That was everything in the agenda. Does anybody have any other items to bring up?

Feanil Patel: Max go for it.

Maksim Sokolskiy: Yeah. Its It's about comments service, but I'm not sure would be it's such important and it doesn't affect all our ecosystem. But it's important for me as a release manager.

00:40:00

Feanil Patel: Yeah.

Maksim Sokolskiy: I want to maybe qualify the status of the maintenance of the comments service.

Feanil Patel: Yeah. I will put that up as one of the things that we need to maintainer for.

Robert Raposa: To make it a surprise for the person who volunteers after the volunteer.

Feanil Patel: No, Jeremy.

Jeremy Ristau: I had a note dropped in the maintenance working group slack Channel related to the Central versus distributed tasks of an upgrade and

Jeremy Ristau: And going forward who might be the owner of those types of central tasks. like Creating PRS or issues in different repos things like that making sure things are visible and communicated out.

Jeremy Ristau: I'm not really expected to have an answer today.

Feanil Patel: Yeah.

Jeremy Ristau: But topic that I want to make sure stays on the radar.

Feanil Patel: Yeah for what it's worth if you're here. I need your help to do that centralized work. So it's going to be distributed amongst this working group. And I will need help from the people who are attending or people who are not attending but can do that work to help do that work because I think historically it was sort of the bomb squad. And I know that the capacity there is not really there for this Michelle's helped has volunteered to sort of support the check-ins, but I think the creation of the tickets and the creation of the technical description is something that will probably need to distribute. So for example today when I asked adolfoot to make the tickets for the npm 18 to 20 upgrade that as I think of once that's created.

Feanil Patel: We will probably be checking in on those regularly and it'll be a combination of Michelle and Adolfo doing those check-ins and maybe Tim also but I think we're gonna need help both with the creation of the technical trajectories. And also with the checking in and seeing how things are going of those. I'm planning on putting together a board.

Feanil Patel: So that we can start tracking everything in one place.

Adolfo Brandes: I think a board is great idea

Feanil Patel: Yeah, yeah, so I'll create that soon. I was thinking about it yesterday and if I had time I would have done it, but I did not yet do it. But yeah, we will have a centralized board for the maintenance work. It'll hopefully feed into the product roadmap, which does have some virtual eventually have line items for the major upgrades as well. I'll be at the product roadmap later today to sort of represent any ongoing maintenance work that we need to make sure either gets accounted for or otherwise needs to be informed about. Otherwise users if you informed about

Robert Raposa: I thought there would the maintenance board.

Feanil Patel: 

Robert Raposa: And it didn't or…

Feanil Patel: But there is a edx upgrades board.

Robert Raposa: is that a totally different thing?

Feanil Patel: But I don't think things have transferred out or are in a place where the community can fully make use of it. I think there is a maintenance board, but the state of it is not super clear. So I'm going to dig into it maybe a reuse of a board, but it certainly going to be making it more public and making it more visible.

Robert Raposa: Yeah.

Adolfo Brandes: I used to look into it that board a lot.

Jeremy Ristau: Yes.

Adolfo Brandes: So it was useful. but as I understand it was maintained by Jeremy Bowman,…

Feanil Patel: Yeah.

Adolfo Brandes: so I don't know what

Robert Raposa: Yeah, sorry. I thought he was transferring it at summer to open that but I guess that didn't happen.

Feanil Patel: Yeah, I don't think that process ever completed. so maybe I'm not sure if making a new board or using the existing one will be easier, but we'll figure it out.

Adolfo Brandes: In practice it was a Fed bomb or a particular for the front and…

Feanil Patel: right

Adolfo Brandes: stuff. So.

Feanil Patel: yeah, I think we need something that gives us a little bit of higher level visibility as well as the low level visibility so

Adolfo Brandes: Yeah, yeah.

Jeremy Ristau: Yeah, and I would say I've gotten feedback from community members that upgrades led by bomb teams might not be as visible as could be and…

Feanil Patel: Yeah.

Jeremy Ristau: and so that I think is a big difference between transparency and visibility right? You can put stuff out there and that makes it transparent but it's the communication and the regular updates provided to stakeholders that make it visible. So I'm all for a board but there's more than just tickets and issues involved with something like this especially if you needs to hand off more of this work, we still would like the Insight. So what I would love to be a part of is looking at the process that exists today the steps documented…

00:45:00

Feanil Patel: Yeah.

Jeremy Ristau: where the communication channels are and everything and then working to bolster that and improve that I'd be happy to nail.

Jeremy Ristau: Something that feels better for this group and the community in general before handing some of those tasks off to the community. That way I also feel comfortable knowing that I will have the visibility needed when upgrades are happening that we're not driving and so, I'm just putting that out there now, I'm certainly happy to help improve the current state.

Feanil Patel: Yeah, yeah, and for what it's worth Jeremy one of the things that I've already planning on adding to the existing processes that we have a maintenance announcement area, which I've been using for. Informing maintenance about things but I think we'll also be where calls to action updates and changes to major maintenance topics will go so when an upgrade is started and we need help when an upgrade is in process and we're stuck and when an upgrade completes, I think all of those things would end up getting communicated in this area. So subscribing to just that Subspace should help people be much better informed than they have historically been.

Jeremy Ristau: Yeah you.

Jeremy Ristau: Yeah, even to the previous point which tests were disabled things like that.

Feanil Patel: Yeah, yeah exactly.

Feanil Patel: cool I don't know how to convert that into an action item or something that will result in next steps. Somebody have words I can use because I feel like Jeremy there's maybe on your side maybe reviewing the existing. Process and…

Jeremy Ristau: Yep.

Feanil Patel: tooling and providing it to the community in some way.

Jeremy Ristau: Yeah, if someone who's on the stakeholder side the people who wish they had better communication side could partner with me. That would be great.

Feanil Patel: Yeah. Yeah, I'm happy to is anybody else on the call interested in this especially non accent folks, I would love for the people who are not getting communicated to be the ones that sort of help build out the communications channels.

Jeremy Ristau: I know it's exciting. Don't everybody jump at it at once.

Jeremy Ristau: Processor is a good times.

Feanil Patel: Okay.

Jeremy Ristau: maybe what I'll say is I am currently in taking any and all feedback. If you have issues with previous upgrades in any form how they were done or how they were communicated. Just send me a slack message.

Feanil Patel: and Jeremy if you have specific questions, do you feel free to reach out to me and I'm happy to sort of provide. An answer it may not be the right answer, but it'll definitely be an answer.

Jeremy Ristau: Yeah, sounds good.

Feanil Patel: Okay, great. Anything else anybody wants to cover?

Feanil Patel: All Sounds like we're done for the day. We've got six minutes to spare. Thank you everybody.

Kyle McCormick: t

Feanil Patel: Yeah, I think that's all I have Kyle go for it.

Kyle McCormick: phenyl and I think Robert and Jeremy ristau are sticking around after this to talk edx specific maintenance. So if anybody else wants to be in edx platform maintainer, right this second figure us help us figure out how to do that stick around. We'll probably be just staying after these meetings for that.

Sarina Canelake: Kyle he's in he wants it.

Kyle McCormick: Great. He gets module. So.

Feanil Patel: Okay.

Sarina Canelake: He's literally running away.

Adolfo Brandes: right

00:50:00

Feanil Patel: Okay.

Feanil Patel: cool

Feanil Patel: I'm just gonna add a section to the bottom here.

Kyle McCormick: actually

Feanil Patel: Okay.

Feanil Patel: Feel like I never fill out the goals section of a meeting. And a part of me feels bad about that.

Kyle McCormick: cool maintenance

Feanil Patel: maintain all the things Yeah.

Kyle McCormick: self-explanatory, right

Feanil Patel: Okay, let's see We got here.

Feanil Patel: Sorry.

Feanil Patel: Okay Kyle, you want to kick us off?

Kyle McCormick: I guess a lot to talk about here. I mean in the long term we want to get to a state that I think is similar to what to you has been doing internally for a while. Which is that? edx platform had a sort of lead maintenance team Which was the bomb squad? with Jeremy Bowman leading it. and then that team, maintained a set of folders, but also took care of the delegation of both bugs and pull requests to different teams depending on the folder within edx platform. that they were centered on right so If you had a bug coming from courseware or a PR that was focused against the course where folder?

Kyle McCormick: That would route to a particular team into you. So I think that was a good model and that will want to essentially build that same system. but with a quark contributors or FedEx platform maintenance in the community instead

Jeremy Ristau: I think if I hear you and say it back to you in a different way historically edx and to you have maintained an internal Tech ownership sheet for all of the directories inside of edx platform with an owner. It's time to evolve that to be Oriented and not to you oriented my essentially saying the Tactical version of what you said. Yep. and…

Kyle McCormick: Yeah, exactly.

Feanil Patel: I think that sounds right.

Jeremy Ristau: we use that internally to Route tickets automatically with automation. I could definitely see that as long as the tooling is available. That's something that could happen as well for GitHub issues.

Robert Raposa: right We're just going to add a little bit more history is that Michelle was actually using that spreadsheet and…

Feanil Patel: Yeah.

Kyle McCormick: Yeah, thank God Robert.

Robert Raposa: doing the routing for most of the history of Michelle was doing that work in general. And then only recently has that.

Robert Raposa: transferred to the bomb team to try to do and that has a semi-successful Transfer so that you…

Feanil Patel: Yeah.

Robert Raposa: and that was really before everything else happened. I so basically, I'm thinking we don't really have a Correct process right now and that's part of what we'll figure out. Hear it…

Kyle McCormick: Got it, I think.

Robert Raposa: what the process of triage should be.

Kyle McCormick: I think I'm conflating two things. So one of them is the automated alerts. that's what Bomb built out so that if you get an opportunity alert that that's automatically routed to a team based on these. Red sheet and…

Robert Raposa: that

Kyle McCormick: that will stay to you internal right because that's your own production and you'll probably still have internal owners for apps regardless of who owns them in the community. But then yeah that Michelle using the sheet to delegate or…

Jeremy Ristau: Correct.

Kyle McCormick: requests and eventually bug tickets get issues that are bugs to particular teams That's what will become community oriented.

00:55:00

Robert Raposa: Right, hopefully and in the last couple of two months or something that tried to move to bomb not too much. I don't remember what happened, but it was right before. Everything changed and it was an experiment that Jeremy Gomez was trying.

Robert Raposa: And it's not trying.

Kyle McCormick: Yeah.

Kyle McCormick: the last thing I'll note before moving on from history is that there are some teams in the community now that the sheet routes to so I think if a PRS against Block store related stuff and edx platform. We'll go to open craft. Based on that two managed spreadsheet.

Tobias Macey: The sounds like a situation that would work. with the code owners file in a repo and for being able to automatically assign PRS and…

Feanil Patel: Christmas

Tobias Macey: do that triage. I think the hard part is just Figuring out. What is the Breakdown of what those segments need to be which it sounds like it's already largely done by virtue of having that spreadsheet but in the harder pieces figuring out who are those owners?

Feanil Patel: All right.

Jeremy Ristau: We leverage code owners for a few directories inside of edx platform now. I wouldn't say that it's very visible.

Feanil Patel: Yeah.

Jeremy Ristau: It isn't a mechanism to help get information to the code owner quickly in my opinion.

Kyle McCormick: Good.

Jeremy Ristau: They're probably things that I can improve in order to get better visibility. But code owners is definitely something that's precedent so far.

Tobias Macey: Yeah, my understanding is that the code owners allows for automatically assigning PRS for code that touches for changes that touch that those sections of the code?

Jeremy Ristau: Yeah, that's right.

Feanil Patel: I mean, I think and Jeremy then.

Jeremy Ristau: Yeah.

Feanil Patel: I think there might be some. Improvements we can make to people actually receiving those notifications and confirming that they're notifications settings are such that they are aware of those things. But I also know that there's a balance between receiving the notification and actually prioritizing that work against other work To you ecosystem. So maybe that's what you're talking about. how it's not sort of conducive to that second thing.

Jeremy Ristau: I might have missed that.

Feanil Patel: I was saying that there's sort of a difference between people getting assigned PRS and the pr is getting prioritized for review and it's that second thing. That's the tricky bit that needs to get figured out.

Jeremy Ristau: Yeah, I mean that is a problem that we have heard about for a long time. Yeah.

Feanil Patel: Yeah.

Feanil Patel: it sounds like what correct me if I'm wrong, but I'm not hearing any. objections to

Feanil Patel: any objections to actually shifting this externally and making an externally visible system for routing with it. it sounds like we're all good with there being a routing system within edx platform.

Kyle McCormick: For I don't think anyone's saying that one team should be responsible for all that explore.

Jeremy Ristau: Yeah, and I think going up a level even more. Yes. I think that's the thing is. I don't think anyone is opposed to having an enumerated list of owners for directories inside of edx platform. That isn't all one group.

Feanil Patel: thread

Jeremy Ristau: Yep.

Robert Raposa: Enough. So I mean it seems reasonable that the good and style will be a part of that solution, especially because of all the GitHub right on that…

Feanil Patel: Yeah automation that comes with it.

Robert Raposa: but I just sent out a notice to The couple of teams that wanted some sort of notifications like hey that file already exists, you created it. It's gone through different iterations.

Feanil Patel: Yeah.

Robert Raposa: I'm like, you are welcome to add your team in there, but we don't have automation yet to deal with synchronizing with other things so that I think that's a larger project that we'll just have to do.

Feanil Patel: Yeah, I think that's Yeah, and I think coderners file does have some downsides. We want to rename and make the team names across the open edx org consistent and by renaming teams, we impact how functional the code owners file is we have to update code on our files as a part of that.

01:00:00

Feanil Patel: so I wonder if we should accelerate that. organizational reorg of teams Kyle before we start sort of the land grab for code owners.

Kyle McCormick: So before we dive into that. I mean, I'll point out that catalog info is the same way.

Feanil Patel: Yeah, that's a fair point.

Kyle McCormick: I want to think what's the highest priority thing for edx platform and for me it's not this distribution of ownership. It's the impending Redwood upgrades that we need to do.

Kyle McCormick: so if I could…

Feanil Patel: Yeah.

Kyle McCormick: what I would propose is for Redwood, we just focus on Getting those upgrades done with the volunteers we have. and then once that's done we tackle the more oriented problem of setting up code owners and getting this distributed maintenance model. And if that sounds like we're putting off. The code under stuff too much. that's only two months right in two months. Redwood has got

Feanil Patel: Yes.

Feanil Patel: Yeah, and in the meantime, I think the current direction that Michelle and Tim have a routing PRS is to seek reviewers amongst the core contributors so that I think they're posting them in the CC edx platform channel for seeking reviewers. And I don't see any reason why. People internal to you can't follow that channel and volunteer to do reviews and merge things if they have capacity, but that other people can do it so that we're not blocked on. sort of finding specific capacity to people have concerns about that or want to change that process in the next couple of months.

Jeremy Ristau: I think the biggest issue with slack communication to you the larger to you group of employees is that when we mirror those on our slack instance, they have to be private channels.

Feanil Patel: Okay.

Jeremy Ristau: So you can't just opt into them.

Feanil Patel: right

Jeremy Ristau: It makes visibility super hard to just say join this Channel and then you'll have what you need. It has to be more. If you're interested then paying me and I'll directly add you to this Channel and you can't see anything linked like it's definitely a hurdle and that's like a cybersecurity imposed restriction on us to have these channels be private since they're internet public on the other side.

Jeremy Ristau: I'm just calling it. Yeah.

Feanil Patel: Yeah, yeah it's not so easy as you guys will have to add everybody who could be at all interested in edx platform, but I think that will have to be true eventually anyway,…

Jeremy Ristau: right

Feanil Patel: because this is a Communication channel for ex platform changes generally, so if it's not already true, I think we should sort of at it as part of the onboarding process for anybody who might be touching it X platform.

Jeremy Ristau: so this kind of gets to the boiling oceans again, right

Feanil Patel: But knowing…

Jeremy Ristau: In the last meeting,…

Feanil Patel: who that is.

Jeremy Ristau: you said there would be a discourse space. And in this meeting, it's slack channel for different Communications. It seems like they have different solutions and…

Feanil Patel: Yeah. Yeah,…

Jeremy Ristau: that makes it difficult I think.

Feanil Patel: Yeah, and for context that. Alex platform specific slack Channel already exists and has been historically used which is why I wasn't trying to change it for mechanisms to communicate with every maintainer across the entire community that didn't exist and discourses sort of keeps history in a much better way and allows for sort of better long-term announcements. So I chose that happy to sort of consolidate those things over time into one place.

Feanil Patel: And if you think that now is that time to say Alex platform. Communications also happen on discourse and then people can just turn discourse. not crazy

Jeremy Ristau: And I'm more calling it out that It's gonna be as less than ideal Communication channel for us.

Feanil Patel: Yeah.

Jeremy Ristau: That's all. It's forces,…

Feanil Patel: Yeah, that's fair. Yeah, yeah.

Jeremy Ristau: what are you gonna do? Yeah.

Feanil Patel: If it helps both of those decisions or at least the slack decision was made by people when we were at to you a long time ago. So it's a really old decision. So it's not unreasonable to revisit it.

Robert Raposa: Wanted to address who it to you would be watching and would be caring so I just mentioned that code on our file recently because Kelly brought up the fact that she wants to hear about things like PRS that are related to jerks, things that she cares about but she doesn't want to have to watch a Channel or anything. She doesn't want to hear about all the things in the next platform because she didn't care about things she cares about specific things.

01:05:00

Feanil Patel: Sure.

Robert Raposa: So the code owners thing like hopefully solves. That problem for her, so she'll hear about the very specific things in terms of the non-specific things.

Robert Raposa: I'm guessing that our team and Jeremy, you can correct me if I'm wrong is the only team. That is going to be willing and interested to actually hear about all of these years and…

Feanil Patel: Yeah.

Robert Raposa: that the rest to you is like I don't want to hear about all these different upgrade peers. Is that so the question to Jeremy?

Feanil Patel: Sure.

Robert Raposa: And then Jeremy if the answer is our team would be the only team interested then it's a simpler like I was is this channel that we're already watching an okay place to just do that for now.

Jeremy Ristau: Yeah, I mean for now, I think we just stick with what we have.

Feanil Patel: Yeah, I mean, I think theres from my side. It was like I was informing of the current state of the process in that, to Kyle's point. I don't think we have capacity to change that and do all the upgrades we need to do in the sort of near-term. So let's not change it or mess with it too much. I agree that it could change and let's talk about that later as much as possible.

Jeremy Ristau: Yeah, I think too. We're going through the auditing of the tech ownership spreadsheet internally.

Robert Raposa: Okay.

Jeremy Ristau: I think that we could probably end that audit process with an update to the edx platform code owners file just publicly enumerating…

Feanil Patel: Okay.

Jeremy Ristau: what we feel like we should be maintenance up And…

Feanil Patel: Yeah, that sounds great.

Jeremy Ristau: then we can give feedback on the code owners process as well.

Robert Raposa: and then Kyle to your prioritization question, I guess the other question I have is Current osprs in edx platform that are supposedly. stuck that may or may not be related to and upgrade. But does feel like they might be important to community members. I don't know what the different things are. And what are stuck and what our stuff that's supposedly, our bomb now as a spreadsheet and that was going to be one of our jobs and I think now this meeting is to figure out how can that be all of our jobs to get those unstuck and not have our Palm be a Blocker on the

Robert Raposa: And I don't know that and we could say that's been a ongoing problem forever. So we'll deal with that also after the Redwood upgrade or we can talk about that sooner.

Feanil Patel: let's Sorry God Kyle.

Kyle McCormick: 

Kyle McCormick: I'd say top priority Road upgrades. next priority unsticking edx platform PR's early is looking through the list that we have. In at least updating people say hey, You're early in the cube. This is lower in the queue. and slightly lower than that is the proper mapping of Maintenance to directories and explore

Feanil Patel: Yeah.

Robert Raposa: But I wonder if we could come up with even a standard response that has some list of that and…

Feanil Patel: Yeah.

Robert Raposa: can either okay. That's it. this don't have to sit and people don't hear anything, but they can hear that.

Feanil Patel: let's move to the upgrades because yeah, I think there are solutions for those other things, but I'd rather talk about solutions to getting mongo and python upgraded first.

Feanil Patel: and especially because for the python upgrade I think we need both the python and Mongo upgrade for Redwood. Which doesn't give us a lot of time to do it. I know that our big bomb had started on that process, but I imagine I don't know what the state of that is there. They were starting by doing an evaluation of the existing packages and what they're classifier claims were in terms of which versions they support but

01:10:00

Feanil Patel: I wonder if a more useful system would be to just start in what we need to upgrade and start by testing it on python to python 312 and figuring out what our blockers are in that direction instead.

Kyle McCormick: Yeah, do folks know how the upgrade was being managed up into the point where Jeremy left? Because they were spread was using repo Health to track which repos had python 310 test enabled.

Feanil Patel: I don't think they had started on the python 310 upgrade at all. The last thing I talked to Jeremy about was they were about to use repo Health to identify, which dependent packages of our ecosystem third-party packages had classifier claims to support Python 3 10 11 and 12.

Kyle McCormick: Got it. And I think that's what he's used for previous upgrades like five to three eight. Okay.

Feanil Patel: Yeah, I think that's But again the two to three and the three five to three eight updates. We've had Fairly major changes. The three eight to three 11 and 312 don't actually have a lot of breaking changes on pieces that we care about. so I think there's The Proposal I'm putting forward is let's start testing everything on 11 and…

Kyle McCormick: Yeah.

Feanil Patel: 12. And figure out where it breaks and make the decision after the words rather than trying to see what people are claiming is supported.

Kyle McCormick: for sure, I just

Kyle McCormick: In previous upgrades, Jeremy's team has opened what? Three dozen PRS across various repos to say test it in the new thing tested a new thing and…

Feanil Patel: Yeah.

Kyle McCormick: somewhere it was keeping track of all that. So we went just walking through bass.in every day.

Feanil Patel: All Rare it.

Kyle McCormick: Figure out what had he doesn't want to.

Jeremy Ristau: From what I can tell digging in so far each of these upgrades are managed with a bespoke isolated Google sheet. Each one has been tracked individually. There are also corresponding public GitHub issues. and…

Feanil Patel: Yeah.

Jeremy Ristau: that seems to be more like the paperwork side as opposed to the actual tracker that the internal team uses which is some little scratch sheet. I'd like to Get away from that and…

Feanil Patel: right

Jeremy Ristau: make the actual progress more visible.

Feanil Patel: Yeah, I think that makes sense.

Jeremy Ristau: I don't have a full enough grasp of everything that was happening yet to be able to Done working on trying to get that.

Feanil Patel: Yeah, that sounds So in the meantime, I was just looking at the tickets on that the Repo in the edxor, which is I think a bunch of that stuff is being tracked for Django 42 and I think where this would have ended up. It feels like the next steps here are to

Feanil Patel: identify all of our open python packages Kyle and pick it up testing them on 311 and 312. Does that sound right to you? Just like adding CI checks for 311 and 312.

Kyle McCormick: Yes.

Kyle McCormick: both

Feanil Patel: Yeah, I think both.

Kyle McCormick: Okay, that could multiply that test runtime by three instead of.

Feanil Patel: That's fair. I think looking at the change log of what's new in 312 and what's removed in 312. It didn't look like There were enough issue there were things that we needed directly, but given that Six Degrees of Separation From Any Given package, we might be using something we don't know about.

Kyle McCormick: What we could do is run them on three and three twelve.

Feanil Patel: Yeah.

Feanil Patel: Okay.

Feanil Patel: Yeah.

Feanil Patel: That sounds good.

Robert Raposa: And we know that just We're not even planning on 311. So the three 11 tests are just if that helps us get to 12 compatibility where we planning on doing three levels.

Feanil Patel: I think if we can get to 312 directly we want to do that. But if there are issues that are blocking for 312. Than being able to fall back to 311 is still valuable.

01:15:00

Feanil Patel: All right. How do you have capacity to do this or I can try and do this?

Kyle McCormick: I don't have capacity, but I can try to do it anyway.

Feanil Patel: I think I have room for that.

Kyle McCormick: cool

Feanil Patel: All right, I'll take it up. Finding all the repos. I think we already have some tooling to find everything. That's a Transitive dependency of a repo using Ned's tool. So I'll try to focus on the things that are in the release first. Because those are the things we actually need by Redwood and everything else if they get upgraded later is okay, as long as all the services that are in the release and it's dependencies get upgraded. Okay.

Kyle McCormick: Sounds great. this is the Xbox we're meeting so obviously but my prioritize that Xavier form ove