Release Testing for the Upcoming Release
Introduction
Every 6 months, the Open edX project creates a new release (named after trees in alphabetical order - Quince, Redwood, Teak, etc). We need the whole community to work together not just to build new features but also to test them - and other already existing ones! - before we can declare the next release ready to go.
Product Testing is our term for the tests that we do that copy a workflow an Open edX user (such as an educator, course team member, or learner) would do as part of their experience on an Open edX site. Anyone and everyone is qualified to be a product tester, no matter your role in the community or on your Open edX team.
Important Dates
See the Open edX Releases Homepage for dates on when the upcoming release will be cut and tested.
Instructions for Product Testing
Testing is fairly simple:
Visit the Release Spreadsheet (for Sumac: https://docs.google.com/spreadsheets/d/1HjmCUAkpe74rQAYM9dllmNy3RKnr70szyhO2rhnhFp0/edit?usp=sharing ), and navigate to the Tests page. To claim a test, add your name to the “Assigned To” column.
Note: Please only claim tests you can test within 24 hours.
In some cases, more than one tester will be testing the same test. In that case, put everyone’s names in that cell.
The spreadsheet will have links to the testing “sandbox”. Mark your test as “In Progress”, visit the sandbox, and perform all the steps of the tests (they will be specified in the spreadsheet).
For Redwood: Sumac LMS, and Sumac Studio
Update the spreadsheet with results: Put “Completed” as the test’s status, and “Passed” or “Failed” as the result. For failure, please file a report - see the next section.
Need help? Ask us in #wg-build-test-release in Slack!
Reporting Failed Tests
You’ll report failed tests on GitHub; if you don’t already have a free account, you can create one here: https://github.com/signup
To report a failed test you will create a GitHub ticket. This is similar to a ticket in other tracking systems, such as Jira or Trello.
Determine the appropriate GitHub code repository (“repo”) to log the failed test in - this will be the repo that holds the code that failed.
Don’t know the right repo? No problem! Go here: https://github.com/openedx/wg-build-test-release/issues
Open up a new ticket with a title that looks like this:
[Test-ID] Description of issue
such as: [TC_LEARNER_9] Sign in with existing user account keeps failingAdd as much detail as possible, including:
Steps to reproduce
Expected result
Actual result
Screenshots and/or screen recordings (Instructions on a Mac)
Submit the ticket
Add the label “release testing” to your ticket by writing - in one comment, with nothing else - the following:
label: release testing
Here’s a great example of a ticket: https://github.com/openedx/wg-build-test-release/issues/222
After you file the ticket…
Please make sure you have email notifications enabled on GitHub.
You may receive questions on your ticket from the team responsible for fixing the bug. Please reply promptly so they may finish their work.
When they’ve marked the bug as fixed, please run the test again (or indicate, on the ticket or in Slack, that you are unavailable to do so). Update the Tests spreadsheet with the result of the re-test (if Passed, you’re good to go! If it’s still failing, update your GitHub ticket - no need to file a new one unless you’re asked to do so).
Spreadsheet for the Next Release
Is here! https://docs.google.com/spreadsheets/d/1HjmCUAkpe74rQAYM9dllmNy3RKnr70szyhO2rhnhFp0/edit?usp=sharing
I want to fix bugs!
Awesome - the project is always in need of more bug squashers! We recommend that you join the Build Test Release working group, by attending a meeting, posting in the forums, and/or joining the Slack channel, to find out where they could best use you. To get started, check out the Communication section of the BTR homepage: Build-Test-Release (BTR) Working Group | Communication
FAQs
Have any questions about this process? Add a comment to this page or reach out in the #wg-build-test-release Slack chat room and we’ll get back to you.