Release Testing for the Upcoming Release
Introduction
Every 6 months, the Open edX project creates a new release (named after trees in alphabetical order - Quince, Redwood, Sumac, Teak, etc). We need the whole community to work together not just to build new features but also to test them - and other already existing ones! - before we can declare the next release ready to go.
Product Testing is our term for the tests that we do that copy a workflow that an Open edX user (such as an educator, course team member, or learner) would do as part of their experience on an Open edX site. Anyone and everyone is qualified to be a product tester, no matter your role in the community or on your Open edX team.
Important Dates
See the Open edX Releases Homepage for dates on when the upcoming release will be cut and tested.
Instructions for Product Testing
Testing is fairly simple:
Visit the Release Spreadsheet (for Teak: https://docs.google.com/spreadsheets/d/1kDn7P4qymNGZeCjTV9pWY9026RAarGsEyMBgWghRDok/edit?gid=0#gid=0 ), and navigate to the Tests page. To claim a test, add your name to the โAssigned Toโ column.
Note: Please only claim tests you can test within 24 hours.
In some cases, more than one tester will be testing the same test. In that case, put everyoneโs names in that cell.
The spreadsheet will have links to the testing โsandboxโ. Mark your test as โIn Progressโ, visit the sandbox, and perform all the steps of the tests (they will be specified in the spreadsheet).
For Redwood: Teak LMS, and Teak Studio
Update the spreadsheet with results: Put โCompletedโ as the testโs status, and โPassedโ or โFailedโ as the result. For failure, please file a report - see the next section.
Need help? Ask us in #wg-build-test-release in Slack!
Reporting Failed Tests
Youโll report failed tests on GitHub; if you donโt already have a free account, you can create one here: GitHub
To report a failed test you will create a GitHub ticket. This is similar to a ticket in other tracking systems, such as Jira or Trello.
Determine the appropriate GitHub code repository (โrepoโ) to log the failed test in - this will be the repo that holds the code that failed.
Donโt know the right repo? No problem! Go here: https://github.com/openedx/wg-build-test-release/issues
Open up a new ticket with a title that looks like this:
[Test-ID] Description of issue
such as: [TC_LEARNER_9] Sign in with existing user account keeps failingAdd as much detail as possible, including:
Steps to reproduce
Expected result
Actual result
Screenshots and/or screen recordings (Instructions on a Mac)
Submit the ticket
Add the label โrelease testingโ to your ticket by writing - in one comment, with nothing else - the following:
label: release testing
Hereโs a great example of a ticket: [TC_LEARNER_41] > date and course outline pages are blank ยท Issue #222 ยท openedx/wg-build-test-release
After you file the ticketโฆ
Please make sure you have email notifications enabled on GitHub.
You may receive questions on your ticket from the team responsible for fixing the bug. Please reply promptly so they may finish their work.
When theyโve marked the bug as fixed, please run the test again (or indicate, on the ticket or in Slack, that you are unavailable to do so). Update the Tests spreadsheet with the result of the re-test (if Passed, youโre good to go! If itโs still failing, update your GitHub ticket - no need to file a new one unless youโre asked to do so).
Spreadsheet for the Next Release
Is here! https://docs.google.com/spreadsheets/d/1kDn7P4qymNGZeCjTV9pWY9026RAarGsEyMBgWghRDok/edit?gid=0#gid=0
I want to fix bugs!
Awesome - the project is always in need of more bug squashers! We recommend that you join the Build Test Release working group, by attending a meeting, posting in the forums, and/or joining the Slack channel, to find out where they could best use you. To get started, check out the Communication section of the BTR homepage: Build-Test-Release (BTR) Working Group | Communication
FAQs
Have any questions about this process? Add a comment to this page or reach out in the #wg-build-test-release Slack chat room and weโll get back to you.