...
Check if someone is already following this flaky test process at the same time:
Search for an existing Github ticket about this issue using the
Github search for flaky tests.flaky-test
label.Search on Github to see if the test was already deleted.
Create (or update) a Github issue:
Create a new issue in the repo containing the test code
Title:
[Flaky test] `path/to/test.py::class::method`
Labels:
flaky-test
Description: Paste the following template:
Code Block This test fails intermittently and will be deleted according to the Flaky Test Process in <https://github.com/openedx/edx-platform/pull/___PENDING___> See docs on [how to address flaky tests](https://openedx.atlassian.net/wiki/spaces/AC/pages/4306337795/Flaky+Test+Process#Handling-a-flaky-test-Github-issue) for why this should be fixed and how to go about it. - Failing CI run: https://... - Subsequent Passing CI run: https://... Failure output: ``` ...test failure... ```
…and fill out these placeholders:
URL of failed test run in GitHub Actions.
URL of passing run for the same commit.
Output from the failing test, including error message, stack trace, and anything else needed to recognize the same failure again. Build logs aren’t kept forever, and this also makes the text searchable.
Now we can remove the test!
...