Test Automation Efforts & Coverage for new Mobile Apps

Owned by Bilal Awan
Last updated: May 27, 2024
2 min read

Current Focus: Automate Critical Path

Our test automation repository (openedx-app-test) is designed to automate the end-to-end tesing of our applications efficiently.
Previously, both Android and iOS legacy apps had around 90% native screen coverage. However, as we are transitioning to new applications and preparing to launch a Minimum Viable Product (MVP), we cannot write automated test cases for the entire app due to its development phase. Therefore, we decided to focus on automating the critical path of the app in the MVP. This path is essential for user navigation from the Landing screen to the course content screen.

Tools and Technologies:

  • Tool: Appium for both Android & iOS

  • Framework: Pytest - Page object Model

  • Language: Python

Critical Path Screens:

  1. Landing

  2. Login

  3. Forgot Password

  4. Register

  5. What's New

  6. Main Dashboard

  7. Course Dashboard

  8. My Courses List

  9. Profile

  10. Logout

Testing Approach & Coverage:

For each screen, we ensure:

  • Navigation Testing: Ensures that users can navigate to the targeted page.

  • UI Testing: Validates the presence and correctness of all UI elements, including their values and strings.

  • Functional Testing: Ensures each element on the screen operates correctly (e.g., button clickability and functionality).

Test Execution Results:

We generate detailed HTML reports after each test run, which include:

  • Pass/Fail Status: Detailed logs and statuses of each test case.

  • Test Run Duration: Total time taken for each test suite execution.

  • Defects Identified: List of bugs or issues found during testing.

Next Steps

Creating CI Pipeline:

Our next major step involves setting up a Continuous Integration (CI) pipeline. This will allow us to run our test cases on every development commit, ensuring that any code changes are immediately validated. Key actions include:

  1. Integrating our test suite with GitHub Actions.

  2. Configuring the pipeline to trigger test execution on every commit.

  3. Automating the generation and distribution of test reports to the development team.

By implementing this CI pipeline, we aim to enhance our testing efficiency, catch defects early, and maintain a high level of quality throughout the development process.

Mobile API Testing:

Currently, the backend test cases are running locally. In case of any failure, the backend developer has to check the flow of all APIs to identify where the issue is occurring in that specific environment.

We want to run an end-to-end API test flow for critical Mobile APIs (IAP) in the staging environment. This will help us identify issues as they occur. Changes in the e-commerce repository will trigger the API test suite, execute the end-to-end flow, and share a report with the team if any failures occur.