Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Once a release is created, it needs to be tested

  • Installation: each installation method must complete successfully
  • Automated tests
    • For devstack: run "paver test"


OpenCraft release testing procedures

OpenCraft uses the OpenCraft Instance Manager to test deployments and upgrades on OpenStack.

Overall procedure:

  1. Review the release notes, configuration changelog, and automatic upgrade scripts and determine if any configuration variables need to be updated etc.
  2. Deploy a new single-VM instance from scratch using the new release candidate and validate it (see below).
  3. Deploy a new multi-VM instance from scratch using the new release candidate and validate it (see below).
  4. Deploy/use a single-VM instance from the previous release, then change its branch to use the new release candidate, deploy the upgrade, and then validate it
  5. Deploy/use a multi-VM instance from the previous release, then change its branch to use the new release candidate, deploy the upgrade, and then validate it

Validation procedure for each deployment:

This manual testing procedure is designed to cover some of the more common failure modes / deployment errors encountered:

  1. Make sure user registration works.
    1. Make sure you received the activation email. If using an external SMTP server (e.g. SES), verify that the email was actually sent via that external service and not directly from the EC2 instance.
    2. Make sure that the activation link in the email works.
  2. Make sure course creation works.
    1. Log into the studio and create a new course.
    2. Add a new unit to the course.
    3. Make sure the "Preview" link works correctly.
  3. Test course in the LMS.
    1. Log into the LMS as a staff user.
    2. Click through each of the tabs ("Home", "Course", "Discussion", ...) to make sure the section loads correctly.
    3. Go to the "Course" tab and make sure the "View unit in Studio" link works correctly.
    4. Go to the "Discussion" tab and submit a new post, and add an image. Make sure the image is uploaded to S3/ObjectStorage successfully, and is displayed in the post correctly.
    5. Go to the "Instructor" tab and select "Data Downloads" section. Click the "Download profile information as a CSV" button and wait until the report is generated. Make sure the link to the report works correctly.
  4. Make sure tracking logs are being synced to S3 (We always want to sync tracking logs even if the instance is not currently using analytics. Note that tracking logs don't rotate until they're larger than 1Mb.)
  5. If using Insights: make sure link from "Instructor" tab points to Insights correctly and make sure that OAuth between LMS and Insights works.
  6. If using a custom theme:
    1. Make sure the theme looks ok in general.
    2. Pay special attention to discussion forums and the wiki (they are using different layouts).
  • No labels