[BB-2382] Allow using encrypted S3 buckets

Description

This PR optionally allows using encrypted S3 buckets everywhere S3 buckets can be used. For implementing this functionality, the `django-storages` library has been upgraded to `1.9.1`, the latest version at the time of creating this PR.

S3 encryption is disabled by default to allow existing instances and S3 buckets to work without any issues.

This PR supersedes the https://github.com/edx/edx-platform/pull/23471 PR created by @alanoe as a contribution from OpenCraft.

*JIRA tickets*: None

*Dependencies*: None

*Sandbox URL*: TBD - sandbox is being provisioned.

*Merge deadline*: None

*Testing instructions*:
1. With the following settings and this PR branch, create an encrypted S3 bucket and then create a sandbox configured to use that.
```
EDXAPP_DEFAULT_FILE_STORAGE: storages.backends.s3boto3.S3Boto3Storage
EDXAPP_LMS_ENV_EXTRA:
AWS_S3_ENCRYPTION: true
EDXAPP_CMS_ENV_EXTRA:
AWS_S3_ENCRYPTION: true

  1. This override is required because the `S3Boto3Storage` doesn't allow the value to start

  2. with a '/' and the default is '/edx/var/edxapp/media'.
    edxapp_media_dir: media
    EDXAPP_PROFILE_IMAGE_BACKEND:
    class: storages.backends.s3boto3.S3Boto3Storage
    options:
    headers:
    Cache-Control: max-age-{{ EDXAPP_PROFILE_IMAGE_MAX_AGE }}
    location: <S3 bucket name>/profile-images
    ```
    2. Verify that the following work:

  • Profile photo upload + rendering.

  • Forum image upload + rendering

  • Instructor dashboard report downloads.

  • Studio course export + import
    3. With this PR branch and the configuration in step 1 with just `AWS_S3_ENCRYPTION: true` removed, create an unencrypted S3 bucket and then create a sandbox configured to use that without enabling encryption. Verify that all the items mentioned in the previous step work okay.

*Author notes and concerns*:
1. The current default value of `edxapp_media_dir` is `/edx/var/edxapp/media` and when using S3, it has to be overridden to use a path that doesn't start with a leading `/` and represents the location within the bucket.
2. Once the changes in this PR are merged, some comments in the `edxapp` role within the `edx/configuration` repository have to be updated.

*Reviewers*

  • [ ] @viadanna

  • [ ] edX reviewer[s] TBD

*Settings*
```yaml
EDXAPP_DEFAULT_FILE_STORAGE: storages.backends.s3boto3.S3Boto3Storage
EDXAPP_LMS_ENV_EXTRA:
AWS_S3_ENCRYPTION: true
EDXAPP_CMS_ENV_EXTRA:
AWS_S3_ENCRYPTION: true
edxapp_media_dir: media
```

Assignee

Ned Batchelder

Reporter

Open Source Pull Request Bot

Labels

Contributor Name

Guruprasad Lakshmi Narayanan

Repo

edx/edx-platform

Customer

Epic Link

None

OSCM Assignee

None

Platform Map Area (Levels 1 &amp; 2)

Developer Experiences - Platform Services

Platform Map Area (Levels 3 &amp; 4)

None

Blended Hour Utilization Percentage

None

Priority

Unset
Configure