How to: Create and Deploy a new Microfrontend inside edX
Create Your Microfrontend's Repository
Note that throughout this How-to we use XXX to represent the name of your repository/app, and YYY to represent the sub-domain you've chosen.
Pick a name. It should follow the convention:
frontend-app-XXX
.We have a GitHub template repository to help bootstrap your new application.
If you have permission to create new repositories:
Click the "Use this template" button.
Put in your chosen micro-frontend application name.
Make sure to create it in the edX organization.
If you do not have permission to create a new repository:
Get someone to do the above steps for you. You can read through How to request a new GitHub Repo to find a way to get someone to help you.
Do a find and replace in your new repository to replace frontend-template-application with frontend-app-XXX.
Similarly, there's an example module which demonstrates a basic module. Feel free to rename the "example" directory or delete it.
Add a link to your new repo to Frontend Repos.
If you are replacing an existing legacy frontend, update [Archive] MFE Rewrite Statusarchived accordingly.
Setup Automated Deployment
We serve our frontend applications from S3 through CloudFlare distributions. We deploy them with GoCD. We manage the creation of these resources/pipelines via configuration in edx/terraform and edx/edx-internal.
In this section you will be creating a series of pull requests for the following:
The AWS resources like the S3 Bucket and Route53 domain name
To add these you will need to open a PR to https://github.com/edx/terraform (e.g. https://github.com/edx/terraform/pull/2728/files)
The Route53 domain and S3 Bucket are created using the
frontend.hostname
variable (the creation of which you can see in the above PR)
The remote config blocks for Related Django services
To add these you will need to open a PR to https://github.com/edx/edx-internal and optionally https://github.com/edx/edge-internal (e.g. https://github.com/edx/edx-internal/pull/2374/files)
The GoCD deployment pipelines
Sandboxes
Beware of dragons, this will require a PR to https://github.com/edx/configuration , please see How To Add an MFE to a Sandbox for more information.
After all your PRs are made, you will create an SRE Support ticket or a ticket to your eSRE for review.
1: edx/terraform
Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. This repo contains terraform modules for deploying our applications on AWS, Cloudflare, NewRelic, Kubernetes and Snowflake.
Here we will leverage an existing frontend terraform module for provisioning new frontend infrastructure.
Create a branch/PR with the following changes:
In edx/terraform, create two files:
plans/edx/prod-frontends/frontend-app-XXXXX.tf
variable "frontend_app_XXXXX_hostname" { type = string description = "Hostname of the frontend-app-XXXXX used for bucket naming and routing" } module "frontend_app_XXXXX_frontend" { source = "../../../modules/frontends/frontend" cloudflare_record_name = "XXXXX" route53_zone_id = var.route53_zone_id_edx_org frontend_hostname = var.frontend_app_XXXXX_hostname internal_frontend_hostname = "${var.environment}-XXXXX.edx.org" aws_region = var.aws_region deployer_user = aws_iam_user.frontend_deployer_user.arn cloudflare_zone_id = var.cloudflare_zone_id_edx_org cdn_type = "cloudflare" }
plans/edx/stage-frontends/frontend-app-XXXXX.tfvariable "frontend_app_XXXXX_hostname" { type = string description = "Hostname of the frontend-app-XXXXX used for bucket naming and routing" } module "frontend_app_XXXXX_frontend" { source = "../../../modules/frontends/frontend" cloudflare_record_name = "XXXXX.stage" route53_zone_id = var.route53_zone_id_edx_org frontend_hostname = var.frontend_app_XXXXX_hostname internal_frontend_hostname = "${var.environment}-XXXXX.edx.org" aws_region = var.aws_region deployer_user = aws_iam_user.frontend_deployer_user.arn cloudflare_zone_id = var.cloudflare_zone_id_edx_org cdn_type = "cloudflare" }
Note the only difference between the stage plan and the prod plan is the
cloudflare_record_name
. For examples, see the existingplans/edx/prod-frontends/frontend-app-*
configurations.Add variables for your application for stage and prod:
plans/edx/prod-frontends/terraform.tfvars
... # frontend-app-XXXXX frontend_app_XXXXX_hostname = "YYYYY.edx.org"
plans/edx/stage-frontends/terraform.tfvarsOptional for edge deployment
2: edx/edx-internal (and edge-internal)
EdX Internal is a private configuration repo for the edx.org deployment. It contains private but not restricted/secure data. Here we will add configuration variable files for your frontend application to edx-internal/frontends.
Create a branch/PR with the following changes.
Create a subdirectory in edx-internal/frontends for your app:
edx-internal/frontends/frontend-app-XXX
Add the following two files:
prod_config.yml
stage_config.yml
For examples, see the existing
frontend-app-*
configurations in edx-internal/frontends.
If you're setting up NewRelic alerts, or Segment tracking there's additional configuration to define (NEW_RELIC_APP_ID
and SEGMENT_KEY
) in this repository that's described below. You may want to do it as part of the same PR to save time.
Add some variables for your application in
EDXAPP_XXX_MICROFRONTEND_URL
is used to let the LMS link to your frontend.EDXAPP_CORS_ORIGIN_WHITELIST
allows requests to LMS from your frontend.EDXAPP_LOGIN_REDIRECT_WHITELIST
allows the LMS login process to redirect to your frontend.edx-remote-config/prod/lms.yml
Do similarly in
Optional: ecommerce access
If you need to call ecommerce, you need to add your domain to the CORS_ORIGIN_WHITELIST variables
Add the following changes to your edx-remote-config PR/branch.
Add your domain in:
https://github.com/edx/edx-internal/blob/master/edx-remote-config/prod/ecommerce.yml
https://github.com/edx/edx-internal/blob/master/edx-remote-config/stage/ecommerce.yml
Also add it in ansible/vars/stage-edx.yml
Optional: edge-internal configuration for edge deployment
GoCD Pipeline Steps
We define our GoCD pipelines and their stages, jobs, and tasks in yaml. Here we will provision GoCD deployment pipelines for our app.
Create a PR/branch with the following changes.
Add your frontend application to in order to provision Go CD pipelines.
See:
https://github.com/edx/edx-internal/blob/master/gocd/README#L31-L39Your pipelines will be generated once your PR is merged.
Enable edge deployment
4: edx/configuration (optional)
This repository is a collection of tools and scripts that edx.org uses to deploy openedx. The purpose of this repository is to share portions of our toolchain with the community. This directory contains ansible playbooks that can be used to configure individual services in the openedx platform.
Create a PR/branch with the following changes:
Edit playbooks/roles/edxapp/defaults/main.yml to add the microfrontend URL defined above in edx/edx-internal.
Also add it to the
lms_env_config
in the same file:
Setup Devstack
You should be able to run your micro-frontend out-of-the-box with:
However, the next two optional steps may make development a bit more seamless.
Allow LMS login redirection in Devstack
Add the netloc (
localhost:$PORT
) of your new micro-frontend to theLOGIN_REDIRECT_WHITELIST
for Devstack LMS.The setting can be found in https://github.com/edx/edx-platform/blob/master/lms/envs/devstack.py.
The setting indicates which sites it is acceptable to redirect back to after a user logs in.
(You may recall that you did something similar for Stage and Prod in the edx/edx-internal step).
Add your MFE to Devstack
Adding your MFE to the devstack repository is optional. The main benefit is that you will be able to bring up the MFE and any required backend services with a singular command: make dev.up.frontend-app-XXXX
. Whether or not you do this step, you should be able to run your MFE directly with npm install && npm start
.
If you choose to add it to Devstack, here is an example PR to follow: https://github.com/edx/devstack/pull/556/files. For the depends_on
key in docker-compose.yml
, specify the backend services that your MFE needs running in order to function.
Setup Segment
See Setting up new Segment projects
See "frontend-app-profile-prod" and "frontend-app-profile-stage" as examples (that use LMS property in Google Analytics).
To create a new source, file a support ticket with Data Engineering. Make sure that you have the necessary permissions to view the write key associated with the source.
Add the Segment write keys (
SEGMENT_KEY
) to prod_config.yml and stage_config.yml in your edx-internal/frontends sub-directory. You created it above during "Setup Automated Deployment".Make sure that you are using
frontend-platform
's builtin support for segment page tracking by usingPageRoute
, which is a drop-in replacement for react’s normalRoute
. (Or alternatively,AuthenticatedPageRoute
for login-required pages.)Verification:
Using Segment Debugger for the new Segment sources (stage and prod), verify that identify/page events are being sent to Segment.
To verify Prod events, go to Snowflake and verify your events are appearing. Or have on-call for (Slack) #data-engineering do it, if it’s a one-off and you don’t have access.
Setup New Relic
We use this procedure to insert Browser's JavaScript snippet for browser monitoring into our frontends, see New Relic’s docs for more details.
See "prod-frontend-app-profile" and "stage-frontend-app-profile" for example configurations.
Go to one.newrelic.com > Browser > Add more data
Click on “New Relic Browser”.
Select Copy/Paste JavaScript Code, leave Pro + SPA checked and switch on Distributed Tracing.
Enter an app name as "prod-frontend-app-XXX" and click Enable
In the Javascript snippet that appears, copy the "applicationID" value out of the last line before the closing </script> tag:
Add the NewRelic application ID you just copied to prod_config.yml as its
NEW_RELIC_APP_ID
value. You created this file during "Setup Automated Deployment" in your edx-internal/frontends sub-directory.Repeat steps 1 through 6 for stage_config.yml. You should have a different application ID for staging.
Setup New Relic alerts in Opsgenie
FYI – Alerts used to be set up via terraform via terraform/plans/newrelic/edx/channels.tf
If your team doesn’t already have a New Relic integration, go to Opsgenie > Teams and choose a team
Click integrations.
Click Add Integration
Search for New Relic (New) and click add
Follow the instructions on the page to use the api key to add a notification channel in New Relic.
Go to New Relic Browser
Under your app go to alert conditions
Click Manage Alert Policies and search for the name of your app to find its alert policy
Add an Apdex and JS Error conditions as seen here https://one.nr/0mMRNPPqlQn for prod-frontend-app-learning
Add a New Relic notification channel
If your team doesn’t already have one, you may need to ask your eSRE or file an SRE support ticket to copy in the API key from the Opsgenie integration.
Create a support ticket with SRE for your PRs
At this point you'll want to create an SRE ticket to review your PRs and give you a sanity check on everything else.
The creation of the ticket depends on whether you have an eSRE on your team. If you do & you’re not sure what process to follow, consult them.
If your team does not have an eSRE, then you should create a SRE Support ticket via the old process.
In filing the ticket, include links to all the PRs you created above.
Include also your Opsgenie New Relic API Key if you want to set up alerting.
Add Annotated Feature Toggle
This is only needed if you want a rollout feature toggle (from LMS for example) to your new page.
Sample annotated rollout feature flag.
Configure i18n Jenkins and Transifex
Follow
New docs in @edx/frontend-platform https://github.com/edx/frontend-platform/blob/master/docs/how_tos/i18n.rst
Old docs for @edx/frontend-i18n i18n how-to documentation
If you created the shell of your application code as described above, many of the steps in this documentation will already be done for you.
Actual i18n code should be completed separately as strings are added to the code.
Backend JWT Cookies Support
At this point, most IDAs support JWT cookie, but not all.
Credentials(?) and Notes do not yet support JWT cookies for example.
See https://openedx.atlassian.net/browse/ARCH-266 for details on how to add support.