Page Comparison

Search for repositories using edx-drf-extensions

...

Note: The api search may not search private repos.

#!/usr/bin/python
from github import Github

# Set this to a personal access token.
# - Select "repo" for the oauth scopes.
# See https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/
g = Github('<YOUR_ACCESS_TOKEN>')

repositories = set()

# Note: Gets rate limited and fails if too many hits
content_files = g.search_code(query='org:edx jwtbuilder')
for content in content_files:
	repositories.add(content.repository.full_name)
	rate_limit = g.get_rate_limit()
	if rate_limit.search.remaining == 0:
		print('Rate limit on searching was reached.')
		break

for repo in sorted(repositories):
	print(repo)

rate_limit = g.get_rate_limit()
print('Search rate limit:')
print(rate_limit.search)

...

...

List of repositories by IDA

The following is a list of IDAs that need to be deployed with updates, including dependencies that may need to be updated.

• edx/course-discovery (IDA)
• edx/credentials (IDA)
• edx/ecommerce (IDA)
• edx/edx-analytics-dashboard (IDA)
• edx/edx-analytics-data-

...

• api (IDA)
  • edx/edx-enterprise-data

...

• • (dependency)
• edx/

...

• edx

...

• -notes-

...

• api (IDA)
• edx/edx-

...

OUT WITH NEXT DEPLOYMENT:

...

• platform (IDA)
  • edx/completion (dependency)
  • edx/edx-

...

• • enterprise (dependency)
  • edx/

...

• • edx

...

• • -organizations (dependency)
• edx/edx-video-

...

• Merged - waiting for deployment

BLOCKED:

edx/ecommerce

• Prod deployments broken (see DEVOPS‌-7953)

Learnings:

...

• pipeline (IDA)
• edx/journals (IDA)

General notes regarding updating repositories

• Creating PRs
  • Expect 0.5-1.5 hours to set up all PRs for upgrades/changes.
    • Longer if all repositories and not just IDAs.
  • Some repos have "make upgrade" available, and some don't.  Handling dependency updates is hit or miss, and can sometimes eat up a lot of extra time.
  • Some speed up when you already have all repos locally and tools to commit across repos (e.g.

...

DEPLOY

...

• • PyCharm).
• Deploying IDAs
  • Expect 0.5-1 hour for IDA deployments.
    
    • Longer if you run into weird deployment issues.
  • All IDAs are deployed from GoCD, except as noted below.
  • Most IDAs have a "you merge it, you deploy it" rule, so deployments should go out as soon as you merge.
  • Some repositories must be rebased before you can merge (ecommerce, course-discovery).
    • Be ready to merge as soon as your rebase passes tests so someone else doesn't merge first.  I had to rebase some PRs several times, even while waiting to merge asap.
  • Some IDAs have unique channels for communicating deployments.  It would be better if this were automated.
  • edx/edx-analytics-dashboard (IDA)
    • Master branch is currently RED.
    • ARCH‌-252 has list of pending PRs to be merged and link to EDUCATOR ticket for fix.  EDUCATOR ticket is actively being worked on.
  • edx/edx-analytics-data-api (IDA)
    • Has no e2e tests.  Use portal to test, and review NewRelic (or Splunk) for errors in stage and prod.
  • edx/edx-notes-api (IDA)
    • Has no e2e tests.  Create a note (in stage) to test and review NewRelic (or splunk) for errors in stage and prod.
  • edx/edx-platform
    • Deployment handled by devops for now.
  • edx/edx-video-pipeline
    • Deployment is handled manually by Educator team.  Schedule with Simon or Scott.
  • edx/journals
    • Deployment handled by White Label for now.  You can let them know after you merge if you want.

Complete list of repositories (as of 10/2018)

This list details the search results that matter and those that can be ignored.

• edx/completion
• edx/course-discovery
• edx/credentials
• edx/ecommerce
• edx/edx-analytics-dashboard
• edx/edx-analytics-data-api

...

• edx/edx-drf-extensions (IGNORE - SELF)
• edx/edx-enterprise
• edx/edx-

...

• enterprise-data
• edx/edx-notes-api

...

TO MERGE (see new related tickets)

...

• edx/edx-organizations
• edx/edx-platform
• edx/edx-platform-private (IGNORE)
• edx/edx-

...

• video-

...

• pipeline
• edx/journals
• edx/programs (IGNORE - ARCHIVED)
• edx/testeng-ci (IGNORE)