Background
This document captures the conventions to be used for all edX REST APIs.
Useful reading:
High Level Requirements
Consumer-Perspective. Design your API from the perspective of the consumer, NOT the perspective of your underlying implementation. For example:
If the underlying implementation requires accessing multiple models or multiple apps/projects, this does not need to be reflected in a public interface. From the perspective of the consumer, it's simply one thing they are requesting.
Keep CRUD operations together within its corresponding resource. Why have the client go to one resource to read it and then another resource to write it?
Simple. Keep the top-leveI resources clear and simple - focusing on what the client is looking to consume. You can hide complexity within the parameters.
Separation of concerns. Do not require the consumer to know any implementation details. For example, an API shouldn't require the client to know the inter-dependencies of fields. Rather, all such business rules should be owned by the server. This allows the client to be lightweight and easily maintainable in the future.
Explicit Support Levels. APIs should make it clear whether they are experimental or more permanent as a part of their documentation.
Discoverability. Support HATEOAS where possible. This allows us to change our URLs without needing to worry about updating the mobile apps if the app discovers its URLs from a base URL.
Conventions
1. URL Naming
Must: Keep your base URL simple and intuitive.
Suggestion: Follow this basic URL structure if you are concerned about collisions within your service. By "collisions", we mean TBD.
/api/{API_NAME}/{VERSION}/... |
If collisions are not a concern (for example, when exposing APIs from context-specific IDAs), then you can eliminate the use of API_NAME.
Must: Keep the API name flat. Use two base URLs per resource: collection/identifier, e.g.
/api/user/v1/accounts/{USERNAME} |
It is not necessary that there is a one-to-one correspondence between an API and the Django app that provides it. In fact, which Django app implements an API is an implementation detail and need not be bound to a public interface.
Resource name
Keep verbs out of the base URL
Must: Use plural rather than singular nouns
Must: Use concrete (e.g., blogs, videos, news) rather than abstract (e.g., items, assets) names
Must: Use Python conventions: use_underscores instead of CamelCase
Verbs
When no resource is involved, be direct and use Verbs instead of Nouns (p 19). But only when absolutely necessary as we try to use nouns as much as possible.
/convert?from=EUR&to=CNY&amount=100
Make it clear in your API documentation that these “non-resource” scenarios are different.
See list of example APIs at the end of this document.
2. Identifying Resources
Nice: Do not expose database IDs where possible (Must for external APIs, per dev ops)
Must: edX resource identifiers
Explicit Filters
Model system resource URL schemes as if all resources are available to all users
Include all necessary filters in the URL such that any user could theoretically access the resource
filtering and authorization – ie, do not return different resource representations via the same URL based on the requesting user
Example 1: Choose "/profiles/john_harvard" over "/profile" |
Example 2: Choose "/courses?username=john_harvard" over "/courses" |
See relevant conversation here for more context.
Remember, it's entirely possible that "/profiles/john_harvard" or "/courses?username=john_harvard" could be requested by the "administrator" user
Several benefits exist from an explicit filtering approach:
Ensures resources/results are individually-addressable
Enables discovery/sharing of resources, among other potential uses
Resource filtering mechanisms can be modified without impacting authorization mechanisms
Intermediate network gear can cache resource representations to improve performance (for open/unprotected resources)
Composite Resource (with multiple dimensions)
If an endpoint represents a relationship among multiple dimensions, the dimensions can be specified in the following ways:
As filter parameters:
/api/enrollments/v1/enrollments/?user={username}&course={course_id} |
As comma-delimited resource keys in the URL:
/api/enrollments/v1/enrollments/{username},{course_id} |
As a UUIDs to uniquely identify the relationship:
/api/enrollments/v1/enrollments/{enrollment_id} |
3. HTTP Verbs
A note on HTTP PATCH
"Plain" HTTP PATCH (RFC 5789) is neutral with respect to content type and only specifies that the request body provide instructions for how to update the resource in question, not the format of those instructions. Several flavors of PATCH specific to JSON documents have more explicit definition, as noted above.
PATCH handling fairly closely resembles what is specified by merge patch, but it does not require (or understand) the "application/merge-patch+json" media type. In edX REST APIs, if an implementation will use DRF's default PATCH handling, the implementation MUST recognize and accept the merge patch media type; API clients SHOULD use this media type, preferring it to the more typical "application/json" type.
The reason for this stricture is to explicitly leave room for supporting multiple PATCH styles in any given API, should this become desirable, without breaking existing clients.
4. URL Parameters
Must: Sweep complexity behind the ‘?’ (e.g., GET /dogs?color=red&state=running&location=park) (p 9)
fields parameter - Allow clients to specify/filter the fields in the response by supporting a fields parameter as a comma-delimited list.
"Partial Response allows you to give developers just the information they need." (p 16)
/dogs?fields=name,color,location
/dogs?fields=title,media:group(media:thumbnail) |
expand parameter - Allow clients to request including data from other resources using the expand parameter as explained further in #expand.
requested_fields parameter - Allow clients to request additional optional fields that will be added to the response. (e.g. GET /comment?requested_fields=author_profile_image,endorse_profile_image)
text_search parameter - Allow clients to perform text-based search using this parameter. (p 22)
global: /search?text_search=fluffy+fur
scoped: /owners/5678/dogs?text_search=fluffy+fur
formatted: /search.xml?text_search=fluffy+fur
5. Errors
6. Version
Location.
Put the version in the URL so the client can see it easily when handling the response logic.
Exception: Put it in the header only if it doesn't change the response handling logic, like the OAuth endpoint.
Numbering.
Must: Versions are major only ex. /v0/ /v1/ /v2/
Must: Major version should only be updated when changing the existing contract in a way that breaks backwards compatibility. The previous major version should continue to be supported for old clients according to the service's deprecation policy.
Must: Additive changes to the contract of the API, or the results, should not bump the major version, unless those changes are non-optional, and break existing contracts with the API.
Future considerations:
(TODO) Responses should have a header containing minor / patch versions.
Deprecation Process: varies from project to project based on product requirements and developer usage, client usage, etc. Should be well documented.
7. Pagination
We use the conventions established by DRF, with some extra fields:
next: A URL to the next page, or null.
previous: A URL to the previous page, or null.
count: The total number of items.
num_pages: The total number of pages.
results: List of results.
The query parameters are as follows:
page - the page of results (zero indexed)
page_size - the number of results to return per page
sort_order - (no direction modifiers)
8. Documentation
The edX API Documentation Tools are installed into LMS, CMS, Registrar, and some other services.
The doc tools will auto-generate documentation for existing APIs under /api/ by looking at the path, serializers, and other metadata it can discern from the code.
Once installed, the doc tools can be used to enrich, modify, or exclude documentation for particular API views.
Many API views still use long, metadata-filled, formatted docstrings to document APIs.
In the past, this was our strategy for generating API documentation; however, the tools we previously used to generate structured documentation from docstrings are no longer supported.
These long docstrings are much better than nothing! However, to avoid duplicating information, these docstrings can be shortened once the endpoint is documented using the edx-api-doc-tools library.
9. Discoverability
Use hyperlinks (with absolute URLs) to represent (primary/foreign) keys.
When there are links to other resources, we should have a hyperlink in the response.
10. Multiple Formats
11. Authentication
By convention, our REST APIs support the following two authentication schemes:
12. Serialization Conventions
To Be Discussed
Expanding referenced objects
APIs can support expanding related objects, e.g. the optionally includes user profile information for each member. This is preferable to requiring the client to make multiple subsequent AJAX calls.
For example, there can be three levels for a related object (using "username" as an example):
"user": {
"username": "andya",
"url": "https://openedx.example.com/api/user/v1/accounts/andya"
} |
"user": {
"username": "andya",
"country": "UK",
"profile_image": {
"has_image": True,
"image_url_full": "http://my-image-storage.net/media/profile_images/123456789_500.jpg",
"image_url_large": "http://my-image-storage.net/media/profile_images/123456789_120.jpg",
"image_url_medium": "http://my-image-storage.net/media/profile_images/123456789_50.jpg",
"image_url_small": "http://my-image-storage.net/media/profile_images/123456789_30.jpg",
},
...
} |
By default:
For objects that have their own APIs, use option (b) and return both an id and a URL
For other objects, use option (a) and just return the id
Support a query parameter indicating an optional list of objects to be expanded: e.g. ?expand=user,team
Review Process
APIs must go through a community review process and review by the architecture council before being accepted into the platform. See the Architecture Review Process for the process for gaining approval.
Example APIs
There are a number of existing APIs that you can crib from:
Here are some representative URLs
/api/course/v1/courses/
/api/enrollment/v1/enrollments/
/api/discussion/v1/threads/{thread_id}/ |