We need a better convention for returning API results from collection (LIstView-style) endpoints.
Currently, when we return a collection of resources in an API, we use the default formatting provided by DRF, which returns a direct JSON list of results. For example:
[
{"id": 1, "item": "First item", "created": "2015-11-19T09:28:44Z"},
{"id": 2, "item": "Second item", "created": "2015-11-19T09:29:03Z"},
{"id": 3, "item": "Third item", "created": "2015-11-19T09:31:30Z"},
{"id": 4, "item": "Fourth item", "created": "2015-11-19T09:35:12Z"}
]
If we then paginate the results, using DRF's default pagination the above gets converted a JSON object that contains "previous" and "next" elements, which contain links to the previous and next pages respectively, a "count" element that contains the total number of results on all pages in the query, and a "results" element that contains the current page's results. If the above example were paginated with three elements per page, the response would look like:
{
"previous": null,
"next": "/path/to/results?page=2",
"count": 4,
"results": [
{"id": 1, "item": "First item", "created": "2015-11-19T09:28:44Z"},
{"id": 2, "item": "Second item", "created": "2015-11-19T09:29:03Z"},
{"id": 3, "item": "Third item", "created": "2015-11-19T09:31:30Z"}
]
}
This is unsatisfactory for a few reasons:
I propose that we update our API conventions for collection endpoints to:
Always return a JSON object, with an element "results", that contains a list of result items, whether or not the result set is paginated.
This puts the actual results in a consistent location within the response, and provides a convenient namespace for the pagination metadata. Any given result can be tested for pagination very easily: If the response object contains a "pagination" element, the results are paginated.
Unpaginated:
{
"results": [
{"id": 1, "item": "First item", "created": "2015-11-19T09:28:44Z"},
{"id": 2, "item": "Second item", "created": "2015-11-19T09:29:03Z"},
{"id": 3, "item": "Third item", "created": "2015-11-19T09:31:30Z"}
]
}
Paginated:
{
"pagination": {
"count": 4,
"previous": null,
"next": "/path/to/results?page=2"
},
"results": [
{"id": 1, "item": "First item", "created": "2015-11-19T09:28:44Z"},
{"id": 2, "item": "Second item", "created": "2015-11-19T09:29:03Z"},
{"id": 3, "item": "Third item", "created": "2015-11-19T09:31:30Z"}
]
}
Non-paginated responses could be adapted to the new scheme in one of two ways:
Paginated responses can be handled writing a custom paginator that overrides the get_paginated_response() method to return the desired format. It should handle serializers that return a flat list of results as well as objects with a "results" element.
Soon:
Initially, we update unpublished APIs (discussion api, course catalog api) to explicitly call the new paginator/renderer classes.
Existing published APIs explicitly define their paginator/renderer as the built-in DRF classes they currently use.
Eventually:
As this is all handled by Paginators, and Renderers, which are defined as class attributes on DRF generic views, and superclasses of our Serializers, we would very likely be able to support multiple versions of the API (where the format is the only change) without much duplicate code by creating views that differ only by a superclass and/or an attribute or two. Pseudocode example:
class View(ListAPIView):
paginator = PageNumberPaginator
# renderer = JSONRenderer
def list(self, request):
return Serializer([1, 2, 3])
class V2View(View):
# Our new paginator and renderer
paginator = NestedPageNumberPaginator
renderer = JSONResultsRenderer
urls = urlpatterns('',
url(r'/api/sample_api/v1/objects/', View.as_view()),
url(r'/api/sample_api/v2/objects/', V2View.as_view()),
)
We could remove duplication by creating a NewPaginationMixin
for V2Views to inherit from:
class View(ListAPIView):
paginator = PageNumberPaginator
def list(self, request):
return Serializer([1, 2, 3])
class NewPaginationMixin(object):
paginator = NestedPageNumberPaginator
renderer = JSONResultsRenderer
class V2View(NewPaginationMixin, View): pass
If we want to leverage existing standards, we could alternatively structure our responses to conform more to something like JSON API http://jsonapi.org/. In this format, our response would look like:
{
"data": [
{
"type": "result":
"id": 1,
"attributes": {"item": "First item", "created": "2015-11-19T09:28:44Z"}
},
{
"type": "result",
"id": 2,
"attributes": {"item": "Second item", "created": "2015-11-19T09:29:03Z"}
},
{
"type": "result",
"id": 3,
"attributes": {"item": "Third item", "created": "2015-11-19T09:31:30Z"}
}
],
"links": {
"previous": null,
"next": "/path/to/results?page=2"
},
"meta": {
"count": 4
}
}
Advantages of this format include:
Disadvantages of this format include:
content = content['results']
.