This is an implementation proposal for Blockstore. The focus will be on defining a separation of concerns for Blockstore as an independently deployable application, and how its concerns should manifest. Where some functionality is delegated to an external service, a brief explanation of how the external component provides that service in the context of Blockstore is given.
Note: There are obvious kinks in this approach, but it's primary concern now is to help understand how we could approach a purely database-based Blockstore with assets still in a third party object storage service.
Blockstore Concerns
Blockstore will be fully dedicated to focusing on storing, referencing, managing, and/or allowing the discovery of Collections, Units, Assets, Links, and Versions. Each of these models has their own detailed section below explaining their purpose & role in Blockstore, as well as their implementation plan.
...
For reference, we group Units and Assets into "Linkables", and those plus Collections into and Links into "Versionables".
Collection
A Collection is an abstraction over a group of linkables to provide common metadata to that group.
Note that the fields in a Collection are mostly metadata, and although many can be argued to be foreign keys to separate models that give a bit more detail for each field, in this first iteration of an implementation draft we simplify things and only use CharFields – the actual implementation can use extra models as necessary.
Collections are versioned and thus derive from Version.
| Name | Type | Description |
|---|---|---|
uuid | UUIDField | Universally unique identifier for the Collection. |
name | CharField | A descriptive title for the Collection. |
description | CharField | A summary of what this Collection contains. |
owner | CharField | Who legally owns any original content in this Collection. |
author | CharField | Who is credited for authorship of original content in this Collection. |
license | CharField | The license bound to all content in this Collection. |
...
| Name | Type | Description |
|---|---|---|
uuid | UUIDField | Universally unique identifier for the Unit. |
| collection | ForeignKey(Collection) | The Collection that this Unit belongs to. |
| content | BinaryField | OLX content for this Unit. |
assetslinks | ManyToManyField(AssetContentType, through="Link")2 | All of the Assets that are linked to this Unit through Link. |
(1): Although this may clash with existing edX Studio vocabulary, we use it because it is the best fit for what it's referring to in practice. To prevent any confusion and be more complete one can say, "Blockstore Unit", however here and internal to Blockstore we stick to "Unit".
(2): I have no clue if this even works, but you get the idea.
Asset
Resources that aren't OLX, that're needed inside of the content defined by OLX, and that have a valid, reachable URL, can be added as Assets to Blockstore.
...
A single unit may require multiple assets in it, and a single asset may be used by multiple units. A Link is an intermediate model which facilitates the logic and metadata needed for the many-to-many relationship between units and assets.Notably, Links themselves are not versioned. Versioned Links would mean
Note that since Links are versioned, new Links would have to be created every single time a Unit is updated, for every single Asset linked single target that is linked to that Unit. It would add much more complexity to the code and be operationally difficult to manage, without buying us much feature-wiseThis could be quite stressful on the database and take up a lot of rows.
| Name | Type | Description | ||
|---|---|---|---|---|
uuid | UUIDField | Universally unique identifier for the Link. | ||
unitsource | | The Unit portion of this link. | asset | ForeignKey(Asset) |
The Asset portion of this linktarget | GenericForeignKey | The target portion of the link, which may be an Asset or a Unit. |
Version
A Version is an abstract model to be used by all "versionable" models for common fields and interfaces to versioning information.
| Name | Type | Description | |||
|---|---|---|---|---|---|
timestamp | DateTimeField(auto_now_add=True) | The time this version of the content was created. | |||
version_id | PositiveIntegerField1 | A counter identifying how many versions of this content have gone past and exist, by being automatically incremented. Can also be used for general identification purposes, but see footnote. | published | BooleanField | Whether the current version of this content is published or number identifying the version of the content in a history. When NULL, this version is a draft. |
garbage_collect | BooleanField | Whether this content's old versions should be garbage collected after some time. |
It's important to note that each time a versionable model is updated, another row is created in the database, because versionable content should be immutable. The flexible part is that this new row could be either the new version, or the very previous version (but we'd have to stick to one protocol).
A good reason this flexibility matters is because if Links aren't versioned, any Link should always point to the latest Unit. However if we create a new row containing the latest version of a Unit, we'd have to update the Link table to point its Unit references to the new row to maintain the requirement that Links point to the latest Units. But if we invert the logic and keep the same row for the latest version and only create new rows for older versions, Links would always be pointing to the latest version by default.
...
| . |
Blockstore REST API
Here we present a relatively simple CRUD REST API for the models discussed above.
...