OAuth2 Protocol and JWT

The mobile app Open edX platform uses the standard OAuth 2.0 protocol for authenticating users to the open edX LMS. The OAuth2 protocol supports authenticating the "client" and/or the "resource owner" by verifying "shared credentials" that were exchanged out-of-band with the "authorization server".  A successful transaction results in a valid "access token". This access token is then used to create a JWT token that the client uses for accessing resources on the "resource server" on behalf of the "resource owner".

In the mobile context:

• the client is the mobile device

• the resource owner is the user using the mobile device

• the resource server is LMS or any service hosted on the edX platform that exports a public API to be called by the mobile app

• the shared credential is either
  

  • the user's "password" (if the user has an edX password), or

  • the user's "social auth credentials" (for 3rd party authentication)

• the authorization server is either

  • LMS (if authenticating with the user's password), or

  • a 3rd party server (if authenticating with a 3rd party social auth)

• the access token is used to authenticate the user in subsequent API calls

Note: We use the term "authenticate" loosely here since we are not necessarily verifying the identity of the caller for all API calls.  In fact, it's possible that the user is not even present when an API call is made.  This is why the OAuth protocol is actually considered an "authorization" framework (or "delegation" to be more exact) since the "resource owner" authorizes the "client" to access resources on the "resource server" on its behalf by virtue of an "access token" obtained from an "authorization server".  However, I still prefer to use "authentication" in this context since from the resource server's perspective, the access token is simply used to "identify" the resource owner on whose behalf the request is made.  Further authorization-related matters such as permission-checking and ACLs are outside this scope.

Mobile API Authentication Classes

All APIs called by the edX mobile app are authenticated using the OAuth2AuthenticationAllowInactiveUser authentication JwtAuthentication authentication class.  It verifies the caller has a valid OAuth2 token and bypasses verification of their email address.  For a streamlined on-boarding experience, the mobile app supports ongoing usage of its features without ever requiring the user to verify their email address.

Additionally, while most edX views support session authentication, some of those APIs are decorated with the SessionAuthenticationAllowInactiveUser authentication class, which also bypasses email verification.  For example, the enrollment API doesn't require the user to be "active" (i.e., doesn't need a verified email address).

Note: The confusion over "active"/"inactive" versus "email-verified"/"non-verified" users strives from the fact that the "is_active" field on a user object is dual-purposed for both user states: email verification (user-initiated) and user deactivation (devOps-initiated).  This means that if we do deactivate users (devOps-initiated), that would also be bypassed by OAuth2AuthenticationAllowInactiveUser.

OAuth2 Access Tokens

The mobile app obtains an edX-issued access token in either of the following ways:

• AccessToken View: username/email + password combo with grant_type=password
  
  • curl -X POST -d "client_id=INSERT_CLIENT_ID&grant_type=password&username=INSERT_USERNAME&password=INSERT_PASSWORD" http://localhost:8000/oauth2/access_token/
    
• AccessTokenExchangeView: 3rd party (social-auth) OAuth 2.0 access token -> 1st party (open-edx) OAuth 2.0 access token

...

Transition from Bearer tokens to JWT tokens

Edx mobile apps have been transitioned to use JWT tokens to access edx API's and Bearer tokens have been deprecated. JWT token is obtained by passing token_type=JWT in the payload to the /oauth2/access_token/ API as shown below in the document.

To support this transition, a temporary Waffle Switch has been introduced called oauth_dispatch.disable_jwt_for_mobile. If this switch is enabled, JWT is disabled and hence the /oauth2/access_token/ API returns Bearer token in response. This switch is intended to be enabled only in case of any error/issue reported during JWT launch to mobile end users, and is by default kept disabled.

OAuth2 Access Token → JWT token

Access tokens are created for each login attempt from mobile, which are then used to create JWT tokens used by mobile app in api calls. The mobile app can issue JWT tokens in either of the following ways:

• AccessToken View: username/email + password combo with grant_type=password
  

  • curl -X POST -d "client_id=INSERT_CLIENT_ID&grant_type=password&username=INSERT_USERNAME&password=INSERT_PASSWORD&token_type=JWT" http://localhost:8000/oauth2/

exchange_
• LoginWithAccessTokenView: 1st party (open-edx) OAuth 2.0 access token -> session cookie
  • Returns a 204 (no content), but with the user's session cookies in the response.
  • curl -X POST -H "Authorization: Bearer INSERT_EDX_ACCESS_TOKEN" --cookie-jar TEST_COOKIE_JAR.txt

    access_token/

INSERT_BACKEND
• For now, the supported backends are "facebook" and "google-oauth2"

Example Response

The response from either of the above endpoints would provide the edX-issued access token as follows:

{"access_token": "5e0a0cb315e66aa96bab910faa8c70ee0ca91236", "token_type": "Bearer", "expires_in": 2591999, "scope": ""}

Authorization Bearer

Once an access token is obtained, it can be used to authenticate the user in any API call that supports the OAuth2AuthenticationAllowInactiveUser authentication class.  The access token is passed in the Bearer field of the Authorization HTTP header, as follows:

• curl -H "Authorization: Bearer INSERT_EDX_ISSUED_ACCESS_TOKEN" http://localhost:8000/api/..

Expiration

Currently, our edX-issued access tokens expire in 30 days for public clients like mobile apps, and 365 days for server-side clients.  These values can be overridden with the settings variables OAUTH_EXPIRE_DELTA_PUBLIC (for public clients) and OAUTH_EXPIRE_DELTA (for confidential clients), which each take a timedelta object for their value.

OAuth2 -> Session Cookie

Additionally, the mobile app can exchange an access token for a session cookie that can be used in a WebView:

• • 
    

• AccessTokenExchangeView: 3rd party (social-auth) OAuth 2.0 access token -> 1st party (open-edx) JWT token

  • curl -X POST -d "client_id=INSERT_CLIENT_ID&access_token=INSERT_THIRD_PARTY_ISSUED_ACCESS_TOKEN&token_type=JWT" http://localhost:8000/oauth2/exchange_access_token/INSERT_BACKEND

  • For now, the supported backends are "facebook" and "google-oauth2"

Example Response

The response from either of the above endpoints would provide the edX-issued JWT token as follows:

{
 'access_token': 'eyJhbGciOiJIUzI1NiJ9.eyJhdWQiOiAibG1zLWtleSIsICJleHAiOiAxNjkxNjgyMTA3LCAiZ3JhbnRfdHlwZSI6ICJwYXNzd29yZCIsICJpYXQiOiAxNjkxNjc4NTA3LCAiaXNzIjogImh0dHA6Ly9sb2NhbGhvc3Q6MTgwMDAvb2F1dGgyIiwgInByZWZlcnJlZF91c2VybmFtZSI6ICJmZWFuaWwiLCAic2NvcGVzIjogWyJyZWFkIiwgIndyaXRlIiwgImVtYWlsIiwgInByb2ZpbGUiXSwgInZlcnNpb24iOiAiMS4yLjAiLCAic3ViIjogIjVjMTBmNjZmMmQ2MzkwYjcwNjYyYzkxNGFhZTdlZjc5IiwgImZpbHRlcnMiOiBbInVzZXI6bWUiXSwgImlzX3Jlc3RyaWN0ZWQiOiBmYWxzZSwgImVtYWlsX3ZlcmlmaWVkIjogdHJ1ZSwgImVtYWlsIjogImZlYW5pbEBheGltLm9yZyIsICJuYW1lIjogIkZlYW5pbCBQYXRlbCIsICJmYW1pbHlfbmFtZSI6ICIiLCAiZ2l2ZW5fbmFtZSI6ICIiLCAiYWRtaW5pc3RyYXRvciI6IHRydWUsICJzdXBlcnVzZXIiOiB0cnVlfQ.1poWa9J7iCLVkU5nPNiBUD9WldDQyRWrLLRJ_gtdnEY',
 'expires_in': 3600,
 'token_type': 'JWT',
 'scope': 'read write email profile',
 'refresh_token': 'pGwqlJXuLUkP1KqtVQJm0nU3TKss9q'
}

Authorization JWT

Once an JWT token is obtained, it can be used to authenticate the user in any API call that supports the JwtAuthentication authentication class.  The JWT token is passed in the JWT field of the Authorization HTTP header, as follows:

• curl -H "Authorization: JWT INSERT_EDX_ISSUED_JWT_TOKEN" http://localhost:8000/

  oauth2/login/
• curl --cookie TEST_COOKIE_JAR.txt http://localhost:8000/api/..
  • Note: This call would be verified with the SessionAuthenticationAllowInactiveUser authentication class.

Note: It looks like the access_token endpoint (above) also returns session cookies along with the user's access_token.  However, this endpoint allows the client to refresh the session cookie without needing the user's credentials.

Expiration

Currently, the returned session cookie from this transaction expires in 2 weeks.

Note: the session cookie obtained on the edX website expires in 2 weeks if Remember-Me is selected, otherwise, it never expires.

OAuth2 Client Type, Client ID, and Client Secret

The OAuth2 RFC categorizes clients into 2 types based on their ability to confidentially store client (not user) credentials. 

Side note:
It is important to distinguish between client credentials and user credentials.
Client credentials authenticate the client (in this case, the mobile device or the mobile device type).  It could be used to identify securely the requesting server in a machine-to-machine interaction.
Whereas, user credentials authenticate the user (or the resource owner in OAuth-speak).  User credentials include the user's password, fingerprint, eye scan, and access and refresh tokens.

Now back to OAuth's 2 Client Types:

Confidential Client Type

...

Public Client Type

 Clients incapable of maintaining the confidentiality of their
 credentials (e.g., clients executing on the device used by the
 resource owner, such as an installed native application or a web
 browser-based application), and incapable of secure client
 authentication via any other means.

The RFC also explicitly calls out and defines native mobile applications, as follows:

...

• api/..

Expiration

Currently, our edX-issued JWT tokens expire in 1 hour.  These values can be overridden with the settings variables JWT_ACCESS_TOKEN_EXPIRE_SECONDS.

JWT -> Session Cookie

Additionally, the mobile app can exchange a JWT token for a session cookie that can be used in a WebView:

• LoginWithAccessTokenView: 1st party (open-edx) JWT token -> session cookie

  • Returns a 204 (no content), but with the user's session cookies in the response.

  • curl -X POST -H "Authorization: JWT INSERT_EDX_JWT_TOKEN" --cookie-jar TEST_COOKIE_JAR.txt http://localhost:8000/oauth2/login/

  • curl --cookie TEST_COOKIE_JAR.txt http://localhost:8000/api/..

    • Note: This call would be verified with the SessionAuthenticationAllowInactiveUser authentication class.

Note: It looks like the access_token endpoint (above) also returns session cookies along with the user's access_token.  However, this endpoint allows the client to refresh the session cookie without needing the user's credentials.

Expiration

Currently, the returned session cookie from this transaction expires in 2 weeks.

Note: the session cookie obtained on the edX website expires in 2 weeks if Remember-Me is selected, otherwise, it never expires.

OAuth2 Client Type, Client ID, and Client Secret

The OAuth2 RFC categorizes clients into 2 types based on their ability to confidentially store client (not user) credentials. 

Side note:
It is important to distinguish between client credentials and user credentials.
Client credentials authenticate the client (in this case, the mobile device or the mobile device type).  It could be used to identify securely the requesting server in a machine-to-machine interaction.
Whereas, user credentials authenticate the user (or the resource owner in OAuth-speak).  User credentials include the user's password, fingerprint, eye scan, and access and refresh tokens.

Now back to OAuth's 2 Client Types:

1. Confidential Client Type

   ^{Clients capable of maintaining the confidentiality of their credentials (e.g., client implemented on a secure server with restricted access to the client credentials), or capable of secure client authentication using other means.}
   
   

2. Public Client Type

   ^{Clients incapable of maintaining the confidentiality of their credentials (e.g., clients executing on the device used by the resource owner, such as an installed native application or a web browser-based application), and incapable of secure client authentication via any other means.}

The RFC also explicitly calls out and defines native mobile applications, as follows:

^{A native application is a public client installed and executed on the device used by the resource owner. Protocol data and credentials are accessible to the resource owner. It is assumed that any client authentication credentials included in the application can be extracted. On the other hand, dynamically issued credentials such as access tokens or refresh tokens can receive an acceptable level of protection. At a minimum, these credentials are protected from hostile servers with which the application may interact. On some platforms, these credentials might be protected from other applications residing on the same device.}

So what does this all mean for the mobile apps?

...

The mobile app is referred to by a Client ID, which is an authorization-server issued string identifier to identify (not authenticate) the client making the request.  On edX production servers, we use one common Client ID for all iOS clients and another common Client ID for all Android clients.  The Client ID is not intended to be a secret, but rather, a way to associate requests with a client (or a group of clients in this case).  In django, Client IDs are managed using the OAuth2 Client page on the django admin interface (/admin/oauth2_provider/clientapplication/).

Although a Client Secret is automatically generated for each OAuth2 Client in django, Client Secrets for mobile apps are not are NOT to be used.  And they definitely should not be distributed, transported to, configured or stored on the apps.  As the RFC states, we assume we cannot authenticate a mobile app since any client credential can be extracted by tampering with the local device.

Note: An authenticated client (and its Client Secret) is needed only for the Authorization Code and Client Credentials OAuth2 grant types.  The Implicit and Password grant types don't require an authenticated client.  The last (Password) grant type is what our mobile apps use.

OAuth2 Refresh Tokens

Since OAuth2 access tokens have a limited lifetime designated by their "expires_in" value, there has to be a strategy for what happens when they expire.  The OAuth2 RFC has provision for this by introducing Refresh Tokens as a means to re-authenticate with the authorization server in exchange for fresher access tokens.

Since possession of a refresh token authorizes requests for reusable access tokens, the refresh token is a proxy for the user's credentials and must be stored in a protected user-specific location (along with the user's access token).  From the OAuth2 and security standpoint, storing a refresh token is preferable over storing a user's raw credentials (passwords, etc). 

Django and Refresh Tokens

While the django library that we use for OAuth2, does support refresh tokens, it does not support Refresh Tokens for Public type Clients for the Password grant type (i.e., not for our mobile apps). 

Here are the relevant links in the django-oauth2-provider code-base:

• AccessToken.password
• AccessToken.access_token_response

However, the latest django-oauth-toolkit via the OAuthLib library (which we would like to migrate to) does support Refresh Tokens for Public type Clients for the ResourceOwnerPasswordCredentialsGrant.Tokens as a means to re-authenticate with the authorization server in exchange for fresher access tokens.

Once a JWT token expires, the Refresh Token can be used to create a new access token and acquire a new JWT token from it.

Since possession of a refresh token authorizes requests for reusable access tokens, the refresh token is a proxy for the user's credentials and must be stored in a protected user-specific location (along with the user's access token).  From the OAuth2 and security standpoint, storing a refresh token is preferable over storing a user's raw credentials (passwords, etc). 

OAuth2 on Refresh Tokens for Public Type Clients

The RFC does explicitly call out the support for refresh tokens for public Clients in Section 10.4.  Here is the relevant quote from the spec:

     Authorization servers MAY issue refresh tokens to web 
     application clients and native application clients.

However, it goes on to suggest that the server should detect unauthorized refresh token usage:

     When client authentication is not possible, the authorization server 
     SHOULD deploy other means to detect refresh token abuse.

     For example, the authorization server could employ refresh token
     rotation in which a new refresh token is issued with every access
     token refresh response.  The previous refresh token is invalidated
     but retained by the authorization server.  If a refresh token is
     compromised and subsequently used by both the attacker and the
  
  legitimate client, one of them will present an invalidated refresh
     token, which will inform the authorization server of the breach.

django-oauth-toolkit Implementation

The django-oauth-toolkit handles refresh tokens with multiple devices and access tokens correctly (not initially, but was eventually fixed).  In essence, it is implemented as a single-use Refresh Token per Access Token.  Whenever a new access token is requested with a refresh token, the presented refresh token is deleted and a new refresh token is returned along with the new access token.

Note: Old access tokens, however, remain in the database even if they have already expired.

Why not long-lived Access Tokens?

It is generally recommended to have short-lived access tokens and longer-lived refresh tokens.  But why?

The main difference between a refresh token and an access token is that the former is used in transactions solely with the Authorization server, while the latter is (obtained from the Authorization server but) presented to the Resource server.  The advantages of decoupling the tokens are: performance optimization, scalability, and revocation.

• Scalability - If access tokens are self-contained and implicitly verifiable by the Resource server, the Authorization Server would not be a single point of failure.
• Performance - Similarly, verifying an access token minimizes connections to the Authorization server and its database.
• Revocation - Compromised refresh tokens and refresh tokens of deactivated users can be centrally revoked, although there is a window of time during which all published access tokens may be in use until they expire.

Given the fact that our Authorization server (LMS) and Resource server (LMS) are one and the same (at this time), it is unclear whether having refresh tokens is a hard requirement for the edX mobile apps (at this time).  Furthermore, the current django implementation requires querying the database every time an access token is verified since they are randomly generated values and not self-contained signed values, so the performance argument doesn't hold water.  And since all tokens are kept in the database with foreign key relationships to the user, it would still be easy to centrally revoke all tokens associated with a user.

Note: Another point to note is that as the edX platform shifts from a monolith to a more distributed microservices architecture, we plan to use OAuth + JWTs and/or a variant of OpenIDConnect as our authentication framework.  With that, JWT tokens will also be present and most probably short-lived.

Proposal for Refreshing Tokens on edX mobile apps

Unfortunately, the edX apps have been released without support for refreshing OAuth tokens, although the OAuth access tokens expire only a month after they are issued.  So for now, we have been extending the expiration date regularly every month using a devOps executed script.

Assuming we still want to refresh edX Access Tokens (see Why not long-lived Access Tokens?), here is an implementation proposal.

Code

...

1. Insights
2. E-Commerce
3. Programs

...

1. send API request with access token
2. If access token is invalid, try to update it using refresh token
3. if refresh request passes, update the access token and re-send the initial API request
4. If refresh request fails, ask user to re-authenticate

...

legitimate client, one of them will present an invalidated refresh
     token, which will inform the authorization server of the breach.

django-oauth-toolkit Implementation

The django-oauth-toolkit handles refresh tokens with multiple devices and access tokens correctly (not initially, but was eventually fixed).  In essence, it is implemented as a single-use Refresh Token per Access Token.  Whenever a new access token is requested with a refresh token, the presented refresh token is deleted and a new refresh token is returned along with the new access token.

Note: Old access tokens, however, remain in the database even if they have already expired.

Why not long-lived Access Tokens?

It is generally recommended to have short-lived access tokens and longer-lived refresh tokens.  But why?

The main difference between a refresh token and an access token is that the former is used in transactions solely with the Authorization server, while the latter is (obtained from the Authorization server but) presented to the Resource server.  The advantages of decoupling the tokens are: performance optimization, scalability, and revocation.

• Scalability - If access tokens are self-contained and implicitly verifiable by the Resource server, the Authorization Server would not be a single point of failure.

• Performance - Similarly, verifying an access token minimizes connections to the Authorization server and its database.

• Revocation - Compromised refresh tokens and refresh tokens of deactivated users can be centrally revoked, although there is a window of time during which all published access tokens may be in use until they expire.

Given the fact that our Authorization server (LMS) and Resource server (LMS) are one and the same (at this time), it is unclear whether having refresh tokens is a hard requirement for the edX mobile apps (at this time).  Furthermore, the current django implementation requires querying the database every time an access token is verified since they are randomly generated values and not self-contained signed values, so the performance argument doesn't hold water.  And since all tokens are kept in the database with foreign key relationships to the user, it would still be easy to centrally revoke all tokens associated with a user.

Note: Another point to note is that as the edX platform shifts from a monolith to a more distributed microservices architecture, we plan to use OAuth + JWTs and/or a variant of OpenIDConnect as our authentication framework.  With that, JWT tokens will also be present and most probably short-lived.

Expiration Values

1. Set the default expiration for Access Tokens to 1 day (the accepted amount of time for a user to continue to use an unexpired token even after revocation).

2. Set the expiration time for Refresh Tokens to 2 weeks (analogous to our session cookies).
   Note: Since a new refresh token is issued at every use, its lifetime indicates for how long a user need not use the app without being asked to log in again.  As long as the user continues to use the app within the lifetime of the refresh token, they never need to log

...

Client Rollout Plan

...

1. in again.

Authentication Flow

The OAuth authentication and refresh flow for mobile in case of login or any API call.