Tuesday, December 3, 2013

Fine-grained Access Control for APIs

OAuth is the de facto standard for API security and it's all about access delegation.

The resource owner delegates a limited set of access rights to a third party. In OAuth terminology, this is the “scope”. A given access token has a scope associated with it and it governs the access token’s capabilities.

XACML (eXtensible Access Control Markup Language) is the de facto standard for fine-grained access control. OAuth scope can be represented in XACML policies.

Say, for example a user delegates access to his Facebook profile to a third party, under the scope “user_activities”. This provides access to the user's list of activities as the activities connection. To achieve fine-grained access control, this can be represented in a XACML policy.
<Policy>
     <Target>
          <Anyof>
                <AllOf>
                     <Match MatchId="urn:oasis:names:tc:xacml:1.0:function:string-equal">
                         <AttributeValue DataType="http://www.w3.org/2001/XMLSchema#string">
                                user_activities
                         <AttributeDesignator AttributeId="urn:oasis:names:tc:xacml:1.0:scope:scope-id" category="urn:oasis:names:tc:xacml:3.0:attribute-category:scope" DataType="http://www.w3.org/2001/XMLSchema#string" MustBePresent="false">
                      </Match>
                </AllOf>
         </AnyOf>
    </Target>
    <Rule Effect="Permit" RuleId="permit_rule">
    </Rule>
    <Rule Effect="Deny" RuleId="deny_rule">
    </Rule>
</Policy>
The above policy will be picked when the scope associated with the access token is equal to user_activities. Authorization Server first needs to find all the scopes associated with the given access token and build the XAML request accordingly. Authorization Server first gets the following introspection request.

token=gfgew789hkhjkew87
resource_id=GET https://graph.facebook.com/prabathsiriwardena/activities

Authorization Server now needs to find the scope and the client id associated with the given token and build the XACML request.
<Request>
      <Attributes Category="urn:oasis:names:tc:xacml:3.0:attribute-category:oauth-client">
           <Attribute AttributeId="urn:oasis:names:tc:xacml:1.0:client:client-id">
               <AttributeValue 
                     DataType="http://www.w3.org/2001/XMLSchema#string">32324343434</AttributeValue>
          </Attribute>
     <Attributes>
    <Attributes Category="urn:oasis:names:tc:xacml:3.0:attribute-category:action">
         <Attribute AttributeId="urn:oasis:names:tc:xacml:1.0:action:action-id">
             <AttributeValue
                     DataType="http://www.w3.org/2001/XMLSchema#string">GET</AttributeValue>
        </Attribute>
    </Attributes>
    <Attributes Category="urn:oasis:names:tc:xacml:3.0:attribute-category:scope">
       <Attribute AttributeId="urn:oasis:names:tc:xacml:1.0:scope:scope-id">
              <AttributeValue  
                      DataType="http://www.w3.org/2001/XMLSchema#string">user_activities</AttributeValue>
       </Attribute>
    </Attributes>
    <Attributes Category="urn:oasis:names:tc:xacml:3.0:attribute-category:resource">
        <Attribute AttributeId="urn:oasis:names:tc:xacml:1.0:resource:resource-id">
              <AttributeValue 
                      DataType="http://www.w3.org/2001/XMLSchema#string"> 
                                       https://graph.facebook.com/prabathsiriwardena/activities</AttributeValue>
        </Attribute>
    </Attributes>
</Request>
The above request will pick the policy we defined first and evaluate the rules. Each rule can define the criteria, whether to permit or deny.

  1. User / System accesses the API passing an access token.
  2. API Gateway intercepts the request - finds the access token and calls OAuth Authorization Server (Introspection endpoint) to validate it. 
  3. Authorization Server, finds the scopes and the client id associated with access token, builds a XACML request can call XACML PDP. 
  4. XACML PDP evaluates the XACML requests against its policy set and returns back a XACML response. 
  5.  OAuth Authorization Server sends back a Introspection Response which indicates the validity of the token. 
  6. API Gateway validates Introspection Response and then invokes the backend business API.

Monday, December 2, 2013

Enterprise API Access/ Security Patterns

Accessing an API secured with OAuth, on behalf of a user logged into the system, with SAML2 Web SSO.

  1. User from domain Foo tries to access a web app deployed in domain Bar. The web app is secured with SAML2 Web SSO. 
  2. Web App finds user does not have an authenticated session. Finds out from the which domain the request was initiated from and redirects the user to the SAML2 IdP in his own domain. 
  3. User authenticates to the SAML2 IdP in his own domain. 
  4. SAML2 IdP from domain Foo sends a SAML response back to the web app in domain Bar. 
  5. Web app validates the SAML response. It has to trust the domain Foo SAML2 IdP. To access backend APIs - on behalf of the logged in user, web app needs an OAuth access token. Web app talks to the OAuth Authorization Server in its own domain, passing the SAML token. 
  6. OAuth Authorization Server trusts the SAML2 IdP in domain Foo. Validates the SAML token and sends back an access token. 
  7. Web app invokes the API with the access token. 
  8. API Gateway intercepts the request - finds the access token and calls OAuth Authorization Server to validate it. 
  9. OAuth Authorization Server validates the token and sends back a JWT (JSON Web Token) which includes end user details to the API Manager. 
  10. API Gateway adds the JWT as an HTTP header and invokes the backend business API. 

Accessing an API secured with OAuth on behalf of a user/system authenticated to a SOAP service with WS-Trust.

  1. User / System from domain Foo authenticates to the WS-Trust STS in his own domain. 
  2. STS returns back a SAML token to access the SOAP service in domain Bar. 
  3. User/System authenticates to the SOAP service in domain Bar with the SAML token. 
  4. SOAP service validates the SAML token. It has to trust the domain Foo STS. To access backend APIs - on behalf of the logged in user, SOAP service needs an OAuth access token. SOAP service talks to the OAuth Authorization Server in its own domain, passing the SAML token.
  5. OAuth Authorization Server trusts the STS in domain Foo. Validates the SAML token and sends back an access token. 
  6. SOAP service invokes the API with the access token. 
  7. API Gateway intercepts the request - finds the access token and calls OAuth Authorization Server to validate it. 
  8.  OAuth Authorization Server validates the token and sends back a JWT (JSON Web Token) which includes end user details to the API Manager. 
  9. API Gateway adds the JWT as an HTTP header and invokes the backend business API. 
 Fine-grained access control with XACML.

  1. User / System accesses the API passing an access token.
  2. API Gateway intercepts the request - finds the access token and calls OAuth Authorization Server to validate it. 
  3. Authorization Server, finds the scopes and the client id associated with access token, builds a XACML request can call XACML PDP. 
  4. XACML PDP evaluates the XACML requests against its policy set and returns back a XACML response. 
  5.  OAuth Authorization Server sends back a JWT (JSON Web Token) which includes end user details to the API Manager. 
  6. API Gateway adds the JWT as an HTTP header and invokes the backend business API.

Wednesday, November 27, 2013

Building an ecosystem for API security around OAuth 2.0

Enterprise API adoption has gone beyond predictions. It has become the ‘coolest’ way of exposing business functionalities to the outside world. Both your public and private APIs, need to be protected, monitored and managed. Here we focus on API Security. There are so many options out there to make someone easily confused. When to select one over the other is always a question – and you need to deal with it quite carefully to identify and isolate the tradeoffs.

Security is not an afterthought. It has to be an integral part of any development project – so as for APIs. API security has evolved a lot in last five years. The growth of standards, out there, has been exponential. OAuth is the most widely adopted standard - and almost the de-facto standard for API security.

OAuth is a result of a community effort to build a common standard based solution for identity delegation. Its design was well fed with pre-OAuth vendor specific protocols like, Google AuthSub, Yahoo BBAuth and Flicker Auth.

The core concept behind OAuth is to generate a short-lived temporary token under the approval of the resource owner and share it with the client who wants to access the resource on behalf of its owner. This is well explained by Eran Hammer, taking a parking valet key as an analogy. Valet key will let a third party to drive, but with restrictions like, only a mile or two. Also you cannot use the valet key to do anything other than driving, like opening the trunk. Likewise the temporary token issued under OAuth can only be used for the purpose its been issued to - not for anything else. If you authorize a third party application to import photos from your Flickr account via OAuth, that application can use the OAuth key for that purpose only. It cannot delete or add new photos. This core concept remains the same from OAuth 1.0 to OAuth 2.0.

What made OAuth 2.0 looks different from OAuth 1.0?

OAuth 1.0 is a standard, built for identity delegation. OAuth 2.0 is a highly extensible authorization framework. The best selling point in OAuth 2.0 is its extensibility by being an authorization framework.

OAuth 1.0 is coupled with signature-based security. Although it has provisions to use different signature algorithms, still it’s signature based. One of the key criticisms against OAuth 1.0 is the burden enforced on OAuth clients for signature calculation and validation. This is not a completely valid argument. This is where we need proper tools to the rescue. Why an application developer needs to worry about signature handling? Delegate that to a third party library and stay calm. If you think OAuth 2.0 is better than OAuth 1.0 because of the simplicity added through OAuth 2.0 Bearer Token profile (against the signature based tokens in 1.0) – you’ve been misled.

Let me reiterate. The biggest advantage of OAuth 2.0 is its extensibility. The core OAuth 2.0 specification is not tightly coupled with a token type. There are several OAuth profiles been discussed under IETF OAuth working group at the moment. The Bearer token profile is already a proposed IETF standard - RFC 6750.

The Bearer token profile is the mostly used one today for API Security. The access token used under Bearer token profile is a randomly generated string. Anyone who is in possession of this token can use it to access a secured API. In fact, that is what the name implies too. The protection of this token is facilitated through the underlying transport channel via TLS. TLS only provides the security while in transit. It's the responsibility of the OAuth Token Issuer (or the Authorization Server) and the OAuth client to protect the access token while being stored. Most of the cases, access token needs to be encrypted. Also, the token issuer needs to guarantee the randomness of the generated access token - and it has to be long enough to exhaust any brute-force attacks.

OAuth 2.0 has three major phases (To be precise the phase - 1 and phase - 2 could overlap based on the grant type).

1. Requesting an Authorization Grant.
2. Exchanging the Authorization Grant for an Access Token.
3. Access the resources with the Access Token.

OAuth 2.0 core specification does not mandate any access token type. Also the requester or the client cannot decide which token type it needs. It's purely up to the Authorization Server to decide which token type to be returned in the Access Token response - which is the phase 2.

The access token type provides the client the information required to successfully utilize the access token to make a request to the protected resource (along with type-specific attributes). The client must not use an access token if it does not understand the token type.

Each access token type definition specifies the additional attributes (if any) sent to the client together with the "access_token" response parameter. It also defines the HTTP authentication method used to include the access token when making a request to the protected resource.

For example following is what you get for the Access Token response irrespective of which grant type you use (To be precise, if the grant type is client credentials, there won’t be any refresh_token in the response).

 HTTP/1.1 200 OK
  Content-Type: application/json;charset=UTF-8
  Cache-Control: no-store
  Pragma: no-cache

  {
    "access_token":"mF_9.B5f-4.1JqM",
    "token_type":"Bearer",
    "expires_in":3600,
    "refresh_token":"tGzv3JOkF0XG5Qx2TlKWIA"
  }

The above is for the Bearer and following is for the MAC.

HTTP/1.1 200 OK
  Content-Type: application/json
  Cache-Control: no-store

  {
    "access_token":"SlAV32hkKG",
    "token_type":"mac",
    "expires_in":3600,
    "refresh_token":"8xLOxBtZp8",
    "mac_key":"adijq39jdlaska9asud",
    "mac_algorithm":"hmac-sha-256"
  }

The MAC Token Profile is very much closer to what we have in OAuth 1.0.

OAuth authorization server will issue a MAC key along with the signature algorithm to be used and an access token that can be used as an identifier for the MAC key. Once the client has access to the MAC key, it can use it to sign a normalized string derived from the request to the resource server. Unlike in Bearer token, the MAC key will never be shared between the client and the resource server. It’s only known to the authorization server and the client. Once the resource server gets the signed message with MAC headers, it has to validate the signature by talking to the authorization server. Under the MAC token profile, TLS is only needed for the first step, during the initial handshake where the client gets the MAC key from the authorization server. Calls to the resource server need not to be on TLS, as we never expose MAC key over the wire.

MAC Access Token response has two additional attributes. mac_key and the mac_algorithm. Let me rephrase this - "Each access token type definition specifies the additional attributes (if any) sent to the client together with the "access_token" response parameter".

The MAC Token Profile defines the HTTP MAC access authentication scheme, providing a method for making authenticated HTTP requests with partial cryptographic verification of the request, covering the HTTP method, request URI, and host. In the above response access_token is the MAC key identifier. Unlike in Bearer, MAC token profile never passes it's top secret over the wire.

The access_token or the MAC key identifier is a string identifying the MAC key used to calculate the request MAC. The string is usually opaque to the client. The server typically assigns a specific scope and lifetime to each set of MAC credentials. The identifier may denote a unique value used to retrieve the authorization information (e.g. from a database), or self-contain the authorization information in a verifiable manner (i.e. a string consisting of some data and a signature).

The mac_key is a shared symmetric secret used as the MAC algorithm key. The server will not reissue a previously issued MAC key and MAC key identifier combination.

Phase-3 will utilize the access token obtained in phase-2 to access the protected resource.

Following shows how the Authorization HTTP header looks like when Bearer Token being used.

Authorization: Bearer mF_9.B5f-4.1JqM

This adds very low overhead on client side. It simply needs to pass the exact access_token it got from the Authorization Server in phase-2.

Under MAC token profile, this is how it looks like.

Authorization: MAC id="h480djs93hd8",
                    ts="1336363200",
                    nonce="dj83hs9s",
                    mac="bhCQXTVyfj5cmA9uKkPFx1zeOXM="

id is the MAC key identifier or the access_token from the phase-2.

ts the request timestamp. The value is  a positive integer set by the client when making each request to the number of seconds elapsed from a fixed point in time (e.g. January 1, 1970 00:00:00 GMT).  This value is unique across all requests with the same timestamp and MAC key identifier combination.

nonce is a unique string generated by the client. The value is unique across all requests with the same timestamp and MAC key identifier combination.

The client uses the MAC algorithm and the MAC key to calculate the request mac.

Either we use Bearer or MAC - the end user or the resource owner is identified using the access_token. Authorization, throttling, monitoring or any other quality of service operations can be carried out against the access_token irrespective of which token profile you use.

APIs are not just for internal employees. Customers and partners can access public APIs, where we do not maintain credentials internally. In that case we cannot directly authenticate them. So, we have to have a federated authentication setup for APIs, where we would trust a given partner domain, but not individuals. The SAML 2.0 Bearer Assertion Profile for OAuth 2.0 addresses this concern.

The SAML 2.0 Bearer Assertion Profile, which is built on top of OAuth 2.0 Assertion Profile, defines the use of a SAML 2.0 Bearer Assertion as a way of requesting an OAuth 2.0 access token as well as a way of authenticating the client. Under OAuth 2.0, the way of requesting an access token is known as a grant type. Apart from making the token type decoupled from the core specification it also makes grant type decoupled too. Grant type defines a protocol to get the authorized access token from the resource owner. The OAuth 2.0 core specification defines four grant types - authorization code, implicit, client credentials and resource owner password. But it does not limit to four. A grant type is another way of extending the OAuth 2.0 framework. OAuth 1.0 was coupled to a single grant type, which is almost similar to the authorization code grant type in 2.0.

SAML2 Bearer Assertion Profile defines its own grant type (urn:ietf:params:oauth:grant-type:saml2-bearer). Using this grant type a client can get either a MAC token or a Bearer token from the OAuth authorization server.

A good use case for SAML2 grant type is a SAML2 Single Sign On (SSO) scenario. A partner employee can login to a web application using SAML2 SSO (we have to trust the partner's SAML2 IdP) and later the web application needs to access a secured API on behalf of the logged in user. To do that the web application can use the SAML2 assertion already provided and exchange that to an OAuth access token via SAML2 grant type. There we need to have an OAuth Authorization Server running inside our domain - which trusts the external SAML2 IdP.

Unlike the four other grant types defined in OAuth 2.0 core specification, SAML2 grant type needs the resource owner to define the allowed scope for a given client out-of-band.

JSON Web Token (JWT) Bearer Profile is almost the same as the SAML2 Assertion Profile. Instead of SAML tokens, this uses JSON Web Tokens. JWT Bearer profile also introduces a new grant type (urn:ietf:params:oauth:grant-type:jwt-bearer).

This provision for extensibility made OAuth 2.0 very much superior to OAuth 1.0. That does not mean it’s perfect in all means.

To be the de facto standard for API security, OAuth 2.0 needs to operate in a highly distributed manner and still be interoperable. We need to have clear boundaries and well-defined interfaces in between the client, the authorization server and the resource server. OAuth 2.0 specification breaks it into two major flows. The first is the process of getting the access token from the authorization server - which is based on a grant type. The second is the process of using it in a request to the resource server. The way the resource server talks to the authorization server to validate the token is not addressed in the core specification. Hence has lead vendor specific APIs to creep in between the resource server and the authorization server. This kills interoperability. The resource server is coupled with the authorization server and this results in vendor lock-in.

The Internet draft OAuth Token Introspection which is been discussed under the IETF OAuth working group at the moment defines a method for a client or a protected resource (resource server) to query an OAuth authorization server to determine metadata about an OAuth token. The resource server needs to send the access token and the resource id (which is going to be accessed)- to the authorization server's introspection endpoint. Authorization server can check the validity of the token - evaluate any access control rules around it - and send back the response to the resource server. In addition to the token validity information, it will further return back the scopes, client_id and some other metadata associated with the token.

Apart from having a well-defined interface between the OAuth authorization server and the resource server, a given authorization server should also have the capability to issue tokens of different types. To do this, the client should bring the required token type it needs in the authorization request. But in the OAuth authorization request there is no token type defined. This limits the capability of the authorization server to handle multiple token types simultaneously or it will require a form of out-of-band mechanism to associate token types against clients.

Both the authorization server and the resource server should have the ability to expose their capabilities and requirements through a standard metadata endpoint.

The resource server should be able to expose its metadata by resource, which type of a token a given request expects, the required scope likewise. Also the requirements could change based on the token type. If it is a MAC token, then the resource server needs to declare which signature algorithm it expects. This could be possibly supported via an OAuth extension to the WADL (Web Application Description Language). Similarly, the authorization server also needs to expose its metadata. These could be, the supported token types, grant types likewise.

User-Managed Access (UMA) Profile of OAuth 2.0 introduces a standard endpoint to share metadata at the authorization server level. The authorization server can publish its token end point, supported token types and supported grant types via this UMA authorization server configuration data endpoint as a JSON document.

The UMA profile also mandates a set of UMA specific metadata to be published through this end point. This couples the authorization server to UMA, which also addresses a bigger problem than the need to discover authorization server metadata. It would be more ideal to introduce the need to publish/discover authorization server metadata through an independent OAuth profile and extend that in UMA to address more UMA specific requirements.

The problem addressed by UMA is far beyond, than just exposing Authorization Server metadata. UMA, undoubtedly going to be one of the key ingredients in any ecosystem for API security.

UMA defines how resource owners can control protected-resource access by clients operated by arbitrary requesting parties, where the resources reside on any number of resource servers, and where a centralized authorization server governs access based on resource owner policy. UMA defines two standard interfaces for the Authorization Server. One interface is between the Authorization Server and the Resource Server (protection API), while the other is between Authorization Server and Client (authorization API).

To initiate the UMA flow, the resource owner has to introduce all his resource servers to the centralized authorization server. With this, each resource server will get an access_token from the authorization server - and that can be used by resource servers to access the protection API exposed out by the authorization server.The API consists of an OAuth resource set registration endpoint as defined by OAuth Resource Registration draft specification, an endpoint for registering client-requested permissions, and an OAuth token introspection endpoint.

Client or the Requesting party can be unknown to the resource owner. When it tries to access a resource, the resource server will provide the necessary details - so, the requesting party can talk to the authorization server via Authorization API and get a Requesting Party Token (RPT). This API once again is OAuth protected - so, the requesting party should be known to the authorization server.

Once the client has the RPT - it can present it to the Resource Server and get access to the protected resource. Resource Server uses OAuth introspection endpoint of the Authorization Server to validate the token.

This is a highly distributed, decoupled setup - and further can be extended by incorporating SAML2 grant type.

Token revocation is also an important aspect in API security.

Most of the OAuth authorization servers currently utilize vendor specific APIs. This couples the resource owner to a proprietary API, leading to vendor lock-in. This aspect is not yet being addressed by the OAuth working group. The Token Revocation RFC 7009 addresses a different concern. This proposes an endpoint for OAuth authorization servers, which allows clients to notify the authorization server when a previously obtained refresh or access token is no longer needed.

In most of the cases token revocation by the resource owner will be more prominent than the token revocation by the client as proposed in this draft. The challenge in developing a profile to revoke access tokens / refresh tokens by the resource owner is the lack of token metadata at the resource owner end. The resource owner does not have the visibility to the access token. In that case the resource owner needs to talk to a standard end point at the authorization server to discover the clients it had authorized before. As per the OAuth 2.0 core specification a client is known to the authorization server via the client-id attribute. Passing this back to the resource owner is less meaningful as in most of the cases it’s an arbitrary string. This can be fixed by introducing a new attribute called “friendly-name”.

The model proposed in both OAuth 1.0 as well as in OAuth 2.0 is client initiated. Client is the one who starts the OAuth flow, by first requesting an access token. How about the other way around? Resource owner initiated OAuth delegation. Say for example I am a user of an online photo-sharing site. There can be multiple clients like Facebook applications, Twitter applications registered with it. Now I want to pick some client applications from the list and give them access to my photos under different scopes.  Let’s take another example; I am an employee of Foo.com. I'll be going on vacation for two weeks - now I want to delegate some of my access rights to Peter only for that period of time. Conceptually OAuth fits nicely here. But - this is a use case, which is initiated by the Resource Owner - which is not being addressed in the OAuth specification. This would require introducing a new resource owner initiated grant type. The Owner Authorization Grant Type Profile Internet draft for OAuth 2.0 addresses a similar concern by allowing the resource owner to directly authorize a relying party or a client to access a resource.

Delegated access control talks about performing actions on behalf of another user. This is what OAuth addresses. Delegated “chained” access control takes one step beyond this. The OASIS WS-Trust (a speciation built on top of WS-Security for SOAP) specification addressed this concern from its 1.4 version on wards, by introducing the “Act-As” attribute. The resource owner delegates access to the client and the client uses the authorized access token to invoke a service resides in the resource server. This is OAuth so far. In a real enterprise use case it’s a common requirement that the resource or the service, needs to access another service or a set of services to cater a given request. In this scenario the first service going to act as the client to the second service, and also it needs to act on behalf of the original resource owner. Using the access token passed to it as it is - is not the ideal solution. The Chain Grant Type Internet draft for OAuth 2.0 is an effort to fix this. It defines a method by which an OAuth protected service or a resource, can use a received OAuth token from its client, in turn, to act as a client and access another OAuth protected service. This specification still at its draft-1 would require maturing soon to address these concerns in real enterprise API security scenarios.

The beauty of the extensibility produced by OAuth 2.0 should never be underestimated by any of the above concerns or limitations. OAuth 2.0 is on the right track to become the de facto standard for API security to address enterprise scale security concerns.

Landscapes in Mobile Application Security

There are different aspects in Cloud and Mobile application security - and in different angles you can look in to it.

Within the first decade of the 21st century – internet worldwide increased from 350 million to more than 2 billion and Mobile phone subscribers from 750 million to 5 billion - and today it hits 6 billion mark - where the world population is around 7 billion. Most of the mobile devices out there - even the cheapest ones could be used to access the internet.

Let me do a quick survey here. How many of you have, password protected your laptops? Answer is obvious - almost all. But do you know that only 30% of mobile users, password protect their mobile devices ? This leaves out 4.2 billion mobile devices - unprotected - out there. I am using multi-factor authentication to secure my corporate email account on Google Apps. I am using world’s deadliest - the most complex password ever to protect my corporate Salesforce account. Now what ? I leave my mobile phone unprotected. I am already logged in to Google Apps - I am already logged in to Salesforce. Now I leave all my confidential information accessible to anyone having access to my mobile device.

How about password reset? Google, Microsoft, Yahoo - and almost all cloud service providers use mobile phone based password resets. Having temporary access to your mobile device, someone can take your accounts for the life time.

Multi-factor authentication for mobile applications is also not well thought yet. That is mostly because with the false assumption - “My mobile is under my control always”. 113 cell phones are lost or stolen every minute in the U.S and $7 million worth of smartphones are lost daily. Both Google step-2 authentication and Facebook Code Generator always rely on a mobile phone for protecting web based access. But, none of their mobile applications are protected with multi-factor authentication. Phone based multi-factor authentication won’t work for mobile applications.

Why do we need to worry about all these at the corporate level? 62% of mobile workers currently use their personal smartphones for work.

These are well known facts or threats in the mobile world. Different vendors have their own solutions. Apple let you lock your device over the Internet or even wipe off  all it’s data. And again most of the mobile device management (MDM) solutions let you take control over your lost device. But, then again - how much time you leave for it to be on wrong hands will do enough damage. MDM solutions out there need to go beyond it’s simple definition to be an integral part of the corporate Identity Management system.

Over the last few years - almost - all the cloud service providers are becoming mobile friendly. All of them have provided RESTful JSON based APIs. Amazon AWS, Google Cloud Storage, Salesforce, Dropbox all of them have REST APIs . Except AWS - all the others are secured with OAuth 2.0. AWS uses its own authentication scheme.

OAuth 2.0 has ‘proven success’ in securing REST APIs. But for mobile applications OAuth 2.0 can be miserably misleading. It has four defined grant types. Authorization Code, Implicit, Resource Owner Password and Client Credentials.

Authorization code and the implicit are mostly being used for browser based applications. Its a mis-belief among application developers, that the Resource Owner and the Client Credentials grant types are for mobile applications. Those require you to provide your credentials to the application - directly. As a practice avoid it. If you develop a mobile application to access a secured cloud API using OAuth - use either the authorization code or the implicit grant type. There, your application needs to pop-up the native browser to redirect the user to the OAuth authorization server.

But - still that does not make you 100% not vulnerable to further attacks. Whenever there is a redirection through the browser - there is a possibility of a phishing attack. A rogue OAuth client application can have a “Login with Facebook” button - which will redirect you to a rogue OAuth authorization server - which looks like Facebook - where you will misinterpret it as Facebook and give away the credentials.

There are many countermeasures that can be taken against phishing. But, sadly - most of the OAuth authorization servers, including Facebook and Twitter do not follow any. Your Facebook or Twitter account credentials can be quite easily phished through your mobile phone - than from a laptop computer. It’s quite easier than you think.

If you have developed mobile applications with OAuth 2.0 - you might have encountered another limitation. You need to bake in - your client key and the client secret into the mobile application itself. This is required in the first phase of the OAuth flow - and it’s the identification of the client to the OAuth authorization server. What would happen if someone steals this from the device ?

Let’s have a look at an example. We have a mobile application which will access the Facebook friend list of an end user and stores that friend list in Google Cloud Storage using its REST API. Facebook friend list belongs to the end-user - but the Google Cloud Storage belongs to the mobile application. Mobile application has to register with Facebook as an OAuth client and gets a client key and a secret. Then using Authorization Code grant type - it can get an access token to - access the end-users friend list on-behalf of him. To store this in Google Cloud Storage, the application has to use client credentials grant type - where the authorization server is Google. To get this done - we need to bake-in the client key and the client secret into the application itself. Anyone getting access to these keys, will get access to the Google Cloud Storage too. This is an area still under research with no permanent solution yet. Solutions over there like - restrictions based on IP addresses, device Ids - will only make things bit harder - but, not fully impossible.

OAuth 2.0 has become the de facto standard for mobile application authentication. This, it self has given the applications a better failover capability in case of an attack. The recent attack against Buffer - a social media management service, which lets users cross-post in to social networking sites like Facebook and Twitter - is a very good example. Twitter, Facebook got flooded with posts from Buffer. But revoking the client key of Buffer sorted out the issue. And also attack on Buffer did not give the full control of users’ Facebook and Twitter accounts to attackers - as it was not storing passwords.

It takes an average of 20 seconds for a user to log into a resource. Not having to enter a password each time a user needs to access a resource saves time and makes users more productive and also reduces the frustration of multiple log-on events and forgotten passwords. Users only have one password to remember and update, and only one set of password rules to remember. Their initial login provides them with access to all resources, typically for the entire day or the week.

What are the challenges in building a single sign-on solution for mobile applications ?

If you provide multiple mobile applications for your corporate employees, to be installed in their mobile devices - its a pain to ask them to relogin to each application separately. Possibly all of them may be sharing the same user store. This is analogous to a case where Facebook users login into multiple third party mobile applications with their Facebook credentials.

In mobile world - this can be done in two ways.

First, each native mobile application - when it is needed to authenticate a user  - should popup the native browser - and start the OAuth flow. Your company should have a centralized OAuth authorization server, running on top of the corporate user store. All your mobile applications will redirect the user to the same Authorization Server - creating a single login session under the domain of the centralized authorization server - which indirectly facilitates single-sign on.






The other approach is known as “Native SSO”. The user experience in Native SSO is very much better
than the browser based SSO. Here you need to have a native mobile application developed for the corporate identity provider (IdP) - or the authorization server - which will be invoked by the other applications to initiate the OAuth flow - instead of popping up the browser. Although Native SSO provides better - improved user experience - it also makes phishing attacks much easier.





The other drawback in Native SSO is - it has a phase - which is not standard based. Your application should know beforehand who your Identity Provider is - and should program according to it’s interface to initiate Native SSO. Currently there is an attempt by OpenID Foundation to  build a standard Single Sign On (SSO) model for native applications installed on mobile devices. This introduces an OpenID Connect Client called an Authorization Agent - which  can obtain tokens on behalf of other installed native applications - thereby provisioning tokens to those applications and so enabling a Single Sign On experience for end users. The spec is at its very initial stage - and would require many more iterations before becoming a standard.

Lets take another example. Previous case we had an assumption that we only have a single user store - which is behind the centralized authorization server. Let’s take that assumption out. We need users outside our domain - say, from federated partners - to access our mobile applications and consume services. There needs to be a bootstrap process to establish trust between those federated partners and our authorization server. Doing this in a standard manner - the partners would require to support one of the federation standards out there. The best would be the SAML. So - we need to add partner SAML IdPs as trusted IdPs to our authorization server. And also we can define an authorization policy against each IdP - so that we know which rights they would have in our authorization server. When user being redirected to the authorization server - either through browser based or native SSO - he can pick against which IdP he wants authenticate. Based on the choice the user will be redirected to his home SAML IdP - and once authenticated - authorization server will resume the OAuth flow. This is a one time thing - and for other subsequent requests from other mobile applications - flow will be seamless to the user and would not required to being redirected to his home SAML IdP.

With all the Single Sign On use cases we discussed above - we still left with one more assumption - all the mobile applications will have a centralized authorization server. Let’s get rid of  that too.

One key requirement for any single sign on scenario is - we should be able to establish direct trust or brokered trust between applications and their users. In most of the cases this is established through IdPs. The first example we took was based on direct trust - while the second is on brokered trust. To accomplish this use case we need to build a trust relationship between all the authorization servers - taking part in - and also a middle man to mediate SSO. This use case is also highlighted in the Native SSO draft specification by OpenID Foundation - but no much details as of now.

Data in transit is another security concern. Forget about NSA and Angela Merkel. NSA has more than 5000 highly capable computer scientists - and they have control over security algorithm designs. So, lets take NSA out of the picture. In most of the scenarios mobile applications depend on TLS for data confidentiality, in transit. TLS has it’s own limitations as its based on transport and the confidentiality of the data terminates as it leaves the transport. Most of the data transport channels used within mobile applications use REST and JSON. We have JOSE working group under IETF,  working currently to produce a standard for doing message level encryption and signing for JSON payloads.

Let me take this discussion to another direction. Managed Cloud APIs. Amazon AWS, Google Cloud, Dropbox, Salesforce all expose APIs over REST and JSON. Even in this case, Twitter and Facebook. We develop mobile applications on top of these cloud APIs - to be used by our corporate employees. For the simplicity of explaining - I’ll take Twitter as the example. We have a corporate account with Twitter - which is used to tweet events related to the company - and mostly used by the marketing team. To tweet through the corporate account we need to share the official twitter password with them. Which is not ideal. Can’t we let them tweet through the same corporate account - but still authenticate with their corporate LDAP credentials ? And also - we need to enforce certain rules and policies. Any tweet mentioning client names should be blocked immediately. Also we need to collect statistics and do access controlling. In other words - to cater for all these requirements we need to turn the simple Twitter API - in to a managed API. Here we introduce an API Gateway - in between your mobile application and the Twitter API. Through API gateway we expose - our own API - which wraps the Twitter API. Now - the marketing team can authenticate to the Twitter Wrapper API using their corporate credentials and Tweet using the corporate Twitter account. The official Twitter credentials are never exposed - and kept just within the API Gateway. Twitter is a simple example - but the same applies to any cloud API - which you want to turn into a Managed Cloud API - to be consumed by your mobile applications.

Sunday, February 17, 2013

The Twitter API Management Model

The objective of this blog post is to explore in detail the patterns and practices Twitter has used in it's API management.

Twitter comes with a comprehensive set of REST APIs to let client apps talk to Twitter.

Let's take few examples...

If you use following with cUrl - it returns the 20 most recent statuses, including retweets if they exist, from non-protected users. The public timeline is cached for 60 seconds. Requesting more frequently than that will not return any more data, and will count against your rate limit usage.

curl https://api.twitter.com/1/statuses/public_timeline.json

The example above is an open API - which requires no authentication from the client who accesses it. But keep in mind... it has a throttling policy associated with the API. That is the rate limit. For example the throttling policy associated with the ..statuses/public_timeline.json API could say, only allow maximum 20 API calls from the same IP address.. like wise.. so.. this policy is a global policy for this API.

1. Twitter has open APIs - where anonymous users can access.
2. Twitter has globally defined policies per API.

Let's take another sample API - statuses/retweeted_by_user - returns the 20 most recent retweets posted by the specified user - given that the user's timeline is not protected. This is also another open API.

But, what if I want to post to my twitter account? I could use the API statuses/update. This updates the authenticating user's status and this is not an open API. Only the authenticated users can access this.

How do we authenticate our selves to access the Twitter API?

Twitter supported two methods. One way is to use BasicAuth over HTTPS and the other way is OAuth 1.0a.

BasicAuth support was removed recently and now the only remaining way is with OAuth 1.0a. As of this writing Twitter doesn't support OAuth 2.0.

Why I need to use the APIs exposed by Twitter - is that I have some external applications that do want to talk to Twitter and these applications use the Twitter APIs for communication.

If I am the application developer - following are the steps I need to follow to build my application to access protected APIs from Twitter.

First the application developer needs to login to Twitter and creates an Application.

Here, the Application is an abstraction for a set of protected APIs Twitter exposes outside.

Each Application you create, needs to define the level of access it needs to those underling APIs. There are three values to pick from.

- Read only
- Read and Write
- Read, Write and Access direct messages

Let's see what these values mean...

If you pick 'Read only' - that means a user who is going to use your Application needs to give it the permission to read. In other words - the user will be giving access to invoke the APIs defined here which starts with GET, against his Twitter account. The only exception is Direct Messages APIs - with Read only your Application won't have access to a given user's Direct Messages - even GETs.

Even you who develop the application - the above is valid for you too as well. If you want to give your application, access your Twitter account - there also you should give the application the required rights.

If you pick Read and Write - that means a user who is going to use your application needs to give it the permission to read and write. In other words - the user will be giving access to invoke the APIs defined here which starts with GET or POST, against his Twitter account. The only exception is Direct Messages APIs - with Read and Write, your application won't have access to a given user's Direct Messages - even GETs or POSTs.

3. Twitter has an Application concept that groups APIs together.
4. Each API declares the actions those do support. GET or POST
5. Each Application has a required access level for it to function[Read only, Read and Write, Read Write and Direct Messages]

Now lets dig in to the run-time aspects of this. I am going to skip OAuth related details here purposely for clarity.

For our Application to access the Twitter APIs - it needs a key. Let's name it as API_KEY [if you know OAuth - this is equivalent to the access_token]. Say I want to use this Application. First I need to go to Twitter and need to generate an API_KEY to access this Application. Although there are multiple APIs wrapped in the Application - I only need a single API_KEY.

6. API_KEY is per user per Application[a collection of APIs].

When I generate the API_KEY - I can specify what level of access I am going to give to that API_KEY - Read Only, Read & Write or Read, Write & Direct Message. Based on the access level I can generate my API_KEY.

7. API_KEY carries permissions to the underlying APIs.

Now I give my API_KEY to the Application. Say it tries to POST to my Twitter time-line. That request also should include the API_KEY. Now once the Twitter gets the request - looking at the API_KEY it will identify the Application is trying to POST to 'my' time-line. Also it will check whether the API_KEY has Read & Write permissions - if so it will let the Application post to my Twitter time-line.

If the Application tries to read my Direct Messages using the same API_KEY I gave it - then Twitter will detect that the API_KEY doesn't have Read, Write & Direct Message permission and the request will fail.

Even in the above case, if the Application tries to post to the Application Developer's Twitter account - there also it needs an API_KEY - from the Application Developer, which he can get from Twitter.

Once user grants access to an Application by it's API_KEY - during the entire life time of the key, the application can access his account. But, if the user wants to revoke the key, Twitter provides a way to do that as well. Basically when you go here, it displays all the Applications you have given permissions to access your Twitter account - if you want, you can revoke access from there.

8. Twitter lets users revoke API_KEYs

Also another interesting thing is how Twitter does API versioning. If you carefully look at the URLs, you will notice that the version number is included in the URL it self - https://api.twitter.com/1/statuses/public_timeline.json. But it does not let Application developers to pick, which versions of the APIs they want to use.

9. Twitter tracks API versions in runtime.
10.Twitter does not let Application developers to pick API versions at the time he designs it.

Twitter also has a way of monitoring the status of the API. Following shows a screenshot of it.

11. Twitter does API monitoring in runtime.

Why OAuth it self is not an authentication framework ?

Let's straight a way start with definitions to avoid any confusions.

Authentication is the act of confirming the truth of an attribute of a datum or entity. If I say, I am Prabath - I need to prove that. I can prove that with something I know, something I have or with something I am. Once I proved my self who I claim I am, then the system can trust me. Sometimes systems do not just want to identify me by just by name. By name could help to identify me uniquely - but how about other attributes of me. Before you get through the boarder control - you need to identify your self - by name - by picture - and also by finger prints and eye retina. Those are validated real-time against the data from the VISA office which issued the VISA for you. That check will make sure its the same person who claimed to have the VISA enters in to the country. That is proving your identity. Proving your identity is authentication.

Authorization is about what you can do. Your capabilities. You could prove your identity at the boarder control by name - by picture - and also by finger prints and eye retina - but it's your VISA that decides what you can do. To enter in to the country you need to have a valid VISA that has not expired. A valid VISA is not a part of your identity - but a part of what you can do. Also what you can do inside the country depends on the VISA type. What you do with a B1 or B2 differs from what you can do with an L1 or L2. That is authorization.

OAuth 2.0 is about authorization. Not about authentication.

OAuth 2.0 is about what you can do ( or on behalf of another user) - not about who you are. So - you cannot use OAuth 2.0 it self for authentication.

I hear what you say. We use OAuth 2.0 based Facebook login to log in to plenty of different web applications. So am I lying ?

Let's pick Facebook it self for an example. I got following image from Facebook login developer documentation. It highlights the OAuth flow with authorization code grant type.

User goes to a Web Application. Clicks on the Login button. Gets redirected to the Facebook for authentication and pass the "code" to the client Web Application. Now the Web Application exchanges the code to an access token. Facebook will only issue an access token for a valid code. And - it also will only issue a code after the user been authenticated. So, the Web Application having access to a valid access token means - and it only means - that particular user is from Facebook. That's it. It's like carrying out a valid, stamped VISA to the boarder control with out having any identification information about the user on it. Looking at the access token - the application cannot say who the user is - it can only say where the user comes from. So it does not help the Web Application to identify the end user uniquely. When the same user logs in next time he could bring a different access token - so the Web Application cannot use the access token as a way of identification. That is why OAuth it self is not an authentication framework.




















Still you think I am lying? Of course you do.

When you login via Facebook - your application also gets certain claims about you. Like your first name, last name and email address - and also your Facebook Id. How does this happen if OAuth does not bring any identity attributes ?

Once the Web Application gets the access token from the Facebook, to get user attributes - the Web Application can call following API with the access token it got.

https://graph.facebook.com/me?access_token=[access_token]

This will get the identity information of the user who authorized the access token.

Now - this helps to identify the user - or to authenticate the user. But, keep in mind that the last step is out side the scope of OAuth and that functionality is provided via Facebook's Graph API.

Not just through Facebook's Graph API - even through SCIM we can identify the user. But with the current specification there seems to be a limitation in doing that in a standard manner.

With the current specification to get the details of a specific user we use the following.

 GET /Users/2819c223-7f76-453a-919d-413861904646
 Host: example.com
 Accept: application/json Authorization: Bearer h480djs93hd8

But, just after the OAuth flow - we only have the access token at the client side. No concrete information about the end user. So we cannot use the above.

By having a pre-defined userid say "me" [just like in Facebook Graph API] - cab be made to provide back the logged in user's info.

 GET /Users/me
 Host: example.com
 Accept: application/json
 Authorization: Bearer h480djs93hd8

In the above case - the information about the user corresponding to the provided Bearer token will be returned back. So, based on that, client can identify the end user with it's attributes.

Apart from the above two approaches - OpenID Connect is the best one built on top of OAuth 2.0 to identify the end user.

With OpenID Connect in addition to the OAuth access token - the client or the Web Application also gets an ID token. The ID Token is a security token that contains claims/identity information about the authentication event and other requested claims from the client. The ID Token is represented as a JWT - and it can be used to prove the identity of the end user.

SAML 2.0 Bearer Assertion Profile for OAuth 2.0

SAML 2.0 Bearer Assertion Profile which is built on top of OAuth 2.0 Assertion Profile defines the use of a SAML 2.0 Bearer Assertion as a means for requesting an OAuth 2.0 access token as well as for use as a means of client authentication.

The OAuth 2.0 Assertion Profile is an abstract extension to OAuth 2.0 that provides a general framework for the use of Assertions as client credentials and/or authorization grants with OAuth 2.0.

This specification profiles the OAuth 2.0 Assertion Profile...

1.  To define an extension grant type that uses a SAML 2.0 Bearer Assertion to request an OAuth 2.0 access token when a client wishes to utilize an existing trust relationship, expressed through the semantics of (and digital signature calculated over) the SAML Assertion, without a direct user approval step at the authorization server.

2.  To define how a SAML Assertion can be used as a client authentication mechanism. The use of an Assertion for client authentication is orthogonal to and separable from using an Assertion as an authorization grant. They can be used either in combination or separately.

This specification further defies the format and processing rules for the SAML Assertion are intentionally similar, though not identical, to those in the Web Browser SSO Profile defined in SAML Profiles.

Now let's have a look at an end to end scenario which explains how to use SAML 2.0 Bearer Assertion as an grant type.


1. User been redirected to the SAML2 IdP and user authenticates to the SAML2 IdP.

2. SAML2 IdP issues a SAML2 Token with following characteristics and Client Web App verifies the SAML2 token and authenticates the user.
  • The Assertion's Issuer element will contain a unique identifier for the entity that issued the Assertion - which is the SAML2 IdP in our case.
  • The Assertion will contain Conditions element with an AudienceRestriction element with an Audience element containing a URI reference that identifies the Authorization Server.
  •  The Assertion will contain Conditions element with an AudienceRestriction element with an Audience element containing a URI reference that identifies the Client Web App, which is the SAML2 Service Provider and Client Web App must verify element.
  •  The Assertion MUST contain a Subject element. The subject MAY identify the resource owner for whom the access token is being requested. When using an Assertion as an authorization grant, the Subject SHOULD identify an authorized accessor for whom the access token is being requested (typically the resource owner, or an authorized delegate).
  • The Assertion MUST have an expiry that limits the time window during which it can be used. The expiry can be expressed either as the NotOnOrAfter attribute of the Conditions element or as the NotOnOrAfter attribute of a suitable SubjectConfirmationData element.
  •  The element must contain at least on element that allows the authorization serve to confirm it as a Bearer Assertion.  Such a element MUST have a Method attribute with a value of "urn:oasis:names:tc:SAML:2.0:cm:bearer".
  • The SubjectConfirmation element MUST contain a SubjectConfirmationData element, unless the Assertion has a suitable NotOnOrAfter attribute on the Conditions element, in which case the SubjectConfirmationData element MAY be omitted. When present, the SubjectConfirmationData element MUST have a Recipient attribute with a value indicating the token endpoint URL of the authorization server (or an acceptable alias). The  SubjectConfirmationData element MUST have a NotOnOrAfter attribute that limits the window during which the Assertion can be confirmed.  The SubjectConfirmationData element MAY also contain an Address attribute limiting the client address from which the Assertion can be delivered.  Verification of the Address is at the discretion of the authorization server.
  • The Assertion MUST have an expiry that limits the time window during which it can be used. The expiry can be expressed either as the NotOnOrAfter attribute of the Conditions element or as the NotOnOrAfter attribute of a suitable SubjectConfirmationData element.
  •  If the Assertion issuer authenticated the subject, the Assertion SHOULD contain a single  AuthnStatement representing that authentication event.
  •  If the Assertion was issued with the intention that the presenter act autonomously on behalf of the subject, an AuthnStatement SHOULD NOT be included.  The presenter SHOULD be identified in the NameID or similar element, the element, or by other available means.
  • Other statements, in particular elements, MAY be included in the Assertion.
3. Now Client Web App wants to access a RESTful service which secured with OAuth 2.0 and it has to get an access token on behalf the logged in user first. Now using the SAML2 grant type, Client Web App Requests an access token from the OAuth 2.0 Authorization Server.

     POST /token.oauth2 HTTP/1.1
     Host: as.example.com
     Content-Type: application/x-www-form-urlencoded

     grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Asaml2-bearer&
     assertion=PHNhbWxwOl...[omitted for brevity]...ZT4


4. Authorization Server validates the SAML token assertion and issues an access token back. Authorization Server does the SAML token verification as mentioned below.
  • The Assertion will contain Conditions element with an AudienceRestriction element with an Audience element containing a URI reference that identifies the Authorization Server and Authorization Server must verify the element.
  •  The SubjectConfirmation element MUST contain a element, unless the Assertion has a suitable NotOnOrAfter attribute on the Conditions element, in which case the SubjectConfirmationData element MAY be omitted. When present, the  SubjectConfirmationData element MUST have a Recipient attribute with a value indicating the token endpoint URL of the authorization server (or an acceptable alias).  The authorization server MUST verify that the value of the Recipient attribute matches the token endpoint URL (or an acceptable alias) to which the Assertion was delivered.  The SubjectConfirmationData element MUST have a NotOnOrAfter attribute that limits the window during which the Assertion can be confirmed.  The SubjectConfirmationData element MAY also contain an Address attribute limiting the client address from which the Assertion can be delivered.  Verification of the Address is at the discretion of the authorization server.
  • The authorization server MUST verify that the NotOnOrAfter instant has not passed, subject to allowable clock skew between systems. An invalid NotOnOrAfter instant on the Conditions element invalidates the entire Assertion.  An invalid NotOnOrAfter instant on a SubjectConfirmationData element only invalidates the individual SubjectConfirmation .  
  • The authorization server MAY reject Assertions with a NotOnOrAfter instant that is unreasonably far in the future.  The authorization server MAY ensure that Bearer Assertions are not replayed, by maintaining the set of used ID values for the length of time for which the Assertion would be considered valid based on the applicable NotOnOrAfter instant.
 5. Now Client Web App can use the access token to access the secured service.

OAuth 2.0 Bearer Token Profile Vs MAC Token Profile

Almost all the implementation I see today are based on OAuth 2.0 Bearer Token Profile. Of course its an RFC proposed standard today.

OAuth 2.0 Bearer Token profile brings a simplified scheme for authentication. This specification describes how to use bearer tokens in HTTP requests to access OAuth 2.0 protected resources. Any party in possession of a bearer token (a "bearer") can use it to get access to the associated resources (without demonstrating possession of a cryptographic key). To prevent misuse, bearer tokens need to be protected from disclosure in storage and in transport.

Before dig in to the OAuth 2.0 MAC profile lets have quick high-level overview of OAuth 2.0 message flow.

OAuth 2.0 has mainly three phases.

1. Requesting an Authorization Grant.
2. Exchanging the Authorization Grant for an Access Token.
3. Access the resources with the Access Token.

Where does the token type come in to action ? OAuth 2.0 core specification does not mandate any token type. At the same time at any point token requester - client - cannot decide which token type it needs. It's purely up to the Authorization Server to decide which token type to be returned in the Access Token response. So, the token type comes in to action in phase-2 when Authorization Server returning back the OAuth 2.0 Access Token.

The access token type provides the client with the information required to successfully utilize the access token to make a protected resource request (along with type-specific attributes). The client must not use an access token if it does not understand the token type.

Each access token type definition specifies the additional attributes (if any) sent to the client together with the "access_token" response parameter. It also defines the HTTP authentication method used to include the access token when making a protected resource request.

For example following is what you get for Access Token response irrespective of which grant type you use.

 HTTP/1.1 200 OK
     Content-Type: application/json;charset=UTF-8
     Cache-Control: no-store
     Pragma: no-cache

     {
       "access_token":"mF_9.B5f-4.1JqM",
       "token_type":"Bearer",
       "expires_in":3600,
       "refresh_token":"tGzv3JOkF0XG5Qx2TlKWIA"
      }

The above is for Bearer - following is for MAC.

HTTP/1.1 200 OK
     Content-Type: application/json
     Cache-Control: no-store

     {
       "access_token":"SlAV32hkKG",
       "token_type":"mac",
       "expires_in":3600,
       "refresh_token":"8xLOxBtZp8",
       "mac_key":"adijq39jdlaska9asud",
       "mac_algorithm":"hmac-sha-256"
     }

Here you can see MAC Access Token response has two additional attributes. mac_key and the mac_algorithm. Let me rephrase this - "Each access token type definition specifies the additional attributes (if any) sent to the client together with the "access_token" response parameter".

This MAC Token Profile defines the HTTP MAC access authentication scheme, providing a method for making authenticated HTTP requests with partial cryptographic verification of the request, covering the HTTP method, request URI, and host. In the above response access_token is the MAC key identifier. Unlike in Bearer, MAC token profile never passes it's top secret over the wire.

The access_token or the MAC key identifier is a string identifying the MAC key used to calculate the request MAC. The string is usually opaque to the client. The server typically assigns a specific scope and lifetime to each set of MAC credentials. The identifier may denote a unique value used to retrieve the authorization information (e.g. from a database), or self-contain the authorization information in a verifiable manner (i.e. a string consisting of some data and a signature).

The mac_key is a shared symmetric secret used as the MAC algorithm key. The server will not reissue a previously issued MAC key and MAC key identifier combination.

Now let's see what happens in phase-3.

Following shows how the Authorization HTTP header looks like when Bearer Token been used.

Authorization: Bearer mF_9.B5f-4.1JqM

This adds very low overhead on client side. It simply needs to pass the exact access_token it got from the Authorization Server in phase-2.

Under MAC token profile, this is how it looks like.

Authorization: MAC id="h480djs93hd8",
                        ts="1336363200",
                        nonce="dj83hs9s",
                        mac="bhCQXTVyfj5cmA9uKkPFx1zeOXM="

This needs bit more attention.

id is the MAC key identifier or the access_token from the phase-2.

ts the request timestamp. The value is  a positive integer set by the client when making each request to the number of seconds elapsed from a fixed point in time (e.g. January 1, 1970 00:00:00 GMT).  This value is unique across all requests with the same timestamp and MAC key identifier combination.

nonce is a unique string generated by the client. The value is unique across all requests with the same timestamp and MAC key identifier combination.

The client uses the MAC algorithm and the MAC key to calculate the request mac.

This is how you derive the normalized string to generate the HMAC.

The normalized request string is a consistent, reproducible concatenation of several of the HTTP request elements into a single string.

By normalizing the request into a reproducible string, the client and server can both calculate the request MAC over the exact same value.

The string is constructed by concatenating together, in order, the following HTTP request elements, each followed by a new line character (%x0A):

1. The timestamp value calculated for the request.

2. The nonce value generated for the request.

3. The HTTP request method in upper case. For example: "HEAD", "GET", "POST", etc.

4. The HTTP request-URI as defined by [RFC2616] section 5.1.2.

5. The hostname included in the HTTP request using the "Host" request header field in lower case.

6. The port as included in the HTTP request using the "Host" request header field. If the header field does not include a port, the default value for the scheme MUST be used (e.g. 80 for HTTP and 443 for HTTPS).

7. The value of the "ext" "Authorization" request header field attribute if one was included in the request (this is optional), otherwise, an empty string.

Each element is followed by a new line character (%x0A) including the last element and even when an element value is an empty string.

Either you use Bearer of MAC - the end user or the resource owner is identified using the access_token. Authorization, throttling, monitoring or any other quality of service operations can be carried out against the access_token irrespective of which token profile you use. 

OAuth 1.0a Bounded Tokens

We had a very interesting discussion today on the $subject.

In fact, one of my colleagues came up with a question - "Why the hell OAuth 1.0 uses two keys to sign messages..? Why Not just the consumer secret ?"

Well.. In fact - the exact reverse of this argument is one key point Eran Hammer highlights in his famous blog post - to emphasize why OAuth 1.0a is better...

"Unbounded tokens - In 1.0, the client has to present two sets of credentials on each protected resource request, the token credentials and the client credentials. In 2.0, the client credentials are no longer used. This means that tokens are no longer bound to any particular client type or instance. This has introduced limits on the usefulness of access tokens as a form of authentication and increased the likelihood of security issues."

Before I answer the question - let me very briefly explain OAuth 1.0a flow..

1. To become an OAuth consumer you need to have a Consumer Key and a Consumer Secret. You can obtain these from the OAuth Service Provider. These two parameters can be stored in a database or in the filesystem.

Consumer Key: A value used by the Consumer to identify itself to the Service Provider.
Consumer Secret: A secret used by the Consumer to establish ownership of the Consumer Key.


2. Using the Consumer Key and Consumer Secret, you need to talk to the OAuth Service Provider and get a Unauthorized Request Token.

Request to get the Unauthorized Request Token will include following parameters.
  • oauth_consumer_key
  • oauth_signature_method
  • oauth_signature
  • oauth_timestamp
  • oauth_nonce
  • oauth_version
  • oauth_callback
Here the certain parameters of the request will be signed by the Consumer Secret. Signature validation at the OAuth Service Provider end will confirm the identity of the Consumer.

As the response to the above token, you will get the following.
  • oauth_token
  • oauth_token_secret
  • oauth_callback_confirmed
Here, the oauth_token is per OAuth Consumer per Service Provider. But can only be used once. If an OAuth consumer tries to use twice, OAuth Service Provide should discard the request.

3. Now the OAuth Consumer, has the Request Token - and it needs to exchange this token to an Authorized Request Token. The Request Token needs to be authorized by the end user. So, the Consumer will redirect the end user to the Service Provider with following request parameter.
  • oauth_token
This oauth_token is the Request Token obtained in the previous step. Here there is no signature - and this token does not need to be signed. The token it self proves the identity of the OAuth Consumer.

As the response to the above, OAuth Consumer will get the following response.
  • oauth_token
  • oauth_verifier
Here, the oauth_token is the Authorized Request Token. This token is per User per Consumer per Service Provider.

The oauth_verifier is a verification code tied to the Authorized Request Token. The oauth_verifier and Authorized Request Token both must be provided in exchange for an Access Token. They also both expire together. If the oauth_callback is set to oob in Step 2, the oauth_verifier is not included as a response parameter and is instead presented once the User grants authorization to Consumer Application. The Service Provider will instruct the User to enter the oauth_verifier code in Consumer  Application. The Consumer must ask for this oauth_verifier code to ensure OAuth authorization can proceed. The oauth_verifier is intentionally short so that a User can type it manually.

Both the above two parameters are returned back to the OAuth Consumer via a browser redirect over SSL.

4. Now the OAuth Consumer has the Authorized Request Token. Now it will exchange that to an Access Token. Once again the Access Token is per User per Consumer per Service Provider. Here the communication is not a browser re-direct but a direct communication between the Service Provider and the OAuth Consumer. This step is needed because, in step 3, OAuth Consumer gets the token through the browser.

The request to the access token will include following parameters.
  • oauth_consumer_key
  • oauth_token
  • oauth_signature_method
  • oauth_signature
  • oauth_timestamp
  • oauth_nonce
  • oauth_version
  • oauth_verifier
Here the oauth_token is the Authorized Request Token from step - 3.

The signature here is generated using a combined key - Consumer Secret and Token Secret [from step -2 ] separated by an "&" character. I will explain later why two keys used here for signing.

As the response to the above, the Consumer will get the following...
  • oauth_token
  • oauth_token_secret
All four steps above should happen over TLS/SSL.

5. Now the OAuth Consumer can access the protected resource. The request will look like following.
  • oauth_consumer_key
  • oauth_token
  • oauth_signature_method
  • oauth_signature
  • oauth_timestamp
  • oauth_nonce
  • oauth_version
Here the oauth_token is the Access Token obtained in step-4. The signature is once again generated by a combined key. Consumer Secret and Token Secret [from step -4 ] separated by an "&" character.

The step - 5 does not required to be on TLS/SSL.

Let's get back to the original question...

Why the hell OAuth 1.0 uses two keys to sign messages..? Why Not just the consumer secret ?

During the OAuth flow there are three places where OAuth Consumer has to sign.

1. Request to get an Unauthorized Request Token. Here the request will be signed by the Consumer Secret only. ( Request for Temporary Credentials )

2. Request to get an Access Token. Here the request will be signed by the Consumer Secret and the Token Secret returned back with the Unauthorized Request Token in step - 2. ( Request for Token Credentials )

Why do we need two keys here ? It's for tighten security. What exactly that means..?

Have a look at step - 3. There the Consumer only sends the oauth_token from step - 2 to get User authorization. And as the response to this the Consumer will get the Authorized Request Token (oauth_token). Now in step - 4 Consumer uses this Authorized Request Token to get the Access Token. Here, the consumer needs to prove that - its the same consumer who got the unauthorized request token, now requesting for the Access Token. To prove the ownership of the Request Token, now the Consumer will sign the Access Token request with both the Consumer Secret and the Token Secret returned back with the Unauthorized Request Token.

3. Request to the protected resource. Here the request to the protected resource will be signed by the Consumer Secret and the Token Secret returned back with the Access Token in step - 4. ( Request for Resources )

Once again we need two keys here for tighten security. If OAuth only relies on the consumer secret for the signature, the person who steals can capture the access token [as it goes to the protected resource on HTTP] and signs the request with the stolen secret key. Then all the users of that OAuth Consumer will be compromised. Since we use two keys here, even though someone steals the consumer secret it cannot harm any of the end users - because it cannot get the Token Secret which is sent to the Consumer over SSL in step - 4. Also - you should never store Token Secrets at the Consumer end - that will reduce any risks of stealing Token Secrets. But, you need to store your Consumer secret as explained previously.

In any case if you have to store the Token Secret, then you need store it encrypted - and also make sure you store it in a different database from where you store your consumer secret. That is the best practice we follow when we store hashed passwords and the corresponding salt values.

Apart from the above benefit - the Token Secret can also act as a salt value. Since the request to the protected resource goes over the wire, a hacker can carry out a known plain text attack to guess the Key used to sign the message. If we only use the Consumer Secret, then all the users of the OAuth Consumer will be compromised. But, if we use the Consumer Secret with the Token Secret, then only that particular user will be compromised - not even the Consumer Secret.

Finally, OAuth presents a Bounded Token protocol over HTTP for accessing the protected resource. That means - each request to the protected resource the Conumer should prove that he owns the token, as well as - he is the one who he claims to be. In the bounded token model, an issued token can only be used by the exact person who the token was issued to. [Compare this with Bearer token in OAuth 2.0]. To prove the identity of the Consumer it self, it can sign the message it self with consumer_secret. To prove that he also owns the token, he needs to sign the message token_secret too. Anyone can know the Access Token, but it's only the original Consumer knows the token_secret. So the key to sign is derived from combining both the keys.