[200 OK]: A Port80 Software Blog

We're all 200 OK: Web, HTTP and IIS Insights
posts - 203, comments - 507, trackbacks - 105

Thursday, June 05, 2008

So Powerful, Amazon Uses Them: LinkDeny Techniques

For many, the most obvious use of our popular LinkDeny filter for Microsoft IIS Web servers is to block the illegal theft of bandwidth (called leeching), usually in the form of direct image hotlinking.  However, LinkDeny wasn’t designed just for that particular problem… In fact, the product aims to provide a multitude of access control mechanisms that can be used on arbitrary URLs.   Think of LinkDeny as the step before you force authentication or even the one after authentication, an extra step where you can effectively keep people from abusing your site or application.  
 
One of the most interesting ideas we included was the idea of providing enforced URLs by using a message authentication code (MAC) on the query string.  It appears that Amazon thinks this is an interesting idea as well, as they have recently added it to the Web Services APIs (http://docs.amazonwebservices.com/AmazonDevPay/2007-12-01/DevPayDeveloperGuide/WebQueryStringAuth.html).  Amazon calls this "Query String Authentication" or "Pre-Signed URLs". 

You can follow the link for the details but here's how it works in a nutshell:
 
Suppose you have a product that stores data using Amazon's Simple Storage Service (S3) and, to keep things simple, that also uses Amazon's billing and account management service (DevPay) to make sure you get paid when someone uses your product.  Now it might be very convenient in this situation if you could give third parties (your customers) direct access to your S3 data, without having to proxy all their requests.  The question arises though: how can you do this while still getting paid for the use of the data?
 
The answer is that you can provide your customers with a URL to use that is "pre-signed" with a secret (a key) that only you and Amazon know about.  If someone tries to get at your data to whom you have not given this magic URL, they don't get a thing.  The URL itself becomes an authentication mechanism for your legitimate customers.
 
The recipe for doing this is not complex.  If you have an Amazon Web Services account, you already have a secret key, plus an identifier (an access key id) that corresponds to it.  When forming the URL, you simply use that private key to hash various elements of the request, and then pass that hash, along with the access key id, on the query string.  To prevent replay attacks, you also pass an expiration time.  On the S3 side, Amazon's authentication routine simply takes access key id, looks up your corresponding access key, and uses it to recompute the hash on the request.  If the hashes match, they know this request is authentic, and the data is sent back.
 
The neat thing about this is that the entire authentication is handled in the URL itself.  There are no other HTTP headers or handshakes or anything of the kind.  Making your data available, while restricting it to authenticated users, becomes as easy as exposing a URL.
 
Now, this happens to be the same basic approach that LinkDeny can take, in order to prevent your valuable external files like images and scripts from being hijacked.  This is called the LinkDeny URL Time Limit feature.  The difference between LinkDeny's approach and the Amazon API is that all you have to do is supply a mask (a sequence of arbitrary characters) in the URLs that are in your source code (for instance, the src attributes of your image tags) -- and LinkDeny does all the rest. 
 
As your pages are sent out from the server, the mask in those URLs is replaced with a time-limited MAC that is based, as in the Amazon scheme, on a private key.  This is analogous to your providing legitimate customers with pre-signed URLs for all your dependent files.  That would be impossible to manage manually of course, but LinkDeny handles that part for you, keeping track of the currently-valid requests and rejecting any that hit their expiration time-outs, or that lack a valid signature.  While the basic use for this is to protect images and other page elements, you could easily adapt it to a RESTful Web service as well.
 
So there you have it, if you want to provide a RESTful Web service via your URLs, either purposefully or by accident, it is time to install LinkDeny and enjoy the same protection Amazon does -- without all the code!
 
Cheers,
Port80

posted @ 10:26 AM | Feedback (2)