I have a friend who works as a security analyst, and I was telling him the other day how much I love the work that is done by everybody else to get good authentication working, so that I don’t have to worry about it. A great example of this is some recent work I was doing in App Engine for the soon to be launched Hello Uni website, where I had to give clients access to resources stored in Google Cloud Storage (GCS), without making them public.
There are a few ways of doing this, but for our purposes signed URL’s worked best.

What’s a signed URL?

Suppose someone is using a web service that stores it’s data with another service. In our case, clients are accessing the Hello Uni website which is being served via App Engine, and want to access restricted videos and other study materials stored in GCS. Without a signed URL, the whole scenario would look something like this:

This behaviour is good. Our files in GCS ought not be publicly available, so a simple GET request to the right URL request shouldn’t let you access them. Naively, the two ways of letting the user securely get the file, these are 1) giving authorised users access to GCS, or 2) letting the app (HelloUni here) get the file from GCS, and pass it to the user. Option 1 isn’t easy to implement and could lead to messy, tangled permissions, and option 2 would consume a lot more bandwidth and processing time than I’d like to pay for. Here’s where Google gives us a third option: Signed URLs. With Signed URLS, a digitally signed string is passed along with the file request as a form field in the URL The signed string is built from information available in the HTTP request. GCS uses the authenticating app’s public key, the signed string, and the HTTP request information that built the signed string to establish if the app’s private key really did sign the signed string, thus establishing the authenticity of the request.

The scenario essentially looks like this now:

Making a signed URL 1: Setting up certificates on the development server

Google makes this wonderfully simple in AppEngine, through the app_identity module. The two functions sign_blob and get_service_account_name give everything you need to make a signed URL. Things get a little trickier on the App Engine development server where you need to tell the app what private key to use when signing. You can download this for your app from the Google Cloud Platform console in IAM & Admin > Service Accounts. I downloaded the private key as a PKCS 12 key (.p12 file) and converted it to a .pem file via:

cat secret.p12 | openssl pkcs12 -nodes -nocerts -passin pass:password | openssl rsa > secret.pem

Then when you start run your development webserver, you’ll need the flags “–appidentity_email_address service@account.email” and “–appidentity_private_key_path /path/to/key”.

Making a signed URL 2: A simple signing function

Now that the functions sign_blob() and get_service_account_name() are working, we can make our signed URL. Establishing the validity of a signature essentially works by knowing three things:

1. A message
2. The public key of the signer
3. A signed version of the message

Based on these three things, we can verify if the message was signed using the private key belonging to the signer. Quite cleverly, the HTTP request made to GCS is used to build the message, and so what we need to pass to the signing algorithm is a text representation of a HTTP request. The full rules for making this string are here. Depending on the complexity of the HTTP request, the string to sign can become complex. For simple requests though the snippet below gives an idea of how a simple function can produce a signed URL.

import time
from google.appengine.api.app_identity import sign_blob
from google.appengine.api.app_identity import get_service_account_name
from base64 import b64encode


def get_signed_url(url, delay=3600):
    """Return a signed URL for Google Cloud Storage access
    args:
        url: url to sign
    kwargs:
        delay: time in seconds from now for which link is valid
    """
    method = "GET"
    content_md5 = ""
    content_type = ""
    expires = str(int(time.time()) + delay)
    # Add string for canonical extension headers if necessary
    if url[0:4] == r"http":
        resource = r"/".join([r""] + url.split(r"/")[3:])
    else:
        resource = r"/".join([r""] + url.split(r"/")[1:])

    blob = "\n".join([method, content_md5, content_type, expires,
                      resource])
    # need str() otherwise blob is unicode
    signature = sign_blob(str(blob))[1]
    signature = b64encode(signature)
    signature = signature.replace(r"+", r"%2B").replace(r"/", r"%2F")
    url = "".join([url, "?GoogleAccessId=", get_service_account_name(),
                   "&Expires=", expires, "&Signature=", signature])
    return url

And that’s all there is too it, pretty cool!

Helpful sources:

• Setting up keys on the devserver.
• Signed URLs with App Engine
• The Python App Identity API
• Wikipedia: Digital Signatures