RESTful APIs with Bedrock (part V)

This is the 5th in a series of blogs on using Bedrock to create a RESTful API.  So far we've checked off several requirements for building a RESTful API.  We've been able to demonstrate the basics of creating a RESTful API using Apache and Bedrock.  We've satisfied requirements #1 through #4 and #6.

1. Parse a URI
2. Determine the operation to be performed
3. Determine the result format requested by the client
4. Retrieve, validate, and de-serialize client data
5. Retrieve data from the back end
6. Serialize data to send back to client
7. Set HTTP response headers
8. Handle errors

We'll round out the discussion by showing how Bedrock can satisfy the 3 remaining requirements.

Retrieving data from the back end

Retrieving data from the back end is a pretty broad topic and can be satisfied in many ways.

• Serving static content
• Accessing data from a database
• Returning prepared data (a report, a PDF, or a spreadsheet for example)

Static Content

Serving static content using Bedrock can be as simple as using Bedrock's <include> tag to include content to be delivered to the client.

<if $env.REQUEST_URI --re 'documents/(?<document\>.*)$'>

   <include $document>

</if>

Why use this model?  Well, using Bedrock's <include> tag we can place our content anywhere on the server, presumably outside of Apache's DocumentRoot and then just tell Bedrock where it can find the.inc files using the INCLUDE_DIR configuration variable in Bedrock's configuration file.  Included snippets are processed as a Bedrock page, so you also get the benefit of being able to create dynamic content using Bedrock include files.

Alternately, you might use a redirect status code to redirect the client to your static content.

<null $header.location( ('/documents/' + $document) )>

You probably want to avoid using a redirect since that requires two requests from your clients merely to get static content.  A better approach might be to host your static content on a CDN like Amazon's CloudFront rather than including static content as part of your REST API to completely offload your server from dealing with static content when low-latency and reducing traffic on your site is important.

You can also consider using Apache's ProxyPass directive if you want to create a consistent and self contained API.  While using this approach won't offload requests from your server, passing the requests through your Apache server might be what you want to do anyway.   This gives you the ability to control access or monitor the usage of those specific assets.

ProxyRequests Off

ProxyPass /api/documents/  http://d111111abcdef8.cloudfront.net/

Don't forget to turn ProxyRequests Off since we don't want to be a proxy for anonymous browsing!

If the static content is to be hosted on your server, you can also use Apache's RewriteRule directives to map your API URIs to the content.

RewriteEngine on

RewriteCond %{REQUEST_METHOD} GET

RewriteRule ^api/documents/([^/]*) /documents/$1

Apache's mod_rewrite is extremely robust and you can do some interesting things, including negotiating content by looking at the Accept headers and rewriting your URIs.

RewriteCond %{HTTP:Accept} application/json [NC]

RewriteRule ^api/contacts/(.*)$ api/contacts/$1?format=JSON

There's a whole lot one can do with Apache directives to support your RESTful API.  A working knowledge of Apache configuration directives could save you a lot of coding.

Accessing Data from a Database

Bedrock's built-in support for database access using the <sql> and <sqlselect> tags is the quickest way to expose data via your API.  In general though, more complex business logic and database access might be best implemented inside Perl classes exposed as Bedrock Application Plugins.  The quick and dirty approach is perfect for prototyping though.

<sqlselect "select * from customer"

                --define-var="result"></sqlselect>

<var --json $result>

Here's an example demonstrating how to access a specific row of a table.

<if $env.REQUEST_METHOD--eq 'GET'>

  <if $env.PATH_INFO --re 'customer/(?<id\?>\\d+)$'>

    <sqlselect "select * from customer where id = ?"

               --bind=$id

               --define-var="result"></sqlselect>

    <if $result.length() --eq "0">

      <raise "404|id not found">

    <elseif $result.length() --gt "1">

      ...this should never happen!

      <raise "300|not unique">

    </if>

  <elseif ...>

  ... handle other URIs

  </if>

</if>

Returning Prepared Data

Sometimes you actually need to do a little bit of work to prepare and deliver the content requested by your clients in order to implement an API's URI.  Let's suppose you need to run a SQL query, translate the data, format a report and possibly create a PDF for the client.  This type of request might take more than a second or two so you might consider an asynchronous pattern.  In that case, use the 202 status code (Accepted) to let your client know that you got the request and it's being processed.   I'll blog about this model in more detail in the future, but generally the pattern might look something like this:

1. Client requests a resource that requires asynchronous treatment
2. Server returns a 202 Accepted status along with a token of some kind which allows the client to query the status of the request
3. The server begins processing the request.
4. Client uses token to request resource until resource is available

While Bedrock doesn't necessary directly support this asynchronous pattern, it doesn't preclude its implementation either.  Any one of a number of techniques can be used to asynchronously handle a client request including invoking a CGI that forks, notifying a daemon process to create a resource or sending a message to a message queue so that some other process can handle the request.

Of these, my preferred method with Bedrock would be to create a unique request id and then place a message on a queue for another worker to process.  As a huge fan of Amazon web services, I'd opt to place a message on an SQS (Simple Queue Service) queue.  In fact, I've created a little Bedrock plugin (BLM::SQSMessage) that helps me with that task.

<sink><plugin:Data::UUID>
<null:uuid $DataUUID.create()>
<null:id $DataUUID.to_string($uuid)>

<plugin:SQSMessage 'myqueue'>
<null $SQSMessage.send('event', 'sales_report',
                       'name', 'monthly',
                       'id', $id)>

<hash:result id $id>

<null $header.set('Status', '202')>
</sink><var --json $result>

My BLM::SQSMessage plugin accepts a hash of key/value pairs that are considered to be the message for my service.  Using this pattern you can decouple long running or CPU intensive operations from your web server creating more scaleable APIs.  The typical method here is to spark up a number of Amazon EC2 servers that specifically are used to handle messages from your application.  You can scale these servers up and down depending on your load.  This helps keep your web server load light and your users happy.  :-)

Not all operations are going to require that you handle them asynchronously and your Bedrock API might easily handle the data preparation tasks and simply return the data as an HTML, JSON or XML document using Bedrock as it was originally intended to be used - as a templating engine.

Setting HTTP Response Headers

We've been doing this one all along, so you might have picked up on it.  Bedrock's $header object allows you to set the HTTP response headers, including the status.  By default, if your Bedrock page is successfully processed, Bedrock will return a 200 status.  You can override this as shown here:

<null $header.set('Status', '202')>

You'll want to become familiar with the various HTTP response codes and when they should be used.  In some cases you may need to use your imagination when fitting your situation to an HTTP response code, however there are some standards regarding their use that you should employ.  Lot's of people have opined on this one...Google it.

Handling Errors

Finally, your API needs to handle errors and the web framework should make it easy to do so.  As described above you should return appropriate HTTP status codes depending on the situation.  Bedrock's <try/catch> mechanism is perfect for this task.  I generally like to use the <catch> tag's ability to filter errors using the regular expression argument.  I then use the <raise> tag to add a message to each specific situation where I want to indicate an error condition.

<try>

...

  <unless $env.PATH_INFO --re 'customer/\\d+'>

    <raise '400|id required'>

  </unless>

...

<catch '400'>

  <null:error $@.split('\\|')>

  <null $header.set('Status', 400)>

  <hash:result error_message ('Bad Request - ' + $error.[1])>

  <var --json $result>

<catch '404'>

...

</try>

Well, there you have it, all of the components to create a RESTful API.  Clearly a first class language like Perl offers much more to the programmer for creating a RESTful API, however for a quick and dirty prototyping exercise Bedrock can more than handle the task.