Programming Amazon Web Services: S3, EC2, SQS, FPS, and SimpleDB eBook Download

With this book, you'll learn all the technical details you need to: Store and retrieve any amount of data using application servers, unlimited data storage, and bandwidth with the Amazon S3 service Buy computing time using Amazon EC2's interface to requisition machines, load them with an application environment, manage access permissions, and run your image using as many or few systems as needed Use Amazon's web-scale messaging infrastructure to store messages as they travel between computers with Amazon SQS Leverage the Amazon FPS service to structure payment instructions and allow the movement of money between any two entities, humans or computers Create and store multiple data sets, query your data easily, and return the results using Amazon SimpleDB. Create HTML POST Forms That Allow Your Web Site Visitors to Upload Files Into Your S3 Account Using a Standard Web Browser By James Murty, creator of the JetS3t Java S3 library and author of Programming Amazon Web Services

Amazon?s Simple Storage Service (S3) provides cheap and unlimited online data storage for anyone with a credit card and an Amazon Web Service (AWS) account. in the examples below I will use a bucket named s3-bucket.

POST Form Web Page

Here is a web page with an S3 POST Form that you can use as a template for your own forms:

S3 POST Form File to upload to S3:

This template demonstrates some important features of an S3 POST form, and the web page that contains it:

The web page that contains the form has a meta tag in the head section that tells web browsers to use the UTF-8 unicode character encoding. This name can be set in advance when you know what information the user will upload, for example: uploads/monthly_report.txt.

If you do not know the name of the file a user will upload, the key value can include the special variable ${filename} which will be replaced with the name of the uploaded file. policy A Base64-encoded policy document that applies rules to file uploads sent by the S3 POST form. If you do not know what type of file a user will upload, you can either prompt the user to provide the appropriate content type, or write browser scripting code that will automatically set this value based on the file?s name.

If you do not set the content type with this field, S3 will use the default value application/octet-stream which may prevent some web browsers from being able to display the file properly. At a minimum, you will need to edit the form?s action parameter to point to your own S3 bucket, and set the value of the AWSAccessKeyId field to your AWS Access Key credential.

To complete the form and make it acceptable to the S3 service, you will also need to generate a policy document and signature value.

Policy Document

S3 POST forms include a policy document that authorizes the form and imposes limits on the files that can be uploaded. When S3 receives a file via a POST form, it will check the policy document and signature to confirm that the form was created by someone who is allowed to store files in the target S3 account.

A policy document is a collection of properties expressed in JavaScript Object Notation, which simply means that the document?s structure and content must conform to a certain format. This policy has an expiration date of January 1st 2009:

{"expiration": "2009-01-01T00:00:00Z", "conditions": [ {"bucket": "s3-bucket"}, ["starts-with", "$key", "uploads/"], {"acl": "private"}, {"success_action_redirect": "http://localhost/"}, ["starts-with", "$Content-Type", ""], ["content-length-range", 0, 1048576] ] }

To create a valid S3 POST form, you must include a policy document whose conditions section contains a rule for almost every input field in the form.

Because our template POST form includes the input fields acl, success_action_redirect, and Content-Type, our policy document includes rules corresponding to these fields. Our policy document also includes an extra content-length-range rule that limits the size of files that can be uploaded.

There are three kinds of rules you can apply in your policy document:

Equality rule, which checks that an input field?s value is set to a given string. In the policy document above, we use starts-with rules for the key and Content-Type fields because we do not know in advance the name of the file a user will upload, or what type of file it will be. The policy document above includes a content length rule that will prevent the form from uploading files larger than 1MB in size (1,048,576 bytes). If there are any discrepancies between the input field values in your form and the rule values in your policy document, or if your form contains input fields that do not have corresponding rules in your policy, the S3 service will reject the form and return an incomprehensible XML error message to your users.

Sign Your S3 POST Form

To complete your S3 POST form, you must sign it to prove to S3 that you actually created the form. If you do not sign the form properly, or if someone else tries to modify your form after it has been signed, the service will be unable to authorize it and will reject the upload.

To sign your form you need to perform two steps:

Base64-encode the policy document, and include it in the form?s policy input field. Here are some example code fragments to do the job with different languages, assuming you have already defined the variables policy_document and aws_secret_key.

Ruby require 'base64' require 'openssl' require 'digest/sha1' policy = Base64.encode64(policy_document).gsub("n","") signature = Base64.encode64( OpenSSL::HMAC.digest( OpenSSL::Digest::Digest.new('sha1'), aws_secret_key, policy) ).gsub("n","") Java import sun.misc.BASE64Encoder; import javax.crypto.Mac; import javax.crypto.spec.SecretKeySpec; String policy = (new BASE64Encoder()).encode( policy_document.getBytes("UTF-8")).replaceAll("n",""); Mac hmac = Mac.getInstance("HmacSHA1"); hmac.init(new SecretKeySpec( aws_secret_key.getBytes("UTF-8"), "HmacSHA1")); String signature = (new BASE64Encoder()).encode( hmac.doFinal(policy.getBytes("UTF-8"))) .replaceAll("n", ""); Python import base64 import hmac, sha policy = base64.b64encode(policy_document) signature = base64.b64encode( hmac.new(aws_secret_key, policy, sha).digest())

Once you have calculated the values for the policy and signature input fields and included these values in your form, the form should be complete. Save the web page and form document as an .html file, open it in your favorite web browser, and test it by uploading some files to your S3 bucket.

Conclusion

The form web page and policy document templates in this article should give you a starting point for creating your own upload forms. With some minor modifications to the template documents and a little coding, you will be able to create authorized forms that make it easy for your web site visitors to upload files to your S3 account.

The S3 service?s POST support is a powerful feature with many potential uses. You could create a single upload form to allow your friends and colleagues to send you files that are too large for email, or you could modify your web applications to generate forms on-demand so your users can store their data in S3 rather than on your own server.
I'd recommend this book for anyone who needs an EC2/S3/AWS reference.


Good Resource For Working With Amazon Web Services
'Programming Amazon Web Services: S3, EC2, SQS, FPS, and SimpleDB' is a good resource for anyone that is using the Amazon suite of web products and need to learn more about how to get the most out of these powerful set of web 2.0 tools.

For anyone that doesn't know what these tools are, here's a quick one-liner about each:

S3 - online storage to store and retrieve data

EC2 - online computing to be able to run jobs on a farm of machines

SQS - web messaging infrastructure for computer-computer communication

FPS - flexible payment system for moving money online

SimpleDB - store and retrieve datasets online

I like the content of this book and feel that it plays an important part in this niche market but my major qualm is that the code is written solely in Ruby in this book.

Computer science students, get this computer ebook by download itProgramming Amazon Web Services: S3, EC2, SQS, FPS, and SimpleDB from Rapidshare/Megaupload/Hotfile for free.