Unlike what we might think of as traditional web services (e.g. send some information to a server, get a map), some of Amazon’s services expose big iron infrastructure through simple interfaces. Amazon’s Simple Storage Service (S3) is an unlimited supply of (really) cheap disk space. Amazon Elastic Compute Cloud (Amazon EC2), still in limited beta, promises processing power when needed. Amazon Simple Queue Service (Amazon SQS) allows systems to communicate with each other.

Given these services, it is conceivable that one could string together a fairly large system without purchasing any hardware or software to support it. Mitch Garnaat, an Amazon Developer, demonstrates this ability with his Mass Video Conversion Application. Mitch’s application is pretty straightforward, but it shows how scalable application can be built with relative ease.

Add Mechanical Turk to this mix of web services, and things get even more interesting. Mechanical Turk is not particularly germane to the mass video conversion application mentioned above, but there are many systems that simply won’t work without a lot of human processing power. A Mechanical Turk Workforce could be particularly effective for short term projects that require a lot of people. For instance, Amazon uses Mechanical Turk to identify the best photograph from a set of similar photographs.

These sorts of web services reduce the high costs associated with bringing CPU and/or storage intensive applications online. With these barriers removed, we should see some interesting projects that otherwise would have difficulty getting funding. Others are also likely to join Amazon in the market to provide these and other services. With more than just Amazon in the game, we should expect to see more and more applications that leverage these technologies.

Mass Video Conversion Application via Simon Willison’s Weblog

Leave a Reply