A number of blogs I read have been talking about Amazon's S3 service a lot recently. I've seen posts from Jeff Atwood, Shelley Powers and most recently Dave Winer. I find it interesting that S3 is turning into a classic long tail service that works for both startups who are spending hundreds of thousands to millions of dollars a year to service millions of users (like Smugmug) to bloggers who need some additional hosting for their cat pictures. One reason I find this interesting is that it is unclear to me S3 is a business that will be profitable in the long term by itself.

My initial assumption was that S3 was a way for Amazon to turn a lemons into lemonade with regards to bandwidth costs. Big companies like Amazon are usually billed for bandwidth using 95th percentile billing, which is explained below

With 95th percentile billing, buyers are billed each month at a fixed price multiplied by the peak traffic level, regardless of how much is used the rest of the time. Thus with the same nominal price, the effective price is higher for buyers with burstier traffic patterns.

So my assumption was that S3 allows Amazon to make money from bandwidth they were already being charged for and not using. As for storage, my guess is that they are either making a miniscule amount of profit or at cost. Where this gets tricky is that, if S3 gets popular enough then all of a sudden it no longer is a way to make money from bandwidth they are being billed for but aren't using but instead impacts their actual bandwidth costs which then changes the profit equation for the service. Without any data on Amazon's cost structure it is unclear whether this would make the service unprofitable or whether this is already factored into their pricing.

On the other hand, Amazon's Elastic Compute Cloud (EC2) isn't something I've seen a lot of bloggers rave about. However it seems to be the service that shows that Amazon is making a big play to be the world's operating system in the sky as opposed to dabbling in providing some of its internal services to external folks as a cost savings measure. With EC2 you can create a bunch of virtual servers in their system and load it up with an Amazon Machine Image (AMI). An AMI is basically a server operating system and the platform components you need on it. Typical AMIs are an instance of a LAMP system (Linux/Apache/MySQL/PHP/Perl/Python) although I did see one AMI that was an instance of Windows 2003 server. You can create as many or as few server instances as you need and are billed just for what you need.

I suspect that the combination of EC2 and S3  is intended to be very attractive to startups. Instead of spending hundreds of thousands of dollars building out clusters of servers, you just pay as you go when you get your monthly bill. There are only two problems with this strategy that I can see. The first is that, if I was building the next Digg, Flickr or del.icio.us I'm not sure I'd want to place myself completely at the mercy of Amazon especially since there doesn't seem to be any SLA published on the site. According to the CEO of Smugmug in his post Amazon S3: Outages, slowdowns, and problems they've had four major problems with S3 in the past year which has made them rely less on the service for critical needs. The second issue is that VC money is really, really, really easy to come by these days judging from the kind of companies that get profiled on TechCrunch and Mashable. If the latter should change, it isn’t hard to imagine dozens of enterprising folks with a couple of thousand dollars in their pockets deciding to go with S3  + EC2 instead of seeking VC funding. But for now, I doubt that this will be the case.  

What I suspect is that without some catalyst (e.g. the next YouTube is built on S3  + EC2)these services will not reach their full potential. This would be unfortunate because I think in much the same way we moved from everyone rolling their own software to shrinkwrapped software, we will need to move to shrinkwrapped Web platforms in the future instead of everyone running their own ad-hoc cluster of Windows or LAMP servers and solving the same problems that others have solved thousands of times already.

I wonder if Amazon has considered tapping the long tail by going up against GoDaddy's hosting services with S3  + EC2. They have the major pieces already although it seems that their prices would need to go down to compete with what GoDaddy charges for bandwidth although I suspect that Amazon's quality of service would be better.


 

Categories: Competitors/Web Companies | Web Development
Tracked by:
http://blog.broadbandmechanics.com/2007/03/end-of-march-07-links [Pingback]
http://projectshave.wordpress.com/2007/04/09/why-is-ec2-so-cheap/ [Pingback]
http://viezbaq.net/linux/index.html [Pingback]
http://yftbsy1.net/internet/sitemap1.html [Pingback]
http://weujmru.net/pets/sitemap1.html [Pingback]
http://restablog.com/household/sitemap1.html [Pingback]
http://ofiyspg.net/sitemap1.html [Pingback]
http://gator393.hostgator.com/~rocata/sitemap1.html [Pingback]
http://fl7vaa4.net/southwest/sitemap1.html [Pingback]
http://biggest-hosting10.com/~rocata/table/index.html [Pingback]
http://host239.hostmonster.com/~blogford/sitemap2.html [Pingback]
http://gator413.hostgator.com/~digital/wedding/sitemap1.html [Pingback]
http://lgicsge.net/tax/sitemap1.html [Pingback]
http://mwtobvc.net/10/index.html [Pingback]
http://rmarvwy.net/dining/index.html [Pingback]
http://d579737.u108.floridaserver.com/sitemap2.html [Pingback]
http://gator442.hostgator.com/~hockteam/leather/sitemap1.html [Pingback]

Thursday, March 15, 2007 4:53:43 PM (GMT Standard Time, UTC+00:00)
I think S3 is designed to make it easier to move to EC2...EC2 forces you to use S3 for persistence because when a VM dies, it dies completely. This gives you a clean split between transient machine state and persistent data, and is one of the things you need if you want the ability to deploy 50 servers under peak load, but roll back to 3 servers in times of idleness.

however, it does force you to rewrite every single application, or at the very least every database, because saving state to the filesystem doesnt cut it any more. And EC2 is not a file system, even though there is a user-mode linux filesys driver for it for sale. This is one reason why I'm more likely to use a 'classic' single host Xen hosting service in the near future.

Also, under EC2, you dont pay for network traffic to and from S3, so it becomes cheaper to host with them once you've adopted S3 behind the scenes. You will get locked in, by cost if not by anything else.
Thursday, March 15, 2007 5:02:50 PM (GMT Standard Time, UTC+00:00)
While VC money may be easy to come by, no amount of money gets around the fact that developing and maintaining server infrastructure is difficult. Having money to throw at the problem just means you can hire more developers, ops engineers etc, but they still have to build the system, which takes a while. Being able to use EC2 + S3 means you chop off a bunch of upfront development time, which is presumably something that is important for a startup.
Alex Mallet
Thursday, March 15, 2007 6:51:47 PM (GMT Standard Time, UTC+00:00)
Dare, EC2 still gives you individual machines, just like you would get if you leased servers. Clustering and scaling solutions are still up to the implementor to take care of. Only the storage aspect is scalable, which you can achieve by leasing SAN space with most providers.

In addition to that, S3 is not good at delivering content - it is more meant for downloads where you don't need latency such as a product download rather than hosting images or video etc.

Also, I don't buy the story that S3 and EC2 are running as an extension to existing Amazon infrastructure - whenever S3 has gone down, the main Amazon website continues to run. As a developer, I have to ask if there is a problem with the network or infrastructure, will Amazon focus on bringing their website up or their S3/EC2 customers?

Investors also don't like this web services strategy - they have seen it all before during the boom. Their IT spend continues to increase quarter-on-quarter while their bottom line and stock price suffer. Their net profits have been in free-fall for the past 2 years, see:

http://www.businessweek.com/magazine/content/06_46/b4009001.htm

I don't think S3 and EC2 are a bad idea, it is an ok product (despite it's failings, but we actually dont use them anymore). It would be great if Google/Microsoft or Yahoo went into this space
Thursday, March 15, 2007 8:20:50 PM (GMT Standard Time, UTC+00:00)
Alex said something to that effect, but I can defititely attest that infrastructure costs are a killer.
Steve L has a great point about lockin. I'd be more concerned about that kind of lockin than what you suggested - naturally if Amazon pulled the rug out from under you you'd be screwed, but as far as service outages, well, that's unavoidable with anyone - hosting provider can screw up, network provider can have a router blow up, co-lo can lose power. The SLA might be a concern, but really, the compensation you get from an SLA is small consolation for taking your site down.
Thursday, March 15, 2007 9:42:08 PM (GMT Standard Time, UTC+00:00)
I wouldn't see a Digg or YouTube going on S3/EC2 but I can see a million other uses from less ambitious web sites/services to hosting, say, your own version of SugarCRM.
pwb
Friday, March 16, 2007 12:55:32 PM (GMT Standard Time, UTC+00:00)
A great post by Dare and good comments. I am particularly surprised by the "been there, done that" comments that indicate that enough companies have seriously tried the service to draw that sort of comment here.

We tend to forget our history with the Internet and I like how Dare brings perspective. Is it possible that the pioneering Amazon Associates program is so old it isn't even relevant? Gee! Amazon has been pushing the boundaries in a serious manner for a while.

One of the most interesting items in the Amazon offerings is rarely mentioned. Maybe it's not Web 2.0/3.0/X.0 enough. Amazon provides inventory, warehouse, and shipping services for other companies. They claim it is very competitive. Anybody else of that scale provide a service like that? It appears to be very flexible, with no fixed contracts. I'd go on, but I'm not trying to advertise their other service, just bring it in to the discussion.
Sunday, March 18, 2007 3:00:16 AM (GMT Standard Time, UTC+00:00)
I agree with all the comments posted, but I also believe there might be more legs to Amazons EC2 and S3 gambit than might be evident at the moment. Our own efforts at developing geospatial mapping services at WeoGeo required us to develop infrastructure in a data center and deal the 95 percentile issues. Our typically file sizes are large 100s MB to 100s GB. We were always going to be stuck with large bandwidth, server, and disk costs.

Our first estimates of upfront costs were >$500K for commitments for hardware and bandwidth just to start. And this was without any insurance for a possible Digg hit, where you suddenly get overwhelmed by the traffic, and just at the point where you would be looking for the greatest marketing exposure. Even if VC money is plentiful, it is never cheap. If a viable alternative to taking outside money is possible, it should be seriously considered.

This is where I think Amazon Web Services is extremely valuable. We made the bet that we could build the AMIs that would deliver an acceptable SLA. A key component of this effort was the development of WeoCEO (see the AWS forum post at http://developer.amazonwebservices.com/connect/thread.jspa?threadID=14382&tstart=0). This AMI allowed us to continue development of our geospatial portal and hosting service on a completely scalable platform. This means we only have to pay for the cycles that we require, which also means we do not have to invest capital into depreciating assets that may never be fully used. Instead, we can direct our capital towards product development and servicing our community.

We also get Digg insurance. Our controlling AMI (WeoCEO) scales new services as a function of a number of different use statistics. If server use goes up, we add more servers automatically. When use drops, we scale back. If Amazon can deliver reasonable service, without a complete outage, were covered. And if Amazon gets too flaky, we can also move back into a custom rack in a data center once we have demonstrated the potential of WeoGeo. This will make acquire VC money a little easier, and hopefully a little cheaper.

In the end, the decision to try AWS is a bet, much like all others in business. However, I think Amazon may be on to something. The idea of commodity computing cycles, not just commodity computers, is a fundamental shift in todays infrastructure and business planning. I personally like not having to pay for the care and feeding of racks of computers, and I bet there will be a fair share of others out there who feel the same way.
Thursday, April 05, 2007 12:51:20 AM (GMT Daylight Time, UTC+01:00)
While the amazon services definitely have their warts, I think that the buzz that we are seeing in the developer community is going to more than make up for any shortcomings, and that will result in at least a couple of "killer apps" being generated on these services.

Keep in mind, EC2 is still closed beta right now. Once this thing opens up, everybody and their dog can use it for their crazy ideas. We are seeing render farms, video encoding, and lots of other sporadic applications being run, not just full time webservers.

Granted, I am a little biased, but I am seeing a "kids in the candystore" mentality about EC2 and virtualization in general right now. Everybody wants in.

These reliability issues will be faint unpleasant memories in a couple of years, and amazon will have the first out of the gate advantage for commodity cloud computing.
Sunday, April 29, 2007 2:13:03 PM (GMT Daylight Time, UTC+01:00)
"I'm not sure I'd want to place myself completely at the mercy of Amazon especially since there doesn't seem to be any SLA published on the site."

Agreed, but ultimately, isn't one of the main benefits of going to a virtualized platform (based on Xen no less) the ability to shift these domU machines around? I haven't played around with EC2 or S3 personally, but it seems to me you could start migrating servers as soon as Amazon starts choking, maybe using something like VMcasting -- http://www.vmcasting.org/ -- and you're in the clear. Don't like Amazon's [eventually published] SLA or ToS, you can migrate those boxes to another cloud -- if one ever comes about. If not, you can build out your own iron, replicate and be done with it.

I don't know too much about "AMI" yet, but I would imagine Amazon's smarter than trying to lock you in when the platform's main benefit is openness. But I suppose time will tell.
Comments are closed.