I mentioned recently that my new kick is storage platforms for Internet-scale services. Sometimes it's hard to describe the problems that large scale services have with modern relational databases. Thankfully, I now have the post I want a big, virtual database by Greg Linden to crib a simple 1 sentence description of the problem and solution. Greg writes

In my previous story about Amazon, I talked about a problem with an Oracle database.

You know, eight years later, I still don't see what I really want from a database. Hot standbys and replication remain the state of the art.

What I want is a robust, high performance virtual relational database that runs transparently over a cluster, nodes dropping in an out of service at will, read-write replication and data migration all done automatically.

I want to be able to install a database on a server cloud and use it like it was all running on one machine.

It's interesting how your database problems change when you're thinking about hundreds of transactions a second over terabytes of data belonging to hundreds of millions of users.You might also be interested in Greg's post on Google's BigTable.

PS: If building large scale services relied on by hundreds of millions of people to communicate with their friends and family sounds interesting to you, we're still hiring for test, developer and PM positions.
 

Sunday, April 16, 2006 8:26:11 PM (GMT Daylight Time, UTC+01:00)
Adam Bosworth ("the father of Microsoft Access") posted a similar set of criteria, but from the users' standpoint, as "Where Have All the Good Databases Gone[?]" http://www.adambosworth.net/archives/000038.html). It seems to me that Adam's users include developers and admins.

The content-addressable BitVault architecture (http://oakleafblog.blogspot.com/2006/04/very-large-databases-bricks-bitvault.html, in process) appears to me to offer the feature set for which Greg is searching but only for read-mostly reference data (so far).

Maybe BigTable running on GFS is the answer for frequently updated data.

--rj
Comments are closed.