Geir Magnusson from 10gen presented a talk titled Cloud Data Persistence or ‘We’re in a database reneaissance – pay attention” today at QCon London 2009. The main message of his talk was that “physical limitations of today’s technology combined with the computational complexity of conventional relational databases are driving databases into new exciting spaces”, or to put it simpler the database landscape is changing and we should keep our eyes on that.
Presenting solutions such as Google’s BigTable and Amazon Dynamo, Magnusson said that distributed databases allow us to manage huge data volumes (BigTable is designed for petabyte tables) but issues of high availability while distributing across a number of systems bring a set of requirements that often requires the solution to diverge from the relational model and lean more towards key-value or document stores.
Magnusson mentioned a number interesting projects to watch closely:
Opensource clones and alternatives to Google’s BigTable:
Opensource clones and alternatives to Amazon Dynamo:
Other interesting opensource projects:
- MongoDB – document store
- CouchDB – JSON document store
- MemcachedDB distributed key/value database accessed via memcached apis
- Drizzle – fork of mysql targetted for the cloud
- Hadoop- Distributed file system, M/R framework
- LightCloud – Distributed key/value database based on Tokyo Cabinet
- Scalaris scalable, transactional, distributed key-value store in Erlang
I will be covering QCon in detail on this blog. Click here for other news and reviews from the conference.