Database? No database?

Apr 28, 2006

There are a lot of interesting discussions and posts going on lately on how certain high profile websites. Particularly, a series of posts on O’Reilly Radar. The first post is about Second Life, where they talk about MySQL (separate master-slave pairs handling the data partitions with one master-slave pair indicating where what data lives).

The second installment features Bloglines which uses MySQL (Users and passwords in one master-slave, feed information in another) but also large parts in some file storage.

A third posting talks about Flickr, who basically started out with the “one database fits all”-methodology on MySQL. And this is where I think experience really comes into play. If you were designing the Flickr database with, for example, Bloglines’ experience under your belt you would not have started out that way. But Flicker too, couldn’t escape the segmentation and divided their data up in what they call “shards” (separate master-slave pairs as I read it).

So, it is pretty apparent that MySQL is being used for some fairly large sites while most of these employ the strategy of segmenting the data across several master-slave combinations. Some are actually using LiveJournals’ memcached too!