Monday, March 17, 2014

The Big Deal with Big Data!

I pre-date the relational database management system world (RDBMS), having been weaned on things you may never heard of, like ISAM (indexed-sequential access method). So, I have to admit that I was a little skeptical about the NoSQL world. It sounded a lot to me like ISAM, and why go backwards? But it is not fair to dismiss something out of hand so of course I had to investigate these NoSQL databases. And while I do not believe at all that the days of RDBMS are numbered as some folks say, I do believe that NoSQL has its place. To understand why, you need to know the terrain.

Onwards to RDBMS

In a simple view of the early days we basically used a sequential table of records. Most of the time the records were fixed length, reading the whole set sequentially was fast, and we used indexes to quickly locate a record based on a key. Does any of that sound familiar? Need to look up records by more than one field? Add an index! When you need to sort the records faster you pick an index and sort that instead of the table itself. An example of a single-index product still prevalent is Ken Thompson's dbm, which was included with Unix in 1979. I would provide a citation, but it would be myself because I first came across it when I upgraded to Unix Version 7.