Business Intelligence Blog from arcplan

Big Data FAQs – A Primer


The big data trend promises that harnessing the wealth and volume of information in your enterprise leads to better customer insight, operational efficiency, and competitive advantage. The marketing hype around big data and the pace of studies, analyst reports, and articles on the subject can be mind-numbing for companies that want to take advantage of big data analytics but do not know how to separate fact from fiction and determine real use cases for their business. So here’s a big data primer for those just getting in the game.

1) What exactly is big data?
“Big data” is an all-inclusive term used to describe vast amounts of information. In contrast to traditional structured data which is typically stored in a relational database, big data varies in terms of volume, velocity, and variety. Big data is characteristically generated in large volumes – on the order of terabytes or exabytes of data (starts with 1 and has 18 zeros after it, or 1 million terabytes) per individual data set. Big data is also generated with high velocity – it is collected at frequent intervals – which makes it difficult to analyze (though analyzing it rapidly makes it more valuable). Additionally, big data is usually not nicely packaged in a spreadsheet or even a multidimensional database and often includes unstructured, qualitative information as well.

2) Is it a new trend?
Not exactly. Though there is a lot of buzz around the topic, big data has been around a long time. Think back to when you first heard of scientific researchers using supercomputers to analyze massive amounts of data. The difference now is that big data is accessible to regular BI users and is applicable to the enterprise. The reason it is gaining traction is because there are more public use cases about companies getting real value from big data (like Walmart analyzing real-time social media data for trends, then using that information to guide online ad purchases). Though big data adoption is limited right now, IDC determined that the big data technology and services market was worth $3.2B USD in 2010 and is going to skyrocket to $16.9B by 2015.

3) Where does big data come from?
Big data is often boiled down to a few varieties including social data, machine data, and transactional data…

Continue reading this post >>