According to industry forecasts, the world’s volume of data doubles every 18 months, and all forms of enterprise data will grow 650% over the next five years. The talk around big data is more than marveling at the mass of information we’re creating. As analysts and data scientists, we’re trying to find the good stuff – the trends, the data that allows us to make better decisions now and in the future, to predict the moves that will make our business more successful down the line.
Big data (explained in our previous article here) might be new to you, but I’ve seen some analyst reports referencing big data ideas as far back as 2001. However, the BI world is talking about it more and more as data volumes grow and we begin to see the potential knowledge to be gained in these data sets.
So maybe you’re thinking, how can big data benefit my company? It’s hard to think conceptually about it, so let’s take a look at some concrete examples of how companies are using big data today. We’ll start with the retail industry. Keep in mind that many of these ideas can be used on a smaller scale for retailers of any size.
Wal-Mart sifts through massive amounts of unstructured social media and search data to find out what products consumers are talking about. They use that information to set their ad buying strategy on sites like Google, with the goal of competing for e-commerce sales – currently dominated by Amazon.com.
Acquiring thorough insight into your data and tapping into the needs and buying patterns of customers are growing needs for businesses striving to increase operational efficiency and gain competitive advantage. Throughout 2011, I noticed a heightened interest in ‘big data’ and ‘big data analytics’ and the implications they have for businesses. In August, Gartner placed big data and extreme information processing on the initial rising slope of their Hype Cycle for Emerging Technologies, so we’re just at the beginning of the big data trend. A recent TDWI survey reports that 34% of organizations are tapping into large data sets using advanced analytics tools with the goal of providing better business insight. The promise of big data analytics is that harnessing the wealth (and volume) of information within your business can significantly boost efficiency and increase your bottom line.
The term ‘big data’ is an all-inclusive term used to describe vast amounts of information. In contrast to traditional data which is typically stored in a relational database, big data varies in terms of volume, frequency, variety and value. Big data is characteristically generated in large volumes – on the order of terabytes or exabytes of data (one exabyte starts with 1 and has 18 zeros after it) per individual data set. Big data is also generated in high frequency, meaning that information is collected at frequent intervals. Additionally, big data is usually not nicely packaged in a spreadsheet or even a multidimensional database and often takes unstructured, qualitative information into account as well.
So where does all this data come from?
Thanks to our loyal readers for inspiring us to share ideas with you every week on this Business Intelligence Blog. We look forward to bringing you more great content in 2012. Here’s some of the ideas we’re cooking up:
- A series on Big Data – the trend explained, and concrete ways big data can help your business outperform the competition
- 10 Ways Out of BI Chaos
- 5 Traps of Mobile BI
- 10 Pieces of Advice for Better BI Reporting
- More on the trend of BI in the cloud
- …and more
Count on arcplan’s blog to keep you informed of the latest BI trends. Have a wonderful holiday season and Happy New Year! We’ll be back in January!