According to industry forecasts, the world's volume of data doubles every 18 months, and all forms of enterprise data will grow 650% over the next five years. The talk around big data is more than marveling at the mass of information we’re creating. As analysts and data scientists, we’re trying to find the good stuff – the trends, the data that allows us to make better decisions now and in the future, to predict the moves that will make our business more successful down the line.
Big data (explained in our previous article here) might be new to you, but I've seen some analyst reports referencing big data ideas as far back as 2001. However, the BI world is talking about it more and more as data volumes grow and we begin to see the potential knowledge to be gained in these data sets.
So maybe you're thinking, how can big data benefit my company? It's hard to think conceptually about it, so let's take a look at some concrete examples of how companies are using big data today. We'll start with the retail industry. Keep in mind that many of these ideas can be used on a smaller scale for retailers of any size.
Wal-Mart sifts through massive amounts of unstructured social media and search data to find out what products consumers are talking about. They use that information to set their ad buying strategy on sites like Google, with the goal of competing for e-commerce sales – currently dominated by Amazon.com. Wal-Mart actually uses free software called Hadoop, which was created by a group of Yahoo developers to analyze raw information better than traditional databases. Hadoop requires a lot of integration and expertise, which is currently in short supply, but expect to hear more about it throughout 2012 and beyond. Wal-Mart also keeps track of products, sales, and customers to such a degree that it uses those petabytes of data to win pricing concessions from suppliers.
If you're a retailer looking to increase profitability, a recent report from the McKinsey Global Institute states that tapping into big data is the way to do it. By embracing big data analysis, US retailers can increase operating margin by up to 60% (a measurement of what proportion of a company's revenue is left over after paying for variable costs of production such as wages, raw materials, etc.). As half of all sales in the US will be done over the web (or at least will be influenced by the web) by 2013, it’s an especially important time for retailers to build their big data capabilities.
So what exactly can big data improve for retailers?
- Marketing strategy (segmenting customers into groups based on extremely granular things, like baby bottle purchases, and marketing to them on a personal level)
- Store layouts (determining which products perform best on a shelf vs. an endcap in various cities)
- Product pricing optimization (figuring out which goods to place on sale at particular times and in particular regions, or which goods are most affected by price fluctuations)
- Labor cost optimization (with store labor accounting for as much as a third of a retailer's fixed costs, managing the mix of customer service vs. amount of staff is imperative)
See the image below for more ideas from McKinsey's report.
I've worked with retailers much smaller than Wal-Mart to implement some of these data analysis strategies, and I can vouch that you don't necessarily need "big data" in order to achieve a lot of the optimization listed above. What you do need are great analysts who are able to influence management and an organization that adapts quickly to changing conditions.
Are you a large retailer that's riding the big data wave or a small chain using big data ideas to improve your business? I'd love to hear about your experiences.