Big Data FAQs – 5 Questions & Answers from arcplan

Berwyn, PA / Langenfeld, Germany March 21, 2012 – The big data trend promises that harnessing the wealth and volume of information in your enterprise leads to better customer insight, operational efficiency, and competitive advantage. The marketing hype around big data and the pace of studies, analyst reports, and articles on the subject can be mind-numbing for companies that want to take advantage of big data analytics but do not know how to separate fact from fiction and determine real use cases for their business. Global BI innovator arcplan, whose platform supports big data reporting and visualization, presents answers to the most frequently asked questions about big data.

1) What exactly is big data?
"Big data" is an all-inclusive term used to describe vast amounts of information. In contrast to traditional structured data which is typically stored in a relational database, big data varies in terms of volume, velocity, and variety. Big data is characteristically generated in large volumes – on the order of terabytes or exabytes of data (starts with 1 and has 18 zeros after it, or 1 million terabytes) per individual data set. Big data is also generated with high velocity – it is collected at frequent intervals – which makes it difficult to analyze (though analyzing it rapidly makes it more valuable). Additionally, big data is usually not nicely packaged in a spreadsheet or even a multidimensional database and often includes unstructured, qualitative information as well.

2) Is it a new trend?
Not exactly. Though there is a lot of buzz around the topic, big data has been around a long time. Think back to when you first heard of scientific researchers using supercomputers to analyze massive amounts of data. The difference now is that big data is accessible to regular BI users and is applicable to the enterprise. The reason it is gaining traction is because there are more public use cases about companies getting real value from big data (like Walmart analyzing real-time social media data for trends, then using that information to guide online ad purchases). Though big data adoption is limited right now, IDC determined that the big data technology and services market was worth $3.2B USD in 2010 and is going to skyrocket to $16.9B by 2015.

3) Where does big data come from?
Big data is often boiled down to a few varieties including social data, machine data, and transactional data. Social media data is providing remarkable insights to companies on consumer behavior and sentiment that can be integrated with CRM data for analysis, with 230 million tweets posted on Twitter per day, 2.7 billion Likes and comments added to Facebook every day, and 60 hours of video uploaded to YouTube every minute (this is what we mean by velocity of data). Machine data consists of information generated from industrial equipment, real-time data from sensors that track parts and monitor machinery (often also called the Internet of Things), and even web logs that track user behavior online. At arcplan client CERN, the largest particle physics research center in the world, the Large Hadron Collider (LHC) generates 40 terabytes of data every second during experiments. Regarding transactional data, large retailers and even B2B companies can generate multitudes of data on a regular basis considering that their transactions consist of one or many items, product IDs, prices, payment information, manufacturer and distributor data, and much more. Major retailers like Amazon.com, which posted $10B in sales in Q3 2011, and restaurants like US pizza chain Domino’s, which serves over 1 million customers per day, are generating petabytes of transactional big data. The thing to note is that big data can resemble traditional structured data or unstructured, high frequency information.

4) What tools do I need to analyze it?
Another reason big data is starting to go mainstream is the fact the tools to analyze it are becoming more accessible. For decades, arcplan partners Teradata (NYSE: TDC), IBM (NYSE: IBM), and Oracle (NasdaqGS: ORCL) have provided thousands of companies with terabyte scale data warehouses, but there is a new trend of big data being stored across multiple servers that can handle unstructured data and scale easily. This is due to the increasing use of open source technologies like Hadoop, a framework for distributing data processing across multiple nodes, which allows for fast data loading and real-time analytic capabilities. In effect, Hadoop allows the analysis to occur where the data resides, but it does require specific skills and is not an easy technology to adopt. Analytic platforms like arcplan, which connects to Teradata and SAP HANA, SAP’s (NYSE: SAP) big data appliance, allow data analysis and visualization on big data sets. So in order to make use of big data, companies may need to implement new technologies, but some traditional BI solutions can make the move with you. Big data is simply a new data challenge that requires leveraging existing systems in a different way.

5) Where is the big data trend going?
Eventually the big data hype will wear off, but studies show that big data adoption will continue to grow. With a projected $16.9B market by 2015 (Wikibon goes even further to say $50B by 2017), it is clear that big data is here to stay. However, the big data talent pool is lagging behind and will need to catch up to the pace of the market. McKinsey & Company estimated in May 2011 that by 2018, the US alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.

The emergence of big data analytics has permanently altered many businesses’ way of looking at data. Big data can take companies down a long road of staff, technology, and data storage augmentation, but the payoff – rapid insight into never-before-examined data – can be huge. As more use cases come to light over the coming years and technologies mature, big data will undoubtedly reach critical mass and will no longer be labeled a trend. Soon it will simply be another mechanism in the BI ecosystem.

About arcplan
arcplan is a leader in innovative Business Intelligence, Dashboard, Corporate Performance and Planning software solutions for desktop and mobile use.  Since 1993, arcplan has enabled more than 3,000 customers worldwide to leverage their existing infrastructure for better decision making. Empowering all users to connect and collaborate with relevant information is crucial for improving business performance. With arcplan – it simply works.

arcplan's flagship product arcplan Enterprise® was rated the #1 3rd party tool for SAP BW, Oracle Essbase, and IBM Cognos TM1 in The BI Survey 10 (2011).

Press Contacts

EMEA Contact
Markus Gißke
Marketing Manager

phone: +49 (02173) 1676 159 
markus.gisske@arcplan.com
U.S. Contact 
Heather Smith
Director of Marketing
 
phone: +1 610 902 0688 x538
heather.smith@arcplan.com