The term Data Visualization loosely refers to the techniques used to communicate data or information by creating visual objects that are contained in graphics. The end goal is to communicate information clearly and efficiently to users via the information graphics selected, such as tables and charts. In his 1983 book “The Visual Display of Quantitative Information”, Professor Edward Tufte defines ‘graphical displays’ and principles for effective graphical display in the following passage: “Excellence in statistical graphics consists of complex ideas communicated with clarity, precision and efficiency. Graphical displays should: Continue reading this post>>
The end of each year warrants a look at the key trends that may take hold in the coming months. Mobility, cloud computing, bring-your-own-device (BYOD), and big data topics are clearly influencing the business intelligence industry. However, major market players such as Amazon, Apple, Google, Samsung and others are producing the technology to make these developments possible. The question becomes: are the trends paving the way for these companies to produce the technology, or is the existence of the technology influencing the trends? I believe the latter – that Amazon, Apple, Google and Samsung are the true pioneers of the BI industry.
In 2014, we’re not going to see much in the way of wholly new trends, even though some experts are already forecasting more innovations. Gartner, for example, has singled out voice-controlled BI as a hot topic. However, the idea of being able to control BI processes by voice instead of through user interfaces is more of a fantasy than a realistic business requirement. Instead, the focus for 2014 will be on developments that are already possible and how to integrate these into the corporate IT environment. Big data for one has been a subject of discussion for years. In 2014, there will be more talk about the benefits and value provided by the increasingly larger amounts of data. Primarily though, it is the underlying technical innovations created by industry giants such as Amazon and Apple that are driving the development of BI and the requirements of the market.
Mobile BI Thanks to Tablets and Smartphones
Sixty-four percent of organizations are already investing in or plan to invest in big data soon according to a recent Gartner report.* That equates to a huge number of individuals who now have to research how to embark on a big data deployment. The prevalence and benefits of big data analytics are undeniable, but there are some considerations to keep in mind before jumping in:
1) Identify a specific business need
Big data projects reap the most benefits when they address specific business needs. Having a use case in mind will help determine what data you need to analyze – social, machine or transactional data. Gartner recommends researching use cases and success stories in other industries; why not get inspired by what’s worked for others? Gartner analyst Doug Laney recently shared examples of big data at work in various industries: using big data analytics, the department store Macy’s was able to adjust prices in near real time for 73 million items based on demand and inventory; Wal-Mart was able to optimize search results and increase web checkouts by 10 – 15%; and American Express used sophisticated predictive models to analyze historical transactions and forecast potential churn. Once you’ve identified the analytic need not met by “small” data analysis, you have the first green light for considering big data technology.
Unearthing previously unimaginable insights from massive data sets is the premise of all the big data hype. Over the past few years as more and more stories come out about how companies are finding competitive advantages in their data, big data has moved beyond the buzz. Enterprises are deploying big data projects at a faster rate every year, and even more plan to do so within the next 2 years.
The extent to which a company can take advantage of big data analysis is determined by the amount of resources and infrastructure it has available. The good news is that now the barriers to entry have been lowered, making it possible for more organizations to meet their goals to transform operations with insights gained from big data. Here are three approaches that companies of any size can take based on their particular situation.
One thing to note is that these are underlying infrastructure approaches, and that you’ll still need an analytic engine like arcplan on top in order to interact with, visualize and distribute your insights.
Lots of resources and lots of infrastructure
Before big data was “big data,” Teradata was the only game in town. They’ve been at it for so long and their functionality is so robust – some of their capabilities are second to none. Now other vendors like SAP (with HANA) and Kognitio have their own massively parallel analytic databases. They offer robust processing and querying power on multiple machines simultaneously, enable near real-time MDX (Multidimensional Expressions, for OLAP querying) and SQL (Structured Query Language, the standard way to ask a database a question) queries, and in the case of SAP HANA and Kognitio, are fully in-memory. Not surprisingly, Teradata and SAP HANA come at a high price, but for that high price, the insights you achieve can be very near the speed of thought.
Why It’s a Bad Idea to Build a Business Intelligence Platform From Scratch
A friend of mine is a Python developer for a billion-dollar corporation. His team is building a custom call center reporting app that connects to the company’s cloud data storage via APIs. I’ve seen some of the application and while it’s impressive for a custom system, it’s mostly tables of numbers with the occasional pie chart. This is after 8 months of work, and the only people accessing the system are a select few big data scientists.
Believe it or not, a number of companies are doing this kind of in-house development of analytical platforms. All the hype surrounding big data has them convinced that they’re missing out on the action. Consequently, companies large and small are devoting huge amounts of time, money and human resources to developing custom business intelligence systems for big data (Google BigQuery, Hadoop, etc.) reporting rather than simply choosing a platform that already exists and is proven to work in a similar environment.
At the heart of this trend is a desire for big data to have a greater impact in the organization. Since it’s usually small teams of data scientists who are dealing with big data, their impact and effectiveness is equivalent to a small drop in a much larger body of water – their ripple effect throughout the organization is often minimal and short-lived. To extend the reach of big data in the company and get important insights out to a greater number of decision makers, a BI platform is a necessary next step – one that leverages big data insights in easily-digestible executive reports and dashboards.
Some companies are going down the road of custom BI platform development, but their efforts are no match for solutions like arcplan that are already available. Below is a list of what you’d need to do to build a BI platform from scratch. You’ll quickly see why the effort and expense aren’t worthwhile.