We've been thinking a lot about the various ways organizations can improve their existing business intelligence applications. Many of arcplan’s customers have been with us 5-10 or more years and are continuously improving their BI along the way. Some of the initiatives we frequently hear about are related to data quality improvement, but this may be an anomaly. According to Ventana Research's recent study, less than half of organizations surveyed have taken on some kind of information management initiative, like data quality or data integration improvements, in the last 2 years due to budget restrictions or lack of employees with the right skills.
I'd argue that data quality initiatives should be a "top 5" priority for organizations in 2012. Why? Because of stories like this: A friend recently told me about a meeting at his company where the regional sales managers were giving their summaries of pipeline opportunities. During one of the updates, a director interjected that he didn’t see the favorable developments mentioned in Salesforce, their CRM system. Based on the information that was present in the system, the director figured that the quarter would be an average one. However, the updates from the sales manager would really swing the potential outcome of the quarter in a positive way. Now I bet that director had to make some decisions that were compromised by the (lack of) current information in the CRM system. He might have started strategizing about how to re-engage with the (assumed) stagnant prospects, started working with marketing on a nurturing campaign, asked the telesales team to reach out…any number of things could have happened based off of the faulty information available to him.
Unfortunately, many organizations have to contend with poor data quality which ultimately results in poor decision-making. After all, decisions are no better than the data on which they're based. Reliable, relevant, and complete data (as opposed to the incomplete data set available to the director in my example above) supports organizational efficiency and is a cornerstone of sound decision-making. So what are some of the consequences of sub-par data quality?
1) Mistrust. Poor data quality often breeds mistrust among internal departments. I read a great example from 1998 (if you can believe it) that could have been written yesterday:
Two departments, call them A and B, each needed data about parts. Their needs overlapped considerably, but each needed a few fields that the other did not. Initially the data was maintained by department A, but the quality wasn’t high enough for department B, so it developed its own database. Soon the databases became horribly inconsistent. The issue became a "lightning rod" and it became impossible for the two departments to work together.[i]
Poor data quality can also breed mistrust externally – for example between a vendor and its customers. An incorrect billing address may lead to an invoice being sent to the wrong location, the customer receiving it late and therefore appearing delinquent. If a vendor can't get even a billing address correct, how can they be expected to deliver the correct order to the customer? I know some sales people who see their CRM as an afterthought, but consider when that CRM is tied to the company's accounting system. Bad data in one place means bad data elsewhere and leads to seemingly small issues like sending an invoice to the wrong place. However, these simple acts can harm a vendor's reputation and lead to more serious consequences like decreased customer satisfaction ratings.
Check out this story about water billing errors in many cities around the US due to incorrect meter readings, equipment failures or human error – all resulting in poor data that has led to lawsuits, unpaid bills, and angry customers. Ouch.
2) Bad or delayed decisions. If you suspect you're dealing with unreliable or incomplete data, you might delay even making your decision in the first place. It becomes a confidence issue – if you don't feel confident in your data, why would you feel confident in making a decision that could come back to bite you? One way to mitigate this is to take responsibility for your data by using alerts:
Most business software applications allow users to set rules so that they can be warned when something falls above or below a certain threshold. One guy I know gets an e-mail every time a key raw material is close to being out of stock. Another guy I know gets a text when a significant customer's receivables balance goes over 90 days. Does your software do this? If not, maybe you can hire a consultant to create "triggers" in your underlying database to do the same. Most common databases, like Microsoft's popular SQL Server, have this feature available.[ii]
3) Wasted money. According to a recent study in the UK, US and France, 16.6% to 18% of departmental budgets are eaten up because of poor data quality. The research also indicates that 90% of surveyed companies admit that inaccurate data – such as duplicate accounts, lost contacts and missed sales opportunities – contributes to budget waste. On top of this, a 2009 Gartner study revealed that the average organization surveyed loses $8.2 million annually because of poor data quality and that most of this is due to lost productivity.
Compromising your company's revenue forecast, breeding mistrust internally and externally, enabling bad decisions, and wasting precious dollars are just some of the most serious consequences of poor data quality. Is there anything you personally can do to combat it? Next time, we'll discuss who is responsible for ensuring data quality. It might just be you.
[i] The Impact of Poor Data Quality on the Typical Enterprise. Communications of the ACM, 1998. Thomas C. Redman: http://ganga.iiml.ac.in/~vivekg/itm/Impact_of_Data_Quality.pdf
[ii] Don’t Sabotage Your Business With Bad Data, May 2010. Gene Marks: http://www.businessweek.com/smallbiz/content/may2010/sb20100528_479709.htm