New TDWI assessment examines today’s data quality maturity
Dr. Fern Halper, TDWI’s Vice President and Senior Research Director of Advanced Analytics, discusses new TDWI tools for assessing modern data quality and maturity.
In this “Speaking of Data” podcast, Dr. Fern Halper talks about the factors influencing data quality today and introduces new tools for organizations to assess the maturity of their implementations. Halper is vice president and senior research director of advanced analytics at TDWI. [Editor’s note: Speaker quotations have been edited for length and clarity.]
“With data becoming such a critical part of a company’s competitiveness, it’s no wonder data quality is becoming increasingly important,” Halper began. “Organizations need better, faster insights to succeed, and that requires better, richer datasets for advanced analytics such as predictive analytics and machine learning. .”
To do this, she explains, organizations are not only increasing the amount of traditional structured data they collect, but also looking for new data types, such as unstructured text data and semi-structured data from websites. Did. Bringing these different types of data together can greatly increase the opportunity for insight, she added.
As an example, Halper mentioned the idea of organizations using notes from call centers (usually unstructured or semi-structured text data) to analyze customer satisfaction for specific products or for the company as a whole. . This information can be fed back into analytics or machine learning routines to uncover patterns and other insights that are meaningful to your company.
“Regardless of data type or end use, the original data must be of high quality. It must be accurate, complete, timely, reliable, and fit for purpose.”
One problem with this, she noted, is that the concept of what constitutes “high quality” is not always so clear-cut.
“For example, what is ‘high quality image data’?” she said. “What is high quality for unstructured text?” “These are questions that most organizations have not yet thought about, given that the types of data involved are so new.” This is done in addition to the fact that it has never actually been resolved.
She said an important reason for releasing a data quality maturity assessment is to help organizations assess where they really stand. For example, in a recent TDWI survey, less than 50% of respondents said they were satisfied with the quality of their data.
“Data quality impacts many other aspects of an organization’s well-being,” Halper says. “For example, this is an important part of data governance; our research shows that it is often at the top of data management priority lists.”
She went on to further elaborate on the five dimensions of the Data Quality Maturity Assessment Model.
- Organizational efforts: Is the strategy defined throughout the organization? Is it conscious? Do you have the funding to maintain data quality practices?
- Roles and responsibilities: Is there someone responsible for data quality? Is there training?
- Data quality control: What tools and processes are in place to ensure data is accurate, reliable, relevant, timely, etc.? To expose and remediate poor quality data What processes are in place?
- Warranty and impact: How well are your data quality processes working? Does your organization actually measure data quality?
- tool: What tools are in place? How advanced are they? Are you using automation or enhancements to help manage tasks related to data quality?
“It also comes with a guide that helps you understand the assessment questions and provides tips on how to improve various aspects of your model,” she said.
Halper added, “Another thing to note is that we plan to publish a comprehensive ‘Data Quality Maturity’ report next quarter that incorporates our assessment results and many other sources of information. It is,” he added. ”
To learn more about the TDWI Maturity Model and Assessment, please visit TDWI.org.
