The General Data Protection Regulation (GDPR) that will come into effect on May 25th 2018 has strong implications for nearly each and every company and organization in Europe. Its principle of privacy by design , that was first postulated by Canadian data protection scientist Ann Cavoukian could lead to a paradigm shift in how businesses develop their marketing campaigns and their customer service.
Many of the articles of the GDPR show the importance of data quality, especially Article 5 (Principles relating to processing of personal data) and Article 16 (Right to rectification). But it is obvious that also other parts of the GDPR demand a high level of quality of data, especially duplicates should be avoided in order to fulfill the data subject’s rights like the Right of access or Right to object properly.
The problem with this insight is that many businesses are struggling with their data quality. Studies and surveys show that a majority of companies is not satisfied with the data quality in their databases and think that it needs improvement.
But what are possible measures to improve data quality?
Technical and organizational measures
In the good old days ” there were dedicated employees called data entry clerks . They were trained to focus on data quality, e.g. properly formatting phone numbers, checking the writing of addresses, figuring out the gender for a first name, etc.; they provided a constant high level of data quality.
Of course entering data by a data entry clerk is both expensive and time-consuming. Many businesses nowadays rely on efficient and fast online processes where the customers enter data on their own. However, only very few customers are data quality experts so they don’t care about using the appropriate fields or about the writing of their address. However, they will complain later on, when a mailing does not reach them because of errors in the address they entered.
Another common process is to fill out paper forms and have them OCR scanned. Although OCR scanning made astounding progress in the past years it is far from perfect when recognizing handwritten characters.
In both scenarios data of bad quality is loaded into CRM systems, marketing automation systems, etc.
As organisational measures (data entry clerks) are expensive and take too much time technical measures need to be evaluated. When data could be checked, corrected and even enriched with additional information during the data entry phase the users could get immediate feedback on errors they made during input. With the advent of business web services, many microservices were published to take care of address checks, e-mail checks, etc.
However, they often have different access protocols (REST, SOAP, RPC, ), different accounting models or have substantial initial or running costs (e.g. by only offering plans for volumes that many organisations don’t need). Only a small handful of API providers (e.g. /data.mill) took the work-intensive way to aggregate a number of useful data services into one platform thus also standardizing the technical access protocol and providing a consistent accounting model.
Upgrade systems to use APIs for data quality
In order to comply with the data quality related articles of the GDPR companies should evaluate the integration of APIs like /data.mill into their web forms, CRM systems, etc.; by doing this data quality can be sustained on a high level and data protection authorities cannot criticize this part of data processing. As the GDPR affects so many other processes in an organization each and every help should be accepted and appreciated.