When a contractor fills your underfloor heating system with untreated water, bacteria begin to multiply inside the pipes. The system runs perfectly. The floor is warm. Everyone goes home satisfied.
Six months later, the contamination has spread through the entire circuit. The heating fails, the system needs to be flushed, disinfected, and partially replaced. And the contractor? You have to track them down.
Nobody questions what happens next: they come back, they fix it, and they pay for it. Because you have a contract, you have standards, and you have the right to demand the work was done correctly — and to demand proof that it was done correctly in the first place. A simple water quality test at handover would have shown you exactly what was flowing through your system before the damage was done.
So why, when a vendor delivers your new Data & Analytics Solution or migration project, do you not ask for the equivalent? A data quality test that shows you exactly what is flowing through your pipelines — before the damage reaches your reports, your decisions, and your bottom line.
The Question You Should Be Asking Every Vendor
You’ve just paid for a new Data Warehouse. Or a migration project. Or an integration between two critical systems. The project is done. The invoice is paid. Everyone congratulates each other.
But here is a question almost no CEO asks in that final meeting:
Where is the test documentation?
Not “did you test it?” — anyone can say yes. Where is the evidence? Where are the results? Which checks were run, against which data, and what were the outcomes? If your vendor cannot hand you a test report at project delivery, you have not received what you paid for. You have received a system that works until it doesn’t, with no way to know when that will be or why.
Would you accept a building without an electrical inspection certificate? Would you commission a new facility and skip the sign-off that the wiring is safe? Of course not — because the certificate is not a formality. It is proof that someone ran the checks, took responsibility, and is accountable if something fails.
Data is no different. The tests are your certificate.
Someone Has to Be Accountable. Make It Contractual.
Here is what C-level conversations about data quality usually look like: a problem surfaces, the blame diffuses across three teams and two vendors, and the cost of fixing it lands in next quarter’s budget as an “operational issue.”
That is not acceptable, and it does not have to be that way.
If a vendor delivers a pipeline or a migration project without documented data quality tests, and errors emerge downstream — in your reports, your AI models, your regulatory submissions — the remediation cost belongs to them. Not to your operations team. Not to your IT budget. To them.
This is not a technical conversation. It is a commercial one. Build it into your contracts. Require test evidence as a condition of final payment. Treat unvalidated data delivery the same way you would treat a building that failed inspection — you do not sign it off, and you do not pay the final installment until the certificate is in your hand.
Control Is Simpler Than You Think
You do not need to understand data engineering to hold vendors accountable for data quality. You need to ask three questions:
What tests were run on this data?
What were the results?
Who is responsible if this breaks?
If the room goes quiet, you have your answer.
Modern data observability tools exist precisely to make these answers visible — not just at delivery, but continuously. Not buried in a technical report, but surfaced in a dashboard a CEO can glance at in sixty seconds. Green means the data is healthy. Red means someone has a problem to fix — and you know exactly who.
A One-Time Test Is Not Enough.
Every Change Must Be Verified Automatically.
Here is a mistake even well-run organizations make: they demand a test report at project delivery, and then they stop. The certificate is filed, the vendor leaves, and the next change — a schema update, a new data source, a pipeline modification — goes out without any validation at all.
Think again about the heating system. You tested the water at handover. Clean result. But six months later, a technician adds a new radiator, adjusts the pressure, and nobody tests again. The bacteria are back — and nobody knows.
Data pipelines are live systems. They change constantly — sometimes daily. Every change is a new risk. Every new data source is a potential contaminant. A test run once at delivery tells you the system was clean on that one day. It tells you nothing about today.
This is why the standard you should demand is not a one-time inspection. It is continuous, automated monitoring. Every time something changes in the pipeline — automatically, without human intervention — the tests run. If something fails, the right people are notified immediately. Not in two weeks when the damage has already reached your board report.
You would not accept a smoke detector that only works on the day it was installed.
Do not accept data quality checks that work the same way.
The Tolerance Problem
Every time a CEO accepts a project handover without test evidence, a signal goes out to the entire vendor ecosystem: this organization does not require proof. Data quality corners can be cut here.
The organizations that are outperforming their peers on AI adoption, on reporting accuracy, on regulatory compliance — they are not necessarily working with better vendors. They are working with vendors who know they will be held accountable.
Demand the inspection certificate. It already exists in every other engineering discipline. Your data infrastructure deserves the same standard.
About the Author
I help organizations build trust in their data by making automated data testing a natural part of every project. At CAT, I work closely with corporate data leaders and our partners to ensure strategies, platforms, and teams rely on reliable, validated data – whether in banking, insurance, e-commerce, AI initiatives or F1 team. With more than 15 years of experience in the data field, I combine technical expertise with business insight to guide clients, mentor teams, and contribute actively to the data quality community.
