“My data is correct! Isn’t it?”

Share it with the world!

Works for me

How often happened to you, to a data engineer or data tester, that you just simply couldn’t find the root cause of a data issue? Everything look alright, but whatever testing tool you use, simply shows there’s an error. And suddenly, few hours later, voila here it is! What a relief then. Let us share with you one more story from our life with CAT.

My data can't be wrong

“CAT indicates our data is wrong, but it actually isn’t! Non-English characters are handled in a wrong way – it cripples letters with accents on one side of the comparison. What code page does CAT use?”
CAT Support:
“CAT just simply runs your queries and compares the results, are you sure your data is OK?”
“The data is correct, something is wrong with CAT, we need to fix it ASAP… or, wait a minute!”

OK, my data ... is wrong

The case turned out the non-English characters got lost between two stages of customer’s DWH load process. So “Väinö Mäkelä” became “Vaino Makela” somewhere on data processing path. Luckily CAT caught that thanks to a very simple test. No bug in the testing tool, rather in the code it was testing 🙂 But that’s what testers life is about ever since – when hunting for bugs, nothing should be trusted.
It is astonishing how much you can check when comparing results of two very simple SELECT statements:
- Test name: We have all customers in DWH
  Description: Compares customers from CRM with dimension Customer in DWH
  First data source: CRM
  First Query: SELECT id, name FROM cusstomer ORDER BY name, id
  Second data source: DWH
  Second Query: SELECT DISTINCT CustomerID, CustomerName FROM Dim.Customer ORDER BY CustomerName, CustomerID
  Expectation: sets match
  • The data in the DWH is fresh – it contains latest customers.
  • The data is complete, the incremental load works and nothing was missed when the load was down for two days.
  • The data is correct, no problems with non-English characters.
  • Sorting works as expected, so collation is OK.

That’s a lot of feedback, considering you just compared two result sets, isn’t it? 

Will you share your stories?

We’d love to hear from you – would you share any funny moments from testing? Please share in comments on our LinkedIn profile.
Thanks and happy testing!

Installing CAT

Install CAT

Click to copy the command and paste it into your PowerShell 7 terminal!

Direct Download

Download CAT directly from PowerShell Gallery and give it a try!