Description
The project migrates data and reporting from a legacy estate to a modern platform built on Databricks and Microsoft Fabric. Quality engineering ensures that migrated data reconciles with the source, pipelines behave as specified, and reports produce consistent results.
Responsibilities
•Design and implement data testing and quality assurance frameworks
•Build automated validation for data pipelines and data products
•Own source-to-target reconciliation between legacy systems and the new platform
•Build repeatable regression packs that run unattended
•Write unit tests that run automatically within CI/CD, with quality gates that can block a release
•Detect and handle schema drift and unexpected structural changes
•Validate reports, dashboards, and semantic models, including measure definitions, filter behaviour, and totals
•Apply AI-assisted techniques to test generation and coverage analysis, with human review
Requirements
•5+ years of experience in quality engineering, including meaningful recent experience testing data or ETL processes rather than applications
•Strong Python and SQL skills used to build test automation, not only run it
•Experience with pipeline validation, including row counts, control totals, referential integrity, and business rule assertions
•Experience with source-to-target reconciliation using defined tolerances
•Experience with test automation within CI/CD, including gates that block a release
•Experience with Azure DevOps
•Experience with SQL Server
•English proficiency suitable for direct client conversations
Nice to have
•Experience with Azure Databricks or Microsoft Fabric
•Data quality frameworks such as Great Expectations, dbt tests, Databricks Expectations, DLT quality rules, Soda, or Deequ
•Designing a test framework from scratch
•Testing semantic models and BI layers
•Working in a regulated financial services environment
•Application test automation alongside data testing using Selenium, C#, SpecFlow, or Playwright
Conditions
No conditions stated in the posting.
