Data Quality Engineer

i4DM•Millersville, MD

About The Position

The Data Quality Engineer will support ATLAS, the VA’s open-architected, multi-cloud enterprise data management and analytics platform. The role designs and automates validation, testing, profiling, and monitoring frameworks to ensure the accuracy, consistency, completeness, and reliability of multi-source data as it is ingested, conditioned, curated, and delivered as enterprise data products. The engineer uses Azure Synapse, Databricks, Collibra Data Quality, Profisee, SQL, Python, and related tools to detect anomalies, enforce governance standards, and improve trust in data across interoperating ATLAS platforms. The role collaborates with data engineers, analysts, data scientists, and product teams to provide consistent, secure access to high-quality data for evidence-based decisions and improved outcomes for Veterans and their families.

Requirements

At least 3 years of experience working in a cloud-based environment.
Experience analyzing and profiling data.
Proficiency querying and working with data using SQL and Python.
Experience building automated testing frameworks for data quality assurance.
Hands-on experience with Azure data engineering and ELT tools, including Azure Synapse and Databricks.
Strong knowledge of best testing practices and patterns in cloud environments.
Ability to work independently within a mid-sized team and rapidly develop test plans using JIRA’s X-RAY.
Proficiency implementing automated test plans using Collibra Data Quality, Microsoft Profisee, or script-based testing with Python or Scala.
Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
Bachelor’s degree or at least 4 years of relevant experience in lieu of a degree.
Experience with Synapse Pipelines, Azure Data Factory, Starburst, Azure Data Lake Service, Azure ML/AI services, and Pyramid Analytics or Power BI for visualization.

Nice To Haves

Familiarity with JIRA and Confluence for project tracking and documentation.
Ability to integrate testing protocols with monitoring applications such as Dynatrace, Splunk, or Log Analytics.
Bachelor’s degree in Computer Science or a related field.
Azure Data Engineer Certification.

Responsibilities

Design and implement data validation and quality assurance processes for cloud-based environments.
Automate data quality checks and develop monitoring solutions to detect data anomalies.
Collaborate with data engineers and analysts to ensure data governance and compliance.
Analyze and profile structured and unstructured datasets for accuracy and consistency.
Build and maintain automated testing frameworks to proactively catch data issues.
Utilize Azure data engineering and ELT tools, including Azure Synapse and Databricks.
Create and execute comprehensive test plans using tools such as JIRA’s X-RAY.
Implement and execute test automation using commercial off-the-shelf tools such as Collibra Data Quality and Microsoft Profisee, or custom scripts in Python or Scala.