Databricks Engineer & AI Enablement Architect - Data Engineer III
About The Position
As a Data Engineer III on Deloitte’s AI & Engineering team, you will lead the governance, architecture, reliability, and operation of secure, scalable Databricks environments supporting the Data Aggregation Contract (CDAC). You will establish workspace governance, Unity Catalog structures, access controls, service principals, and policies to protect protected health information (PHI) and other sensitive healthcare data. You will define medallion architecture standards and develop reusable ingestion and transformation frameworks for CMS claims, clinical, provider, quality, and oncology data. You will improve data quality and production reliability through validation, reconciliation, lineage, observability, service-level agreement monitoring, dashboards, and proactive alerts. You will also establish CI/CD standards, review code, data models, and pipeline designs, and assess GenAI and LLM use cases for production readiness. This role is part of Deloitte’s Government & Public Services (GPS) practice and AI & Data offering, which serve government and public higher education clients with data, AI, and analytics platforms, products, insights, and services. The role is available in Baltimore, Maryland; Rosslyn, Virginia; and Washington, District of Columbia. Average travel is 10–15%, based on the work and clients served.
Requirements
- Bachelor’s degree.
- Ability to obtain and maintain the required clearance for this role.
- At least 4 years of Databricks engineering experience.
- At least 4 years of experience with Python, PySpark, SQL, Delta Lake, and Medallion Architecture.
- At least 2 years of experience in Databricks architecture.
- Ability to travel 10–15% on average, based on work and clients served.
- Legally authorized to work in the United States without employer sponsorship now or in the future.
- Ability to work independently and collaboratively as part of a team.
- Effective written and verbal communication skills.
- Meticulous attention to detail and quality of work product.
- Ability to build and sustain professional relationships.
- Ability to manage and prioritize multiple tasks in a fast-paced, dynamic environment.
- Strong interpersonal skills and professional demeanor.
- Ability to meet deadlines and provide clear guidance to others.
Nice To Haves
- Experience supporting federal healthcare clients or working with CMS claims, Medicaid or Medicare claims, clinical and provider data, quality measures, or longitudinal patient data.
- Databricks Certified Data Engineer Professional, Data Analyst Associate, Machine Learning Professional, or Generative AI Engineer Associate certification.
- Experience administering Databricks platforms, including Unity Catalog, workspace administration, cluster policies, service principals, secrets management, and role-based access controls.
- Experience implementing Databricks Asset Bundles and Git-based CI/CD across development, test, and production environments.
Responsibilities
- Lead governance, architecture, reliability, and operation of secure, scalable Databricks environments supporting CDAC.
- Establish workspace governance, Unity Catalog structures, access controls, service principals, and policies to protect PHI and sensitive healthcare data.
- Define medallion architecture standards and build reusable ingestion and transformation frameworks for CMS claims, clinical, provider, quality, and oncology data.
- Improve data quality and production reliability through validation, reconciliation, lineage, observability, SLA monitoring, dashboards, and proactive alerts.
- Establish CI/CD standards; review code, data models, and pipeline designs; and assess GenAI and LLM use cases for production readiness.
- Lead projects or workstreams and provide clear guidance to others.
- Manage and prioritize multiple tasks in a fast-paced environment while meeting deadlines.
Benefits
- Discretionary annual incentive program, subject to program rules and applicable factors.
- Broad range of employee benefits.
- Opportunities for professional development, including on-the-job learning, formal development programs, mentorship, and leadership opportunities.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
