Data Engineer (Generative AI & Healthcare Data)

Altarum

31d•$144,771 - $174,723•Remote

About The Position

Altarum is building modern data and AI infrastructure to support public health, Medicaid, analytics, evaluation, and interoperability. As a Data Engineer focused on generative AI and healthcare data, you'll independently design, build, deploy, and operate data pipelines, platform services, and production AI applications — with an emphasis on generative AI systems grounded in healthcare data. This is a mid-level individual contributor role, typically requiring 3–5 years of relevant experience. You'll own defined projects from requirements and technical design through testing, deployment, and production support. Working with principal engineers and project partners, you'll make implementation decisions within shared architecture and security standards, guide small project teams, and help colleagues build data and AI engineering skills.

Requirements

Bachelor's degree in computer science, data science, information systems, engineering, mathematics, statistics, public health informatics, or a related field, or equivalent practical experience
Typically 3–5 years of relevant experience in data engineering, AI engineering, software engineering, analytics engineering, cloud engineering, or applied technical research, including independent delivery of production systems
Strong Python and SQL skills, with experience building maintainable code, data models, automated tests, and reliable ingestion/transformation pipelines
Hands-on experience with a cloud platform and a modern data platform such as Databricks or an equivalent, including distributed processing with Spark or comparable technology
Experience using Git, CI/CD, environment-specific configuration, and automated deployment to deliver and support production workloads
Hands-on experience building and deploying generative AI applications with LangChain, LlamaIndex, or comparable frameworks, including retrieval-augmented generation or document processing, prompt design, API integration, evaluation, and monitoring
Ability to turn requirements into technical designs, manage defined work, troubleshoot independently, and guide collaborators through clear explanations and constructive feedback

Nice To Haves

Databricks Certified Generative AI Engineer Associate certification strongly preferred; comparable generative AI credentials with relevant practical experience also valued
Databricks Mosaic AI, Vector Search, Model Serving, MLflow evaluation and tracing, and Unity Catalog; experience delivering governed generative AI applications
Healthcare and Medicaid data, quality measures, regulated reporting, FHIR or HL7, or accessible document generation
Databricks workflows, Asset Bundles, Delta Lake, PySpark, dbt, orchestration, or infrastructure as code
Experience leading small technical teams, mentoring colleagues or students, teaching technical subjects, or supervising applied research

Responsibilities

Design and operate cloud data platform components, including lakehouse storage, compute, SQL and ELT engines, and reusable transformation workflows using Databricks or comparable platforms
Develop production Python and SQL code, versioned reusable packages, APIs, and configuration-driven frameworks with automated tests and documentation that help other teams adopt them
Own CI/CD for data pipelines, AI applications, and supporting services, including environment configuration, automated validation, deployment, release verification, and rollback
Implement infrastructure as code and secure deployment patterns in AWS, AWS GovCloud, Azure, Azure Government, or other approved environments
Own ingestion, transformation, and normalization pipelines for structured, semi-structured, and unstructured healthcare and public health data, including batch and streaming workloads
Build reusable connectors, data contracts, and curated datasets using Python, SQL, Spark or PySpark, Delta Lake, dbt, or comparable technologies
Integrate data such as Medicaid and Medicare claims and encounters, FHIR R4, US Core, HL7 v2, social determinants of health, geospatial data, surveys, and qualitative sources as projects require
Translate analytics, reporting, evaluation, and data science requirements into tested data models and dependable datasets with documented lineage
Define and implement data quality rules, validation, lineage, metadata, and monitoring for owned pipelines and services
Diagnose and resolve production failures, data quality issues, and performance bottlenecks; implement root-cause fixes and maintain operational runbooks
Establish reliability and freshness targets with stakeholders and own monitoring, incident response, and recovery for assigned services
Optimize compute, storage, partitioning, and pipeline execution; track cost and performance and implement guardrails
Independently own AI application delivery from requirements and solution design through implementation, evaluation, deployment, and production support, integrating models, data, retrieval, and external services into maintainable applications
Build retrieval-augmented generation and document-intelligence applications — including ingestion, chunking, embeddings, vector search, and retrieval — to produce source-grounded outputs from healthcare documents and other unstructured data
Use MLflow or comparable tools to manage generative AI evaluation and tracing, evaluation datasets, custom scorers, and prompt/model versioning; assess output quality, factual accuracy, latency, and cost before and after deployment
Develop LLM applications and agentic workflows using Python, LangChain, LlamaIndex, or comparable frameworks, with approved services such as Azure OpenAI or Azure AI Foundry
Deploy and operate AI applications and model endpoints using Databricks Model Serving or comparable services, with governed data access, monitoring, and CI/CD
Build reusable AI services and automation for healthcare reporting, document extraction, and knowledge retrieval, partnering with data scientists on predictive models when needed
Apply privacy by design, encryption, least-privilege access, secrets management, and secure networking; minimize sensitive information in logs, traces, and evaluation datasets
Implement technical controls and documentation for applicable HIPAA and 42 CFR Part 2 requirements, IRB and data-use agreement restrictions, and responsible AI practices informed by the NIST AI Risk Management Framework
Work with data science, security, and program teams to document model behavior, evaluate fairness and explainability, and maintain model cards and other governance artifacts
Serve as technical lead for defined projects or workstreams: break down work, coordinate contributors, review implementation choices, and help resolve blockers alongside principal engineers and project leads
Mentor colleagues through pairing, code reviews, and technical guidance; lead workshops and create tutorials, runbooks, and examples that help teams adopt reusable data and AI tools
Explain technical concepts and tradeoffs to varied audiences, and contribute technical materials for proposals and client engagements