Senior Data Platform Engineer

Komodo Health•San Francisco, CA

9h•Remote

About The Position

Komodo Health’s Data Foundations team builds core data products that power its Healthcare Map and related customer-facing products, internal innovation, and AI/ML use cases. The Senior Data Platform Engineer designs, builds, operates, and improves large-scale pipelines and foundational data systems, processing complex healthcare datasets with a focus on reliability, performance, scalability, and efficiency. The role involves transforming healthcare claims, EHR, and reference data into trusted, serving-ready products; making storage and data-layout decisions; and maintaining pipelines that ingest data from many external sources despite frequent schema changes. The engineer also contributes to system design and technical direction, production support, and collaboration across Data Product and Engineering teams. AI-assisted engineering tools are expected to be part of daily work.

Requirements

Strong hands-on experience building, owning, and debugging production data pipelines at multi-terabyte scale.
Advanced Python skills, including writing performant, memory-efficient code for large-scale data processing.
Strong SQL skills.
Deep production experience with Airflow, including operating and scaling it.
Understanding of physical data storage, including columnar versus row formats and open table formats such as Iceberg, Delta Lake, or Hudi, including updates, compaction, small files, and partitioning.
Experience with Spark or a comparable distributed processing framework.
Proven experience designing and operating data systems in AWS.
Strong instincts for data correctness, root-cause analysis, and production troubleshooting.
Ability to communicate technical trade-offs clearly with engineering, product, and data partners.
Comfort using AI-assisted engineering tools for coding, debugging, documentation, and technical exploration.

Nice To Haves

Experience with containers and orchestration, such as Docker and Kubernetes.
Strong core software engineering fundamentals and experience in more than one language.
Experience with data governance and lineage.
Experience delivering external-facing data products through APIs, serving layers, or customer access patterns.
Experience applying AI or agentic workflows to engineering, data quality, delivery, or operations.
Success in high-growth or ambiguous environments.

Responsibilities

Build, operate, and optimize large-scale batch pipelines using Python, SQL, Spark, and Airflow on AWS.
Make storage and data-layout decisions, including table formats, partitioning, file sizing, and compaction, to improve query performance and cost efficiency.
Ingest data from external sources and maintain pipeline correctness as schemas change.
Transform healthcare claims, EHR, and reference data into trusted, serving-ready data products for customers and internal teams.
Improve reliability through data quality checks, validation, lineage, observability, and alerting.
Debug complex data, system, and performance issues across compute-heavy workflows.
Contribute to system design, code quality, testing, documentation, and CI/CD.
Participate in rotational production support.
Partner with Data Product and Engineering teams to deliver scalable, production-grade data solutions.
Contribute to technical initiatives that improve pipeline performance, scalability, reliability, observability, and cost efficiency.
Use AI-augmented engineering tools for tasks such as coding, debugging, documentation, test creation, and technical exploration.
Support the team through mentorship, design reviews, and engineering best practices.

Benefits

Comprehensive health insurance.
Dental insurance.
Vision insurance.
Flexible time off and holidays.
401(k) with company match.
Disability insurance.
Life insurance.
Leaves of absence in accordance with applicable laws, regulations, and company policy.
Potential eligibility for performance-based bonuses.
Potential eligibility for equity awards.