Senior Data Engineer

Absentia Labs • San Francisco, CA

20h • $115,000 - $200,000

About The Position

Absentia Labs is building data and intelligence infrastructure for biomedical discovery, transforming fragmented scientific data into reliable, machine-learning-ready knowledge. The Senior Data Engineer will own the design and evolution of the company’s biomedical data platform, working autonomously on long-horizon architectural decisions while contributing hands-on to implementation. The role focuses on representing complex scientific data correctly and reliably, as well as building systems that operate at scale.

The engineer will design end-to-end data systems for large biomedical datasets and create schema-driven models that reconcile noisy, semi-structured, heterogeneous sources. The position establishes practices for data quality, validation, provenance, lineage, and versioning; builds cloud-native infrastructure and pipelines for batch and streaming workloads; and partners with ML engineers, scientists, and product leads to turn research needs into durable data abstractions. It also provides technical leadership through design reviews, architectural guidance, and mentorship, while addressing risks in data integrity, scalability, and operational complexity.

Requirements

• At least 5 years of experience in data engineering, platform engineering, or ML infrastructure roles, with clear ownership of production systems.
• Proven experience designing and operating large-scale, production-grade data pipelines.
• Strong proficiency in Python and data-centric software engineering practices.
• Deep experience with a cloud platform such as AWS, GCP, or Azure, including storage, compute, and security primitives.
• Familiarity with distributed data processing and orchestration systems such as Spark, Beam, Ray, Airflow, or Dagster.
• Experience supporting ML/AI workloads, including dataset generation, feature pipelines, and reproducible training workflows.
• Strong architectural judgment and ability to communicate technical decisions clearly across disciplines.
• Ability to own poorly defined problems and develop robust solutions through thoughtful design and iteration.
• Understanding of schema design, normalization, and semantics as important engineering concerns for biomedical data and AI-driven settings.

Nice To Haves

• Prior work with biomedical or life-science data, such as omics, assays, molecular representations, clinical data, or toxicology data.
• Experience with streaming platforms such as Kafka, Pub/Sub, or Kinesis.
• Exposure to ontology-aware data modeling or schema evolution in scientific domains.
• Infrastructure-as-code and systems experience with tools such as Terraform, Docker, or Kubernetes.
• Experience in early-stage startups or research-heavy environments.
• Open-source contributions or technical publications.

Responsibilities

• Architect and lead end-to-end data systems for large-scale biomedical datasets, including chemical, biological, toxicology, omics, assay, clinical, and experimental data.
• Define and evolve schema-driven data models to reconcile noisy, semi-structured, heterogeneous sources into coherent, interoperable representations.
• Establish best practices for data quality, validation, provenance, lineage, and versioning for scientific and machine-learning workflows.
• Build and maintain scalable, reliable cloud-native data infrastructure, including data lakes, warehouses, object storage, and streaming systems.
• Design pipelines supporting batch and streaming access for ML training, evaluation, and inference.
• Partner with ML engineers, scientists, and product leads to translate research needs into durable data abstractions.
• Make production-system trade-offs involving performance, cost, flexibility, and correctness.
• Provide technical leadership through design reviews, architectural guidance, and mentorship of other engineers.
• Identify and proactively address systemic risks in data integrity, scalability, and operational complexity.

Benefits

• Competitive compensation, including meaningful equity participation.
• Flexible remote or hybrid work arrangements.
• High autonomy, high trust, and ownership over critical systems.
• A deeply technical, low-ego culture focused on learning and rigor.
• Opportunity to architect the data backbone of an AI-driven biomedical platform and directly affect how scientific data is translated into machine intelligence.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed