Principal Backend Engineer, Platform & Data Ingestion

Verana Health•San Francisco, CA

6d•Hybrid

About The Position

Verana Health is seeking a Senior Backend Engineer to join their Platform & Data Ingestion team. This role involves building and optimizing high-throughput systems that transform complex, unstructured clinical text into scalable oncology data models. The position is critical in connecting the data ingestion engine with the platform architecture, directly impacting patient care by structuring medical documentation for research and clinical outcomes. The role encourages the use of modern AI coding assistants to streamline development. The engineer will be responsible for scaling OCR pipelines, optimizing OpenSearch for data accessibility, and expanding the core oncology data model.

Requirements

4+ years of professional experience building scalable, robust APIs, microservices, and high-volume ETL/ingestion processes.
Strong professional experience with Functional Programming languages (Scala and Elixir) and/or experience in mainstream typed languages (Java, C#, or C++).
Demonstrated experience working with OpenSearch (or Elasticsearch) for complex indexing, alongside exposure to OCR pipelines, document parsing, or natural language text processing.
4+ years of experience with data modeling techniques for structured/unstructured datasets, strong SQL coding, and query optimization skills.
Proven ability to design for and navigate both transactional (OLTP via Postgres, MySQL, etc.) and analytical (OLAP via Databricks, Snowflake, BigQuery, etc.) data systems.
Comfort using coding assistants, automated scripts, or agentic tools to streamline your daily development workflow and cut down on repetitive tasks.
Deep understanding of distributed server-side architectures, reactive principles, and cloud integrations (AWS and/or GCP), including messaging/streaming infrastructure (e.g., Amazon SQS, Kafka, and Google PubSub).
Exceptional written and verbal communication skills, with a proven ability to collaborate cross-functionally across multiple engineering and product teams.
Must be legally authorized to work in the United States.
Applicants must live within 50 miles of either the Bay Area or NYC office hubs.
Must be authorized to work for any employer in the U.S.

Nice To Haves

Real world experience with libraries within the Scala Typelevel ecosystem such as Cats, Shapeless, Circe, and Doobie among others.
Willingness to occasionally work with reactive front-end technologies (TypeScript, Angular, or React) to enhance internal project management user interfaces.
Prior experience in healthcare technology, clinical data environments, or a deep passion for improving patient outcomes in healthcare.

Responsibilities

Build and optimize resilient data ingestion pipelines that process high volume streams of clinical data and integrate them into our core oncology platform architecture.
Own the indexing, querying, and performance tuning of our OpenSearch cluster to ensure fast text and data discovery across our oncology data layers.
Implement and refine complex OCR (Optical Character Recognition) processing workflows to transform raw, unstructured patient timelines into clean, structured data.
Expand our core oncology data model by maintaining and enhancing microservices that run automated, medically defined quality checks.
Use AI tools to automate routine setup and boilerplate, keeping your day-to-day focus on building robust, high-impact backend systems.
Design and implement impactful ETL pipelines and high-performance RESTful APIs/microservices.
Participate in thorough code and design reviews, and guide technical decisions for system components while mentoring peer engineers.
Lead the charge in investigating data and quality issues, getting into the weeds of the code to understand why things don't work as expected, and then patching the problem.