Senior Backend Engineer, Infrastructure

Ellipsis Health•San Francisco, CA

2d•$160,000 - $250,000

About The Position

Ellipsis Health is creating AI/ML products to address healthcare staffing challenges and administrative burdens through conversational AI and patented vocal biomarker technology. The Senior Backend Engineer will be a core contributor on the AI Research & Engineering team, building the real-time voice infrastructure behind the company’s conversational AI products. The role involves designing and operating Python microservices for live voice conversations, spanning audio capture, streaming transcription, LLM logic, synthesis, and playback. The engineer will collaborate with ML, product, data engineering, and client-SDK teams to deliver secure, low-latency, HIPAA-compliant clinical applications.

Requirements

At least 4 years of experience building production back ends in modern Python.
Proven experience with real-time streaming systems such as WebRTC, WebSockets, or gRPC streaming.
Proficiency with asyncio, FastAPI, or similar asynchronous frameworks.
Deep understanding of concurrency, buffering, audio codecs such as Opus and PCM, and distributed tracing.
Solid understanding of AWS, GCP, or Azure, including container orchestration such as Kubernetes/EKS/GKE, message queues such as Kafka/SQS/Pub/Sub, and infrastructure as code such as Terraform.
Solid understanding of relational and in-memory data stores, including PostgreSQL and Redis, and ability to model and persist conversational state.
Excellent communication skills and a bias for measured, observable, continuously deployable software.

Nice To Haves

B.S. or M.S. in Computer Science, Electrical Engineering, or a related field.
Familiarity with voice-agent frameworks.
Hands-on experience with telephony integrations such as Twilio or Telnyx, SIP, or PSTN.
Experience integrating multimodal inputs such as vision and text chat into voice agents.
Familiarity with GPU inference and streaming pipelines.
Prior experience in regulated industries such as healthcare or finance, and comfort preparing for SOC 2 or HIPAA audits.

Responsibilities

Expand and maintain the real-time voice pipeline by designing, implementing, and maintaining Python microservices for conversational AI orchestration, including audio capture, streaming transcription, prompt and LLM logic, synthesis, and playback.
Integrate providers and transports by adding plug-ins for ASR, TTS, LLM, and memory services; connecting WebRTC, SIP, or phone endpoints; and building adapters that support component changes without downtime.
Build API endpoints.
Profile asynchronous pipelines and optimize buffering, concurrency, and back-pressure handling to deliver sub-500 ms round-trip latency.
Instrument pipeline stages with structured OpenTelemetry traces, metrics, and logs, and define SLOs for first-token latency, end-to-end latency, and streaming reliability.
Implement production safeguards including graceful retries, idempotent message passing, circuit breakers, and HIPAA-compliant security measures such as encryption in transit, per-tenant isolation, and secrets rotation.
Collaborate with ML, product, data engineering, and client-SDK teams on features such as voice cloning, multimodal hand-offs, and domain-specific memory retrieval.