Senior Software Development Engineer in Test - SF

Medeloop • San Francisco, CA

2d

About The Position

We're looking for a Senior SDET who thinks deeply about quality in systems that are inherently non-deterministic. Agentic AI doesn't fail the same way traditional software does — and testing it requires a new toolkit: eval frameworks, prompt regression, tool-call reliability, adversarial scenarios, and more.
You'll own the entire quality infrastructure across our product portfolio — from test data and CI pipelines to the standards and culture of how we ship. You'll work directly with product, devops, and AI engineering, with no layers between your decisions and their impact.

Requirements

• 8+ years of hands-on SDET experience, with recent work building or testing agentic AI systems (single- or multi-agent) in production
• Experience in healthcare or life sciences — you understand what's at stake when a system fails in this domain
• A true tester's mindset: you seek out edge cases, adversarial inputs, and failure modes others overlook
• Proficiency across the full test pyramid — unit, integration, system, performance, and exploratory — plus familiarity with LLM-specific evaluation approaches
• Strong debugging skills across multi-tier web and mobile architectures; comfortable jumping into production incidents
• Proficiency with testing frameworks such as Jest, React Testing Library, Supertest, and pytest.
• Hands-on experience with testing tools like Cypress, Playwright, Supertest, and pytest (including requests or Selenium-based testing)
• Experience testing RESTful APIs using tools like Postman or Supertest
• Solid command of JavaScript and Python

Nice To Haves

• Multi-cloud experience (AWS, Azure, GCP)
• Experience with red-teaming or adversarial testing of AI systems
• Native mobile testing experience (iOS, Android)
• Prior work with 21 CFR Part 11, GxP, or similar regulated-software validation frameworks

Responsibilities

• Test infrastructure, test data, test processes across the entire product portfolio while working with Devops and Infrastructure engineers
• Test Framework - build and enhance automated testing frameworks and tools that facilitate automated testing across different layers of application.
• The reliability bar for all Web applications, Mobile applications and AI agent outputs — from hallucination detection to latency regressions and tool-call correctness
• Test infrastructure, test data, and test processes across the entire product portfolio, alongside DevOps and Infrastructure engineers
• Automated testing frameworks that span all layers of the application — unit, integration, contract, and end-to-end
• Evaluation frameworks designed for LLM-based systems: non-deterministic output scoring, prompt regression, and adversarial test suites
• HIPAA-aware test data management — de-identification pipelines, synthetic data generation, and audit trail validation
• Integration of automated tests into CI/CD pipelines for continuous delivery confidence
• Build stability monitoring and release gate enforcement before any deployment
• Documentation of test plans, test results, and evaluation standards to support knowledge sharing
• The "safety net" for product quality — you define what done looks like

Benefits

• Ownership from day one: small team, high-trust, no layers between your work and its impact
• Technically ambitious: you'll build AI-powered workflows, not just support them
• Real-world stakes: your work accelerates drug development, addresses health equity, and improves clinical research for institutions that matter
• Strong foundation: Series A, top-tier investors, and a data asset (200M+ patient records) that most companies spend years trying to build

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed