Staff Software Engineer, Product Quality

Recruiting From Scratch • San Francisco, CA

10h • Onsite

About The Position

We're representing a fast-growing AI infrastructure company behind one of the world's most widely used LLM gateways. Its open-source platform provides a standardized interface for accessing 100+ LLM providers and is trusted by major organizations across technology, media, government, and enterprise environments. With such a broad integration surface and significant open-source contribution volume, product quality is a core infrastructure problem. The company is hiring a Staff Engineer to own quality across testing, CI, release infrastructure, and provider integrations—building systems that automatically catch regressions and upstream model-provider changes without slowing engineering velocity. This is a deeply technical individual contributor role for an engineer who wants to remain in the code while setting the quality bar across a high-scale AI developer platform.

Requirements

• Senior-to-Staff level software engineering experience with deep technical ownership of production systems.
• Experience working at a Seed through Series D startup and comfort operating across multiple areas as priorities change.
• Proven experience building test, CI, release, or developer infrastructure for complex production software.
• Experience supporting products with a large external integration or API surface.
• Strong debugging ability with a track record of identifying underlying root causes rather than patching symptoms.
• Experience designing automated test suites engineers trust enough to gate production releases.
• Strong understanding of CI/CD, release engineering, integration testing, and production reliability.
• Track record of improving engineering quality without becoming a bottleneck to shipping.
• Experience contributing to or maintaining open-source software.
• Comfortable taking organization-level technical ownership as an individual contributor.

Nice To Haves

• Maintaining CI or release infrastructure for a large open-source project with significant outside contribution volume.
• Building infrastructure around third-party APIs where upstream behavior can change unexpectedly.
• Background in AI infrastructure, LLM platforms, developer tools, API gateways, or cloud infrastructure.
• Testing LLM functionality including streaming, tool/function calling, structured outputs, and model-provider error handling.
• Strong Python experience.
• Designing canary releases, synthetic monitoring, or automated production validation systems.
• Familiarity with multi-channel release strategies balancing rapid iteration with enterprise stability.
• Meaningful GitHub or other public open-source contributions.
• Strong opinions about testing architecture, engineering quality, and what makes automated test infrastructure genuinely trustworthy.

Responsibilities

• Own product quality end-to-end across a platform integrating with more than 100 LLM providers.
• Design and build comprehensive provider conformance testing covering streaming, tool calling, structured outputs, authentication, error mapping, and cost tracking.
• Build and own CI infrastructure that is fast, deterministic, and reliable enough to serve as a true release gate.
• Develop automated testing strategies that accommodate a large volume of external and open-source contributions.
• Build canary testing and synthetic traffic systems that identify upstream provider changes before they impact customers.
• Design release infrastructure that enables rapid daily shipping while maintaining stable release channels for enterprise customers.
• Debug complex failures across unfamiliar code paths, third-party integrations, and provider APIs.
• Perform deep root-cause analysis and convert escaped production defects into permanent regression coverage.
• Improve testing architecture, developer tooling, and release workflows without introducing unnecessary process or slowing engineering velocity.
• Raise the technical bar through hands-on coding, architecture decisions, debugging, and code reviews.

Benefits

• Competitive equity package.
• Opportunity to build quality and release infrastructure for an AI gateway supporting 100+ LLM APIs.
• Work on infrastructure used by major organizations while contributing to a highly visible open-source ecosystem.
• High-ownership startup environment where engineering quality is driven through code, automation, and strong technical judgment rather than additional process.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed