Senior Software Engineer - AI Automation

Roku•San Jose, CA

1d•$141,300 - $360,700•Hybrid

About The Position

Roku is the #1 TV streaming platform in the U.S., Canada, and Mexico, and we've set our sights on powering every television in the world. Roku pioneered streaming to the TV. Our mission is to be the TV streaming platform that connects the entire TV ecosystem. We connect consumers to the content they love, enable content publishers to build and monetize large audiences, and provide advertisers unique capabilities to engage consumers. From your first day at Roku, you'll make a valuable - and valued - contribution. We're a fast-growing public company where no one is a bystander. We offer you the opportunity to delight millions of TV streamers around the world while gaining meaningful experience across a variety of disciplines. With more than 100 million people using our products globally, we’ve become well-known for products that “just work” right out of the box and integrate almost by magic. That doesn’t happen by accident, which is why we are committed to making sure our products aren’t just intuitive; they’re obvious. Our goal is to help people find what they want and make it easier for people to stream. We do this with state-of-the-art technology and engineering, keeping the customer at the center of everything we do. We're looking for a hands-on, systems-oriented Senior Software Engineer in Test (Sr. SDET) to join our Browse and Discovery Team. You'll own quality for the APIs and data pipelines — the automated test frameworks, containerized quality checks, CI/CD workflows, and infrastructure as code that let the rest of the organization ship with confidence. You'll do that work with agentic AI as your default mode of working: driving agents to write and maintain integration tests, keep suites healthy, triage failures, and debug production issues, and building the tools, harnesses, and evaluations that make those workflows trustworthy. This is a role for someone who treats agent design as an engineering discipline — grounding agents in the right context, integrating them with the systems they act on, proving them reliable through rigorous automation and evaluation, and turning what works into the reusable components and paved paths other teams at Roku build on.

Requirements

5+ years of experience in software testing and automation, software engineering, or adjacent domains, with strong software engineering fundamentals and the ability to build production-grade systems.
Expertise in Python or Java with familiarity in the other; experience with C/C++ or another systems language is a plus.
Experience designing test plans, test cases, and automated test frameworks for APIs, services, and data pipelines.
Hands-on experience with LLM-based systems, including prompt design, retrieval, tool use, memory handling, and agent orchestration patterns.
Experience building and maintaining retrieval and context pipelines, agent frameworks, and MCP servers or equivalent function-calling architectures.
Experience with CI/CD tooling such as GitLab runners, GitHub Actions, Jenkins, or Travis CI.
Experience with cloud platforms — AWS (preferred), GCP, or Azure — REST APIs, and containerization and orchestration tools such as Docker and Kubernetes.
Experience with infrastructure as code: Terraform or CloudFormation.
Working knowledge of Linux and Bash scripting.
Experience with observability, evaluation, experimentation, and feedback loops for AI systems in production, including monitoring tools such as DataDog, Prometheus, or Grafana.
Ability to work independently, manage ambiguity, move quickly, and deliver incrementally in a fast-paced environment.
Strong analytical and problem-solving skills, excellent communication and collaboration skills, and sound engineering judgment.
Bachelor's or master's degree in Computer Science, Computer Engineering, Electrical Engineering, Data Science, or a related technical field — or equivalent engineering experience.

Responsibilities

Design and develop automated test frameworks for APIs and data pipelines, and containerize automated quality checks for complex orchestrated services.
Use agentic AI workflows day to day — authoring and maintaining integration tests, keeping test suites healthy, triaging failures, and debugging production issues.
Build the context and retrieval pipelines that ground these agents in the right code, schemas, telemetry, and business logic, and keep them aligned as those sources evolve.
Implement tool-calling and MCP-style integrations so agents can safely act on the systems around them — test runners, CI, ticketing, logs, and data services.
Build and maintain CI/CD workflows and infrastructure as code using Terraform or CloudFormation, so deployments and test execution are repeatable and auditable.
Establish evaluation, observability, and monitoring for the signals that matter here: test reliability and flake rate, mean time to triage, agent task success rate, latency, and cost.
Build safeguards that improve production readiness and reliability — controlled rollouts, drift detection, and mechanisms that prevent error amplification in multi-step agent workflows.
Create reusable templates, modular components, and paved-path patterns that accelerate adoption across teams, and partner with development, data engineering, and product teams to improve end-to-end testing and release processes.