QA Engineer

h2o.ai•Mountain View, CA

1d•Hybrid

About The Position

You will be a core QA engineer responsible for the end-to-end quality and reliability of h2o.ai's product portfolio. This is a hybrid position based in Mountain View, CA.

Requirements

2-4 years of QA experience with a strong mix of automation and hands-on manual/exploratory testing.
Decent Python skills and experience building maintainable test frameworks from scratch.
Real-world experience testing modern React/TypeScript web applications and writing bulletproof Playwright or Selenium tests.
Hands-on Kubernetes experience in a testing or test-environment context (kubectl, Helm, writing manifests, debugging pods).
Proven ability to use generative AI tools daily to accelerate debugging, test-data creation, and log analysis (you’ve already used h2o.ai's product portfolio, ChatGPT, Claude, or similar in your current QA workflow).
Comfort reading and triaging complex logs from LLM frameworks, vector DBs, and tracing systems.
Solid grasp of CI/CD (GitHub Actions preferred) and infrastructure-as-code concepts.

Nice To Haves

Prior experience testing RAG systems, agentic workflows, or enterprise chat/assistant platforms.
Experience with visual diffing of generated outputs (documents, charts, markdown).
Chaos engineering on Kubernetes (Chaos Mesh, Litmus) or GPU workloads.
Familiarity with Chaos engineering principles.
Basic understanding of containerization (Docker/Kubernetes concepts like pods and kubectl) in a testing context.

Responsibilities

Design, build, and maintain Python-first automation frameworks (pytest + Playwright + async) for UI, API, and end-to-end workflows of h2o.ai's product portfolio.
Perform deep exploratory and manual testing of the h2o.ai's product portfolio web UI (chat interfaces, document upload/processing, agent builder, maker suite, evaluation hub, enterprise admin console).
Use h2o.ai's product portfolio itself (and other GenAI tools) in creative ways to: Generate realistic test documents, datasets, and edge-case prompts. Auto-generate or refine test cases via prompt engineering. Rapidly summarize and debug massive, complex log files (e.g., Kubernetes pods, etc.). Explain cryptic LLM traceback chains or hallucination root causes in seconds instead of hours.
Root-cause of difficult, intermittent failures in distributed RAG/LLM systems by combining traditional log analysis with GenAI-assisted debugging.
Create and execute chaos experiments targeting LLM routing, vector database latency, GPU OOM, retrieval failures, and token-limit edge cases.
Build and manage ephemeral h2o.ai's product portfolio clusters on Kubernetes for testing (Helm, custom operators).
Own UI regression suites (Playwright) and accessibility testing.
Write reproducible, high-quality bug reports that developers love and regularly verify fixes across the full stack.
Collaborate closely with the h2o.ai's product portfolio feature teams in an Agile environment and influence testability from the design phase.