QA Engineer

Ruby Labs

1d•Remote

About The Position

At RubyLabs, we're seeking a QA Engineer to own the validation of product experiments from preview to production. You'll work closely with the Growth team to ensure experiments are correctly implemented, accurately tracked, and safe to release. This role goes beyond traditional QA. You will validate experiment logic, user flows, targeting conditions, and analytics accuracy across all variants, ensuring teams can make reliable product decisions based on trustworthy experiment results. Because experiments directly impact product growth and revenue, your work will play a critical role in maintaining product quality and data integrity.

Requirements

Experience testing A/B tests, feature flags, or growth experiments.
Experience with Mixpanel, Amplitude, or similar analytics platforms.
Ability to validate API responses and investigate backend-related issues.
Experience using session replay or observability platforms for production validation.
Experience working in fast-moving product-led environments.
Strong ability to identify issues across UI, business logic, and data tracking.
Ability to evaluate features from both a user experience and business perspective.
Takes responsibility for the quality and outcome of released experiments.
Comfortable investigating complex behaviors and uncovering root causes.
Ability to manage multiple experiments and priorities simultaneously.
Clear and structured communication with Product, Growth, Design, and Engineering teams.

Responsibilities

Own end-to-end QA validation of experiments from preview environments through production release.
Validate all experiment variants (A/B/C/etc.) and ensure expected behavior across different user scenarios.
Verify audience targeting logic, eligibility rules, and experiment trigger conditions.
Ensure analytics events, conversions, and experiment metrics are correctly tracked and reported.
Validate experiment behavior throughout the entire user journey.
Ensure experiment implementations meet design requirements and provide a consistent user experience.
Perform post-release validation using production environments and session replay tools.
Identify unexpected behaviors and issues beyond predefined test cases.
Ensure experiments do not negatively impact core product functionality.
Provide clear release recommendations and final QA approval before launch.