Growth Experimentation Manager

Reflex Media, Inc. • ,

1d • $110,000 - $145,000 • Remote

About The Position

Seeking.com is looking for a Growth Experimentation Manager to own the full experimentation program end-to-end as a hands-on operator. This role sits at the intersection of statistics, marketing, product, and data. The manager will set the testing roadmap, run rigorous experiments across paid media, lifecycle, landing pages, onboarding, and creative, and translate findings into concrete channel and creative decisions. The company is seeking a builder who can audit existing tests, identify issues, and ship better-instrumented experiments. This role is crucial for driving the brand transformation by ensuring every channel decision, creative call, and onboarding tweak is informed by clean, well-designed experiments rather than opinions. The current experimentation program needs more rigor, velocity, and a single source of truth.

Requirements

• 3 to 5+ years of hands-on experimentation, CRO, growth marketing, or analytical marketing experience, preferably at a consumer subscription, marketplace, mobile-first, or dating/social platform.
• Ability to audit an existing test program, identify what is misdesigned, and ship a better-instrumented experiment in week one.
• Deep understanding of experimental design: control/treatment setup, sample sizing, statistical significance, power calculations, novelty/seasonality/SRM threats, and how to mitigate them.
• Hands-on, in-the-tool expertise with at least one experimentation or A/B testing platform (Optimizely, VWO, Statsig, LaunchDarkly, Eppo, Convert, AB Tasty, or comparable).
• Strong analytical fluency. Comfortable in SQL and directly in the warehouse, plus a product analytics tool (Amplitude, Mixpanel, Looker, Heap, or similar) to pull your own analysis and pressure-test Data team outputs.
• Proven track record of running experiments across at least two of: paid media, email/lifecycle, landing pages, onboarding, paywall, or in-product marketing, with numbers you can speak to.
• Outstanding communication and storytelling. Ability to write a crisp experiment brief and present nuanced results to a non-technical audience with equal confidence.
• Experience with holdout group design and geo-based experiments.
• AI fluency. Experience using AI tools like ChatGPT, Claude, Cursor, Cowork, or similar in daily workflow for drafting test plans, generating variant copy and creative, analyzing results, writing SQL, and automating tasks.
• Operator’s discipline. Ability to ship on time, ensure tight QA, prevent cross-channel contamination, and document every result.

Nice To Haves

• Experience at a consumer subscription, marketplace, or dating/social platform where engagement and retention are primary growth levers.
• Familiarity with Bayesian experimentation methods and a clear view on when to apply them vs. frequentist approaches.
• Background in behavioral economics or consumer psychology.
• Experience building or contributing to a company-wide experimentation program from the ground up, including governance, tooling selection, and team training.
• Exposure to multi-armed bandit testing, contextual bandits, or adaptive experimentation.
• Experience designing experiments around AI-driven personalization or recommendation systems.
• Bachelor’s degree in Statistics, Economics, Mathematics, Computer Science, Marketing, or a related field, or an equivalent track record of shipping rigorous experiments.

Responsibilities

• Build, maintain, and prioritize a comprehensive testing roadmap across paid media, email/lifecycle, SEO, landing pages, onboarding, paywall, and creative.
• Develop a hypothesis backlog informed by quantitative funnel analysis, qualitative user insights, competitive research, and channel team input.
• Define and enforce a structured prioritization framework to stack-rank tests by potential impact, confidence, and ease of execution.
• Establish testing velocity benchmarks and ensure the team runs the optimal number of concurrent, non-interfering experiments at all times.
• Maintain an always-current view of what is being tested, what has been learned, and what is next, and communicate it weekly to leadership and channel owners.
• Design A/B, multivariate, holdout, and geo-based experiments with proper control groups, adequate sample sizes, and pre-defined success metrics.
• Partner with Data & Analytics and Engineering to ensure correct instrumentation, event tracking, and attribution before any test launches.
• Define primary, secondary, and guardrail metrics for every experiment to capture both intended lift and unintended side effects.
• Apply the right statistical method for the test type, traffic volume, and decision urgency, and know when to call a test early vs. wait for significance.
• Identify and mitigate threats to validity: novelty effects, seasonality, sample ratio mismatch, network effects, and interaction effects between simultaneous tests.
• Conduct deep post-experiment analysis going beyond top-line win/loss to understand segment-level effects, interaction effects, and downstream impact on LTV.
• Distinguish between statistical significance and business significance.
• Build and maintain a results repository capturing test parameters, outcomes, learnings, and confidence levels, making institutional knowledge searchable and actionable.
• Present findings to cross-functional stakeholders in a clear, non-technical narrative that connects test outcomes to business strategy.
• Deliver a structured recommendation for every completed experiment: ship it, iterate on it, or kill it, with supporting rationale and proposed next steps.
• Translate winning test results into channel-specific updates: campaign and targeting changes for Paid Media; flow logic and segmentation for Lifecycle/CRM; landing page and onboarding updates with Product; creative direction briefs for the Creative team; SEO and UX recommendations from engagement tests.
• Flag when a result should trigger a broader strategic shift vs. a tactical tweak, and escalate those moments proactively.
• Build iterative test sequences where each experiment compounds prior learnings, rather than running isolated one-off tests.
• Connect the dots across channels. Surface patterns from paid media tests that should inform lifecycle messaging, and vice versa.
• Build shared creative and messaging frameworks derived from test learnings that every channel owner can apply.
• Partner with the Lifecycle Marketing & CRM Manager, the SEO Manager, and the Performance Media Manager to ensure experimentation coverage across organic, paid, and owned funnels.
• Serve as the primary liaison between Marketing and Data & Analytics for testing, translating business questions into test designs and analytical outputs back into marketing action.
• Evaluate, implement, and manage experimentation tooling: A/B testing platforms, feature flagging, geo-experiment tools, and analytics integrations.
• Define and document testing standards, naming conventions, QA checklists, and launch/kill criteria so experiments are run consistently across the team.
• Champion a test-and-learn culture. Run workshops, share weekly wins and learnings, and build org-wide confidence in data-driven decision making.
• Identify gaps in tracking, attribution, or event reliability that limit clean experimentation, and advocate for fixes with Engineering and Data.

Benefits

• Health, dental, vision, 401(k), and a standard benefits package.
• Direct exposure to the Director, CMO, and Co-CEOs from day one.
• Clear growth path to Senior Growth Experimentation Lead within 12 to 18 months, based on outcomes.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed