Manager, Testing Optimization & Analytics

Roberts Recruiting•Boston, MA

About The Position

We are looking for a Testing Manager to join our Business Analytics team and lead efforts to solve some of the most complex testing and optimization scenarios in a fast-growing ecommerce company leveraging the trove of online data from our 35M members. This position requires a combination of strong testing, analytics and statistical skills. Solid understanding of ecommerce businesses that operate in multiple channels and cater to an engaged customer base. As a Testing Manager you will help accelerate learnings from a variety of UX/UI, site features, services/offer tests and foster a data driven test-and-learn culture.

Requirements

Degree in a quantitative field such as statistics, mathematics, computer science or economics.
5+ years working within an eCommerce and multi-channel (iOS, mobile web and desktop experiences)
4+ years working in a testing & optimization focused role, A / B and multivariate testing.
4+ years of experience running Site/App features, Pricing, Shipping and Marketing experiments.
4+ years of hands-on SQL experience.
Experience working on scripting languages like Python, Viz tools (Tableau, Looker).
Solid understanding of statistics and analytical frameworks required to implement and interpret test results.
Excellent communication skills, working cross-functionally to achieve results and a high level of comfort delivering analyses and insights.
Strong problem-solving skills and demonstrated ability to take initiative.

Nice To Haves

Experience with testing platforms built-in house and / or 3rd parties (Optimizely, Target, Optimize, SiteSpect, AB Tasty).
Experience in conversion optimization, digital marketing, UX design and / or consumer psychology and research.

Responsibilities

Manage our testing & optimization program to empower technology, product and business teams with data driven insights.
Act as primary testing & optimization subject matter expert.
Maintain a testing backlog, coordinate execution of key tests across the organization while correcting for cross-impact.
Proactively monitor live tests and ensure they are appropriately implemented with proper test design, sampling, metrics, statistical and analytical frameworks.
Build interactive dashboards and perform analyses needed to support the outcome and interpretation of test results.
Proactively communicate summarized test results and recommendations to stakeholders.
Work crossfunctionaly with data science, business intelligence, product and business teams to ensure business requirements and testing standards are met for all tests.
Drive continuous improvements and advancements in the testing program, including implementing industry best practices in methodologies, technologies and automation.
Create a testing knowledge base by recording successes, failures and trends accumulated across tests, and effectively communicate these learnings and insights across teams.
Train product managers, analysts and business stakeholders on testing best practices, interpreting results and recommendations to drive an active test-and-learn culture.
Identify gaps and emerging trends in testing platforms, methodologies.
Provide recommendations on new solutions to continually improve testing capabilities and efficiencies.
When needed, work directly with vendors to drive technological advances and to resolve issues.
Expected to perform these duties with little daily oversight.