Staff Software Engineer - Experimentation

Headway

2d•$265,000 - $331,000•Remote

About The Position

Headway is building a new mental healthcare system to make care accessible to everyone. We have automated the administrative work of insurance, and now over 75,000 providers use our software, serving over 1 million patients. We are developing tools for therapists, improving the process of finding a therapist, and building the platform foundations for scale. We are a Series D company with significant funding, seeking exceptional individuals to help us achieve our mission and make this the most meaningful experience of their careers. The engineering team at Headway is small but impactful, using technology to build the future of mental healthcare. Our guiding principles include focusing on the mission, shipping small and learning fast, treating everything as a product, and optimizing for trust through continuous learning and a safe environment for asking questions and discussing mistakes. We utilize a tech stack including Python 3, TypeScript, FastAPI, SQLAlchemy, React, Remix, LaunchDarkly, Statsig, dbt, Snowflake, Postgres, Redis, AWS, Cloudflare, Spark, Kafka, Datadog, PagerDuty, Sentry, and Github.

Requirements

Staff level engineer with a desire to measure impact by what other teams can achieve.
Systems thinking approach and discomfort with unexplained numbers.
Experience owning an experimentation or feature flagging stack end-to-end, including accountability for result correctness.
Experience building components like persistent assignment, accurate exposure tracking, cross-device/session identity resolution, and early diagnostics for broken experiments.
Fluency in statistics to partner effectively with data scientists, distinguishing between weak and invalid results, and holding a position on measurement questions.
Experience debugging complex measurement bugs (e.g., uneven traffic splits, users in multiple arms, effects vanishing upon exposure analysis, identity issues between anonymous and logged-in sessions).
Experience building a program (not just a platform) that measurably improved an organization's experimentation capabilities through non-code means.
Experience making and living with build versus buy decisions for a platform, reasoning about vendor risk.
Experience migrating teams to new systems by making the new path easier, not by mandate.
Experience using AI as genuine leverage, with clear opinions on where human oversight is necessary.
Proven ability to drive technical direction beyond one's own team.
Willingness to flex across the stack (backend, frontend, CDN, data warehouse).

Nice To Haves

Experience with multi-sided marketplaces.
Experience with growth or search and ranking systems.
Experience in healthcare.

Responsibilities

Own the platform for running experiments at Headway.
Design and build core experimentation primitives, including user assignment, recording user exposure, and recognizing users across devices and states (anonymous/logged in).
Evaluate and consolidate the long-term experimentation and feature flagging stack, currently using LaunchDarkly and Statsig.
Integrate AI into the experimentation workflow for validating setups and automatically detecting broken experiments.
Expand experimentation capabilities to include market-based testing for Headway's multi-sided marketplace, testing geographic markets and holding groups out for long-term impact.
Manage both the technical foundation and the program of experimentation across the company, including defaults, reviews, documentation, and education.
Gather needs from various teams, sequence them into a roadmap, and communicate decisions clearly, acting as a technical product manager for the experimentation platform.
Own assignment, exposure, and experiment data pipelines, working closely with Data Science on statistical methodology and with teams owning event tracking and metric definitions.
Raise the bar for experimentation quality across the organization through defaults, templates, review, and teaching.
Make build versus buy decisions for the experimentation platform, considering vendor risk and warehouse-native approaches.
Drive technical direction beyond the immediate team through design documents, reviews, and planning cycles.
Mentor senior engineers.
Flex across the full stack, including backend services, front-end rendering, CDN behavior, and the data warehouse.