Head of Experimentation

About this role

About the Job:

Feature management and experimentation have converged into a single market, and the buying dynamic at the top has shifted. Engineering teams are no longer the sole evaluator — data scientists and data-focused PMs now carry equal weight on the largest deals. The bar for statistical depth, warehouse ergonomics, and experiment-first workflows is rising quickly.

In traditional experimentation we have built the foundation: a trusted runtime control plane, a growing experimentation engine, and early warehouse-native capabilities. We are winning lower-maturity buyers at healthy rates. We are not yet consistently winning the most sophisticated data organizations. Closing that gap is the job.

In AI experimentation, we have an early lead: the AI-native tooling category has invested in evaluation and conceded production experimentation, and we already have the primitives (statistical significance, multi-armed bandits, experiment-aware guardrails) that no AI-native competitor ships. Extending that lead is the other half of the job.

This leader will own whether LaunchDarkly becomes the definitive experimentation platform in an AI-accelerated world.

Responsibilities:

• Own the Experimentation pillar. Direct leadership of the Product team. Partner with Engineering and Design counterparts in a triad model. Accountable for the pillar's strategy, roadmap delivery, and commercial outcomes. Make the investment case across the in-product experimentation experience, the warehouse-native analysis layer, and the infrastructure that scales them.

• Make experimentation the measurement layer of the AI SDLC. Partner with our AI product, observability, and core feature management leaders to productize the capabilities we already have as AI-native primitives. Build a closed loop from offline evaluation through production experiments, to automatic promotion and rollback, to a self-improving feedback loop for agents.

• Win the high-maturity buyer. Earn the technical confidence of senior data scientists and data-focused PMs. Decide what statistical depth, warehouse coverage, and experiment-first workflow capabilities are non-negotiable, and get them shipped on a timeline that wins pivotal reference deals.

• Make warehouse-native a weapon. Expand coverage across major data warehouses and query layers. Deliver parity on analysis-only mode, variance reduction, ratio and percentile metrics, exposure validation, and arbitrary-window analysis.

• Operate a high-performing function. Run a disciplined roadmap, ship predictably against quarterly commitments, drive AI-assisted engineering productivity inside the org, and hire where gaps exist.

• Be the external face of the category. Credibly represent the product with Data scientists, PMs, experimenters, analysts, and partners. Translate the strategy to the field and equip sales to win head-to-head.

How you'll be measured:

• Win rate on experimentation-involved deals, especially head-to-head competitive evaluations — step change in the first year, sustained improvement thereafter.

• Reference-grade customers at the top of the maturity curve, including named strategic logos.

• Monthly active customers and active-account ARR growth against plan.

• Experimentation attach rate on new and expansion enterprise deals.

• Engineering throughput — roadmap delivery velocity and AI-assisted development adoption inside the function.

Qualifications:

• Senior product leader (GM, VP, or equivalent) with a track record of owning a product line that competes on statistical rigor and data infrastructure.

• Deep, operator-level fluency in experimentation methodology: causal inference, variance reduction, ratio metrics, sequential testing, exposure design, multi-armed bandits, and composite/multi-objective metrics — and the realities of running these at scale against production data warehouses and against non-deterministic systems where output variance, not just user variance, drives sample-size and significance decisions.

• Has earned credibility with data science leaders and experimentation specialists at sophisticated organizations — and can recruit them.

• Has led a function that includes engineering, design, and data science. Comfortable setting a multi-quarter roadmap, championing investment allocation, and reporting results to an executive team and board.

• Clear, direct communicator. Decides fast with incomplete information. Prefers shipping and learning to requirements documents.

• Opinionated about where experimentation is going in an AI-native world — and specifically, how agents and autonomous systems will use experimentation infrastructure differently than human teams do.

Preferred Qualifications:

• You have built or scaled experimentation at an organization where it was core infrastructure, not a secondary analytics capability.

• You have personally won competitive evaluations where a sophisticated data-science organization was the deciding voice.

• You have shipped warehouse-native data products and understand the operational realities of running experiments directly against customer data infrastructure.

• You see experimentation as how software teams prove that any change — whether built by a person or an AI agent — actually worked. That evidence layer is core infrastructure, not a reporting afterthought.

Pay:

Target pay ranges based on Geographic Zones* for Level M5:

• Zone 1: San Francisco/Bay Area or NYC Metropolitan Area, Boston, Seattle - $ 301,000 - $414,000 **

• Zone 2: Irvine, LA, Monterey, Santa Barbara, Santa Rosa, Austin, Portland, Philadelphia, Chicago - $ 271,000 - $373,000 **

• Zone 3: All other US locations - $ 256,000 - $352,000 **

• All Zones inclusive of 20% Bonus

LaunchDarkly operates from a place of high trust and transparency; we are happy to state the pay range for our open roles to best align with your needs. Exact compensation may vary based on skills, experience, and location.

*Within the United States, our ge

Similar jobs

• Engineering Manager, Experimentation — Launchdarkly · Remote
• Applied Scientist, Experimentation — Coframe · SF Bay Area
• Engineering Manager, Core Experimentation — Openai · Seattle
• Senior Product Manager, Experimentation Tooling — Lightningai · New York, New York, United States; San Francisco, California, United States
• Technical Product Manager, Experimentation Platform — Kayak · Concord Office
• Software Engineer, Product - Core Experimentation — Openai · Seattle
• Staff+ Software Engineer, Experimentation — Anthropic · San Francisco, CA | Seattle, WA
• Full Stack Engineer, Experimentation — Launchdarkly · Remote