OpenAI's Platform team needs to drive a data-driven culture for its API and B2B solutions, defining key metrics, measuring impact, and partnering to improve model quality, reliability, latency, and cost, ultimately shaping how thousands of products adopt agentic AI.
Requirements
- Depth in SQL and Python, with a track record proposing, designing, and running rigorous experiments
- Experience defining and operationalizing metrics from scratch (including reliability/latency/cost and safety)
- Background in developer platforms, observability, or usage-based pricing/quotas
- Experience connecting offline evals to online product impact
- Prior work in NLP/LLMs or agentic systems, or in enterprise analytics for B2B products
Responsibilities
- Define north-star metrics across the developer funnel (activation, retention, growth), as well as latency/cost guardrails for new features and models
- Design and interpret A/B tests and controlled rollouts (e.g., new model versions, pricing/limits, new API features, new B2B products)
- Build source-of-truth dashboards and self-serve data tools for product, engineering, and go-to-market teams
- Translate product learnings into actionable feedback for Research (e.g., failure modes, eval gaps, model response quality)
Other
- Embed with the Platform product team as a trusted partner, uncovering ways to improve developer experience, reliability, and usage growth
- 5+ years in a quantitative role in ambiguous, high-growth environments (platforms, APIs, or B2B products a plus)
- Strong cross-functional communication with PMs, engineers, and executives
- Strategic instincts beyond p-values—clear thinking about tradeoffs and business impact
- This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees.
