NBCUniversal's Experimentation team needs to own the technology that enables experimentation and act as subject matter experts to help teams measure the performance of features and campaigns. The Sr. Manager, Experimentation (Technical) will own the statistical frameworks and processes that power analytics and operational solutions to support self-service insights and decisioning within the organization.
Requirements
- Direct experience working with 1st and/or 3rd party experimentation platforms (i.e. in-house, Optimizely, Eppo, Stat-sig, or Similar)
- Expert knowledge of experimental design, statistics and other analytical concepts (i.e., RCTs, Frequentist and Bayesian analysis)
- Advanced coding skills to implement and automate computational methods and algorithms, related but not limited to causal inference and experimentation (Python, R, SQL, etc.)
- 6+ years of applied experience in experimentation at an ecommerce, social network, direct to consumer, media entertainment or similar tech company
- 3+ years of experience leading a technical team and developing talent
- Past relevant experience in computational programming in either industry or academia
Responsibilities
- Develop statistical frameworks and processes designed to analyze experiments to enable fast, reliable and accurate decision-making for product, marketing, ads and marketing teams
- Enhance experiment analysis methodologies, including Bayesian and frequentist approaches, variance reduction techniques, and sequential testing
- Collaborate with engineering to integrate statistical algorithms and new metrics into our experimentation platform
- Develop automated solutions for experiment diagnostics, anomaly detection, and guardrail monitoring
- Stay ahead of industry advancements, improving our statistical engine with the latest methodologies and best practices
- Communicate statistical concepts to stakeholders, including product managers, analysts, and engineering teams
- Lead Center of Excellence for Experimentation – Define best practices and frameworks for trustworthy online controlled experiments; drive cultural adoption of data-driven decision making across product and engineering teams.
Other
- Hybrid: This position has been designated as hybrid, generally contributing from the office a minimum of three days per week.
- A great storyteller and communicator that can use data to help influence product decisions and change opinions
- Combination of technical skill sets and business savvy to interface with all levels of the organization
- Ability to work independently, multitask, and drive your own projects
- A passion for TV and movies and defining the future of streaming entertainment
