Deepen understanding of how models and product experiences impact user outcomes for Raya's recommendations and search systems.
Requirements
- Strong SQL and Python skills, with experience in pandas, NumPy, and data visualization libraries
- Deep understanding of causal inference and experimentation, including methods such as CUPED, diff-in-diff, matching, or propensity-based estimators
- Experience designing and interpreting A/B tests and translating results into product recommendations
- Experience with model tuning and evaluation as well as common ML libraries (e.g. scikit-learn, XGBoost)
- Proficiency with modern data tooling, such as dbt and Snowflake/Databricks for transformation and modeling; Mixpanel or Segment for product instrumentation; and Looker, Omni, or Tableau for visualization
- Familiarity with off-policy evaluation and counterfactual analysis methods (e.g., inverse propensity scoring, doubly robust estimation) used to evaluate recommender or personalization models offline.
Responsibilities
- Run causal and impact analysis for experiments and product changes, helping the team understand not just what happened, but why
- Design and maintain measurement frameworks for key systems like recommendations, ranking, and personalization
- Partner with ML Engineers to evaluate model performance and offline metrics, and to develop scalable evaluation pipelines
- Work closely with Analytics to ensure high-quality experimentation and metric consistency, supporting clear, reliable insights for product and model improvements
- Dig into user behavior and engagement patterns to generate insights and hypotheses that shape product direction
- Contribute to analytical data workflows – for example, defining custom metrics or refining evaluation tables to improve reproducibility and self-serve capability
- Communicate clearly and proactively with product and engineering partners, bringing clarity to complex systems and metric
Other
- 4–7 years of experience in data science, analytics, or a related quantitative field (product or ML-facing experience preferred)
- Strong communication and storytelling skills, with the ability to align stakeholders on insights and measurement strategy
- Comfortable working cross-functionally with ML Engineering, Analytics, Product, and Infrastructure teams
- Excited to grow our use of machine learning, AI, and other sophisticated algorithms to build a better user experience.
- Believe in Raya’s vision, which is to enrich lives by fostering relationships through quality, in person interactions.
