Medal is looking to improve the accuracy, completeness, and reliability of its video datasets and labels to enhance the machine learning features that power the platform.
Requirements
- 5+ years in data analytics or data science with a focus on media or ML data quality in production systems.
- Fluency in SQL and Python (Pandas/NumPy); you’re comfortable building reproducible notebooks and code-reviewed pipelines.
- Strong measurement chops: you’ve defined and computed label & model quality metrics (precision/recall/F1, mAP, AUROC, calibration, temporal IoU) and can explain their trade-offs.
- Data validation & ETL experience: Great Expectations/TFDV (or equivalent), dbt, and an orchestrator (Airflow/Prefect).
- Warehouse & BI: BigQuery (or similar) plus Looker/Mode/Tableau (or similar); you build clear dashboards and know when to run deep dives.
- Experimentation: A/B testing design and analysis; comfort with pitfalls and guardrails.
- Experience running annotation programs (Label Studio, CVAT, Scale or custom tooling) and crafting labeling taxonomies for actions/events/scenes.
Responsibilities
- Own the video data quality program: define quality KPIs (coverage, precision/recall, calibration, temporal alignment, label latency, drift) and build dashboards that make them visible company-wide.
- Audit datasets at scale using SQL and Python: create automated checks for codec/bitrate/fps/resolution, audio/video sync, corruption, duplicates, and long-tail coverage by game, device, and region.
- Design ground-truth pipelines: human-in-the-loop reviews and labeling guidelines; measure annotator agreement, and iterate to improve label quality.
- Diagnose model-data issues: collaborate with ML to localize failure modes, quantify data gaps, and prioritize data collection or relabeling to move accuracy on real user content.
- Detect bias and drift across games, platforms, and cohorts; propose mitigations and monitor post-launch.
- Instrument product and ingestion to capture the metadata ML needs (e.g., encoding, device, frame rate, content type) while respecting privacy and safety constraints.
- Run experiments: design and analyze A/Bs and holdouts to connect data quality improvements to model and product outcomes.
Other
- Work on-site at our NYC office 5 days a week.
- Product sense & communication: you turn ambiguous problems into measurable roadmaps and communicate findings clearly to technical and non-technical partners.
- A love for gaming, however you define it.
- Prior history as a Medal user—share a clip or your profile!
- Work with a passionate team that values ownership, craftsmanship, and speed.
