Mercor is seeking independent experts to support a study focused on evaluating the quality, novelty, and methodological rigor of modern ML research
Requirements
- Strong academic or professional background in artificial intelligence, machine learning, or related research domains
- Experience interpreting and analyzing ML papers; familiarity with top-tier venues (e.g., NeurIPS, ICLR, ICML) is preferred but not required
- Ability to read and evaluate Python code, experimental pipelines, and quantitative logs
- Demonstrated experience with academic literature review tools
Responsibilities
- Critically review machine learning papers for methodological soundness, novelty, and clarity
- Inspect Python source code and experimental logs to validate reported metrics against underlying data
- Conduct literature reviews using tools such as Google Scholar or Semantic Scholar to identify prior work, detect plagiarism, and assess research originality
- Provide structured feedback on “human-quality” research assessments and potential ethical or safety-related concerns
- Submit non-identifying background details (e.g., affiliation, highest degree, publication links) to establish evaluator credibility for the study
Other
- Doctoral students, industry researchers, and published authors are encouraged to provide services
- High attention to detail and ability to assess complex technical arguments
- Submit your resume or CV to begin
- Complete a brief form outlining your technical background and relevant publications
- Additional screening questions or sample evaluations may be requested to confirm expertise
