Pathos is looking to design, train, ship, iterate on, and innovate on the AI brains behind their AI Therapist. This involves combining research, data science, and engineering to create models, orchestration, and evaluation systems that make therapy conversations deeply effective, clinically grounded, and safe.
Requirements
- Demonstrates strong experience with large language models, including fine-tuning, training data design, and model selection.
- Knows how to move core metrics on conversation quality and user outcomes, rather than chasing generic benchmarks.
- Can look at evals, transcripts, and metrics and quickly form grounded hypotheses for improvement.
- Experience shipping production-level code and/or maintaining an AI system in production.
- Can set up production-level data pipelines for training new models, evals, analysis, etc.
- You formulate hypotheses, and you are good at evaluating them (eg through experiments, data analysis, etc).
- You are consistently learning at the cutting edge, and you’re able to leverage and communicate those learnings to make the entire company more successful.
Responsibilities
- Design, train, ship, iterate on, and innovate on the AI brains behind Pathos’ AI Therapist.
- Combine research, data science, and engineering to create models, orchestration, and evaluation systems that make therapy conversations deeply effective, clinically grounded, and safe.
- Deliver measurable improvements in conversation quality, therapeutic alliance, and user outcomes through fine-tuning strategies, training data curation, building RL environments, new model architectures and other AI innovations.
- Improve on and maintain a robust eval stack that includes scripted tests, LLM-as-judge evaluations, human ratings, and safety checks.
- Build, maintain, and iterate on the production codebase that delivers AI therapy and supports the evaluation and iteration of our AI.
- Own the path from notebook to production: training jobs, model packaging, deployment, monitoring, and rollback strategies.
- Work with clinicians and internal experts to encode clinical guidelines into prompts, reward functions, tools, and filters.
Other
- Translate product and clinical requirements into concrete model and system changes.
- Partner with full-stack product engineers so that new AI capabilities are easy to integrate and maintain in the product.
- You are keenly aware of how to provide company value and to prioritize projects accordingly.
- Refuses to ship subpar work, continuously improving the codebase.
- Prioritizes speed by leveraging AI, breaking down complex tasks, shipping early, optimizing for learnings, iterating quickly, and avoiding over-engineering.
- You can work collaboratively in a positive way.
- Personal or other experience with therapy or coaching
- Domain knowledge of psychology, neuroscience, therapy, or coaching.
