Assembled is looking to solve the problem of maximizing AI agent performance and driving volume automation across voice, chat, and email channels by building sophisticated evaluation systems, automating knowledge generation, optimizing AI agent accuracy, and creating the infrastructure needed to continuously improve agent intelligence at scale.
Requirements
- Have experience with AI evaluation systems, data pipelines, or AI model optimization
- Have worked with retrieval systems, knowledge bases, or information extraction
- Python/Golang for evaluation systems and data processing
- LLMs and prompt optimization frameworks
- Vector databases and retrieval systems
- AI evaluation and benchmarking tools
- Data pipeline and automation infrastructure
Responsibilities
- Build foundational evaluation infrastructure: Develop comprehensive evaluation systems from the ground up, including golden dataset creation, automated benchmarking, and model comparison tools. You'll help create the frameworks that enable us to measure and optimize AI agent performance across all communication channels.
- Automate knowledge generation: Design and implement systems that automatically create synthetic guides, documentation, and metadata to improve agent knowledge bases. You'll work on cutting-edge approaches to knowledge extraction and augmentation.
- Optimize AI agent accuracy: Enhance our retrieval systems, implement advanced prompt optimization techniques, and build tools that continuously improve agent responses through automated evaluation and refinement.
- Develop intelligence infrastructure: Architect systems that enable rapid model upgrades, A/B testing of different AI approaches, and scalable evaluation pipelines that support enterprise deployment.
- Drive volume automation: Focus on the north star goal of maximizing automated resolutions across voice, chat, and email by building the intelligence systems that make it possible.
Other
- Have 5+ years of experience in software engineering as an individual contributor
- Are passionate about building systems that measure and improve AI performance
- Enjoy building tools and infrastructure that enable other engineers and AI systems to perform better
- Are highly ambitious and driven, setting high goals for yourself and others
- Put customers first, focusing on solving real problems that impact support quality
