At LegitScript, the business problem is to make the internet and payment ecosystems safer and more transparent by identifying which commercial entities play by the rules, and which do not, through the development and implementation of cutting-edge AI solutions.
Requirements
- Familiarity with Large Language Models, Transformers, Vector Embeddings, and other Generative AI techniques in a SaaS environment.
- Strong foundation in Traditional Machine Learning approaches.
- Experience with COTS Generative AI solutions.
- Experience with Weights and Biases platform for measuring and tuning as it relates to Generative AI.
- Strong hands-on experience with MLFlow (Databricks MLFlow experience is a plus).
- Exposure to Mosaic AI (Agents and Vector Search).
- Expertise with Spark/PySpark for large-scale data processing.
Responsibilities
- Implement and optimize AI orchestration frameworks like crewAI and LangChain to manage complex, multi-agent AI workflows and improve task delegation efficiency.
- Develop and maintain a comprehensive testing pipeline using industry-standard tools for experiment tracking, model versioning, and performance visualization.
- Design and execute A/B tests for prompt engineering, comparing different versions to maximize output quality and consistency.
- Establish best practices for prompt templating, version control, and documentation to ensure scalability and maintainability of AI systems.
- Develop a robust evaluation framework for assessing AI model outputs, including automated metrics and human-in-the-loop feedback mechanisms.
- Implement prompt chaining and reasoning techniques to break down complex tasks into manageable sub-tasks for improved accuracy and explainability.
- Integrate AI systems seamlessly with the existing backend and frontend stack, ensuring full-stack implementation and optimization of AI features within the application architecture.
Other
- A degree in computer science or a related field, with a preference for advanced qualifications.
- Exceptional written and verbal communication skills, capable of inspiring and engaging others.
- Self-motivation, integrity, and a track record of delivering impactful results with a high degree of autonomy.
- Agility and adaptability to thrive in fast-paced work environments.
- Visa sponsorship is not available for this position.
