The company is looking to shape, optimize, and maintain advanced AI-driven features and user experiences within its product suite, ensuring outputs are accurate, trustworthy, and closely aligned with client needs.
Requirements
- Experience with AI/ML applications (LLMs, NLP, machine learning pipelines)
- Strong understanding of prompt engineering and generative AI interaction design
- Basic knowledge of data modeling, relational database concepts, and SQL
- Familiarity with model evaluation techniques and performance metrics
- Proficiency with Python or JavaScript for prototyping and evaluation
- Experience with Agile SCRUM development processes
- Familiarity with RAG (retrieval-augmented generation), embeddings, MLOps, and responsible AI practices
Responsibilities
- Design, evaluate, and refine prompts, templates, and conversation flows for reliability, cost effectiveness, and user alignment.
- Maintain prompt libraries, track changes, and manage versioning.
- Systematically test AI model outputs for accuracy, consistency, compliance, and tone, applying rerankers, guardrails, and validation layers to ensure business rules are met.
- Collaborate with engineering and QA to promptly integrate feedback and resolve model drift or new scenarios.
- Proactively gather requirements of moderate scope and complexity from diverse sources (clients, market analysis, SMEs) to drive product development and AI enhancements.
- Establish systematic evaluation frameworks for AI model outputs, leveraging quantitative and qualitative benchmarks.
- Run experiments, A/B tests, and structured evaluations to compare tuning strategies and track performance metrics (accuracy, coverage, hallucination rates, latency)
Other
- Bachelor’s degree in Computer Science, Management Information Science, Business, or related field OR equivalent work experience
- 1+ years’ experience in software, AI/ML, or related product analysis roles
- Excellent planning, organization, and interpersonal communication skills
- Ability to work independently and collaboratively across functional groups
- Experience in ERP systems, enterprise software, or complex business domains
