Amazon Nova family of models requires advancements in accuracy and efficiency for Artificial General Intelligence (AGI) systems, specifically in the areas of inference and evaluation.
Requirements
- 5+ years of programming with at least one software programming language experience
- 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- 5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
Responsibilities
- Lead the development and maintenance of key software components for inference and evaluation for Nova family of models.
- Design and implement solutions that scale from running single evaluations to a whole suite of evaluations on a regular cadence for multiple models
- Propose strategies to optimize the use of compute infrastructure to manage costs
- Champion best practices for maintaining a high bar on operational excellence
- Illuminate trade-offs, educate the team on best practices, and influence technical strategy.
- Collaborate closely with Applied Scientists to investigate and plan experiments, prototype new technology, and evaluate technical feasibility.
- Automate hosting of multimodal LLMs and run evaluations against industry standard as well as customized benchmarks to generate model cards, identify gaps in model performance, and inform strategies to bridge those gaps.
Other
- Manage project prioritization, deliverables, and timelines.
- Communicate effectively with both technical and business partners.
- Operate in a dynamic environment to deliver high quality software against competitive schedules with rapidly-evolving priorities.
- Experience as a mentor, tech lead or leading an engineering team
- If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information.
