NVIDIA is seeking to advance AI research and development to create groundbreaking technologies that enable anyone to harness the power of AI and benefit from its potential by defining and building benchmarks for MLPerf Inference and optimizing vLLM performance on NVIDIA's latest GPUs
Requirements
- Deep understanding of deep learning algorithms, distributed systems, parallel computing, and high-performance computing principles
- Hands-on experience with ML frameworks (e.g., PyTorch) and inference engines (e.g., vLLM and SGLang)
- Experience optimizing compute, memory, and communication performance for the deployments of large models
- Familiarity with GPU programming, CUDA, NCCL, and performance profiling tools
- Background with building and optimizing LLM inference engines such as vLLM and SGLang
- Experience building ML compilers such as Triton, Torch Dynamo/Inductor
- Experience working with cloud platforms (e.g., AWS, GCP, or Azure), containerization tools (e.g., Docker), and orchestration infrastructures (e.g., Kubernetes, Slurm)
Responsibilities
- Design and implement highly efficient inference systems for large-scale deployments of generative AI models
- Define inference benchmarking methodologies and build tools that will be embraced across the industry
- Develop, profile, debug, and optimize low-level system components and algorithms to enhance the throughput and the latency for the MLPerf Inference benchmarks on the newest NVIDIA GPUs
- Productionize inference systems with uncompromised software quality
- Collaborate with researchers and engineers to productionize trending model architectures, inference techniques and quantization methods
- Contribute to the design of APIs, abstractions, and UX that make it easier to scale model deployment while maintaining usability and flexibility
- Participate in design discussions, code reviews, and technical planning to ensure the product aligns with the business goals
Other
- Bachelor’s, Master’s, or PhD degree in Computer Science/Engineering, Software Engineering, a related field, or equivalent experience
- 5+ years of experience in software development, preferably with Python and C++
- Ability to work closely with both research and engineering teams, translating pioneering research ideas into concrete designs and robust code
- Excellent problem-solving skills, with the ability to debug sophisticated systems
- A passion for building high-impact software that pushes the boundaries of what’s possible with large-scale AI
