DoorDash is looking to solve the problem of designing, developing, and optimizing systems that support real-time ML inference at scale to drive the next generation of their inference platform, ensuring it can handle complex models with low latency, high throughput, and cost efficiency.
Requirements
- Deep familiarity with ML inference, serving ecosystems, and deployment frameworks
- Proficiency in leveraging and extending open-source frameworks such as NVIDIA Triton, TensorRT, ONNX Runtime, or vLLM
- Experience with deep learning frameworks like PyTorch and TensorFlow
- Hands-on experience with Kubernetes, microservice architectures, and large-scale orchestration for inference workloads
- Cloud platform experience (AWS, GCP, Azure) focusing on scaling, observability, and cost optimization
- Strong understanding of hardware acceleration (GPU, TPU, CPU) and heterogeneous hardware management
- Experience with building or operating large-scale ML serving systems
Responsibilities
- Design and develop scalable ML inference serving systems capable of handling complex models at low latency
- Operationalize inference optimizations such as caching, batching, attention mechanisms, and quantization to improve performance and cost-efficiency
- Create abstractions and primitives that enable broad application of serving improvements across multiple workloads
- Leverage and contribute to open-source serving ecosystems, integrating vendor solutions and developing custom extensions as needed
- Implement autoscaling, scheduling, and resource management strategies across heterogeneous hardware platforms
- Ensure system reliability, observability, and security through robust monitoring, alerting, and best practices
- Collaborate with ML engineers, infrastructure teams, external vendors, and open-source communities to evolve the serving stack
Other
- 8+ years of engineering experience
- Excellent collaboration, mentorship, and communication skills
- Ability to make pragmatic decisions balancing performance, reliability, and cost
- Competitive salary with performance-based incentives
- Comprehensive health, dental, and vision insurance plans
