Upstart is building foundational infrastructure that democratizes access to generative AI/LLMs for every product and engineering team across the company. The Core GenAI Platform team is responsible for building a unified interface for securely interacting with large language models, abstracting away the complexity of model selection, prompt orchestration, evaluation, and scaling.
Requirements
- 6+ years of experience in object-oriented software engineering, with a strong systems engineering background
- Proficiency in backend development with Python (leveraging frameworks like FastAPI, Flask, or similar), microservices architecture, and infrastructure tools like Kubernetes, Docker, and Terraform
- Hands-on experience building ML platforms or infrastructure supporting LLMs or inference systems
- Familiarity with observability tools (e.g., Prometheus, Grafana, Datadog) and data processing pipelines
- Full Stack development skills including hands on experience with React or similar
- Proficiency with Kotlin or Java and the Spring framework ecosystem
- Experience with LLM toolchains such as LangChain, LlamaIndex, or OpenAI APIs
Responsibilities
- Build the Core GenAI platform powering generative AI use cases across all of Upstart Engineering.
- Design and implement reliable, observable infrastructure for model inference, prompt orchestration, and data workflows.
- Drive cross-functional adoption of AI tooling by delivering reusable components and intuitive interfaces.
- Increase developer productivity through automation, metrics dashboards, and streamlined GenAI integrations.
- Set the technical roadmap for platform capabilities, balancing innovation with reliability and compliance.
- Partner with ML researchers, product engineers, and design to bridge experimentation and production.
- Ensure the platform adheres to emerging standards of security, fairness, and explainability for LLM systems.
Other
- Proven experience independently leading impactful, cross-functional, multi-quarter projects with mid-large teams
- Strong written and verbal communication skills and a collaborative approach to cross-functional work
- Strong stakeholder management skills
- Ability to influence platform direction and drive alignment across multiple engineering teams
- Awareness of LLM-specific risk domains, including hallucinations, fairness, and data privacy
