Databricks needs to design and build systems that power the company's service-to-service communication across thousands of clusters in a multi-cloud environment, and create abstractions to simplify networking complexity for product teams.
Requirements
- 5+ years of experience designing and building large-scale distributed systems
- Strong proficiency in one or more languages such as Java, Scala, Go, or C++
- Experience with service-oriented architectures and large scale distributed systems
- Familiarity with cloud platforms (AWS, Azure, GCP) and container/orchestration technologies (Kubernetes, Docker)
- Track record of shipping infrastructure that supports mission-critical workloads at scale
- background in service discovery, DNS, load balancing, Envoy, or related networking systems
Responsibilities
- Build the control plane for Databricks’ global ingress layer.
- Enable programming of API gateways with static and dynamic endpoints.
- Simplify service onboarding, and make it easy to expose APIs securely across clouds.
- Design scalable mechanisms for service discovery and load balancing across thousands of clusters.
- Provide networking abstractions so product teams don’t need to worry about underlying connectivity details.
- Build intelligent rate limiting and admission control systems to protect critical services under high load.
- Ensure reliability and predictable performance for both customer-facing and internal workloads.
Other
- BS (or higher) in Computer Science or related field
- Individuals looking for employment at Databricks are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics.
