Databricks is looking to build the Data Intelligence Platform to automate decision-making across the entire company, manage and run Databricks on Databricks at scale, and provide leadership in developing the long-term vision and requirements for the Databricks product.
Requirements
- 15+ years of industry experience building and operating large-scale distributed systems.
- 6+ years of experience providing technical leadership on planet-scale production data systems
- Experience leading architecture efforts for performance-sensitive systems (e.g., latency-critical services, multi-tenant platforms, large-scale indexing pipelines).
- AI first mindset - Your personal productivity is augmented with AI to drive high-impact, technically complex initiatives and code that delivers clear value.
- Build reliable data pipelines and solve data problems using Databricks, our partner’s products, and other OSS tools.
- Develop tooling and infrastructure to efficiently manage and run Databricks on Databricks at scale, across multiple clouds, geographies, and deployment types.
- This includes CI/CD processes, test frameworks for pipelines and data quality, and infrastructure-as-code tooling.
Responsibilities
- Design and run the cross-company Data Intelligence Platform, which contains every business and product metric used to run Databricks.
- Develop tooling and infrastructure to efficiently manage and run Databricks on Databricks at scale, across multiple clouds, geographies, and deployment types.
- This includes CI/CD processes, test frameworks for pipelines and data quality, and infrastructure-as-code tooling.
- Partner with our engineering teams to provide leadership in developing the long-term vision and requirements for the Databricks product.
- Build reliable data pipelines and solve data problems using Databricks, our partner’s products, and other OSS tools.
- Provide early feedback on the design and operations of these products.
- Represent Databricks at academic and industrial conferences & events.
Other
- Strong communication skills and the ability to operate effectively in fast-paced, cross-functional environments.
- A strategic mindset with the ability to align engineering execution with longer-term product and company goals.
- Passion for mentoring and developing other engineers.
- BS/MS/PhD in Computer Science or related majors, or equivalent experience.
- Founded by engineers — and customer obsessed — we leap at every opportunity to tackle technical challenges, from designing next-gen UI/UX for interfacing with data to scaling our services and infrastructure across millions of virtual machines.
