Apple is seeking to enhance its data-driven decision-making capabilities across various products and organizations, including Siri, Search, Media, iCloud, News, Apple Intelligence, and Maps, by building and operating a managed Lakehouse platform.
Requirements
- Good understanding of open source data processing frameworks like Apache Spark, Apache Flink, Apache Iceberg, and REST Catalog to operate Lakehouse
- Proven expertise in operating large Lakehouse leveraging Spark, Iceberg, REST Catalog
- Experience building and growing teams focussed on Lakehouse concepts
- Strong problem-solving and analytical skills
- Experience with storage and compute architectures, metadata management, and scalable data processing frameworks
- Knowledge of Lakehouse principles
- Experience with Apache Spark, Iceberg, REST Catalog
Responsibilities
- Define and drive the technical roadmap for our Lakehouse, ensuring alignment with Apple’s overall data strategy and advancing the adoption of open formats and governance standards
- Drive innovation in data processing techniques by exploring emerging technologies and approaches to enhance performance, scalability, and efficiency across the Lakehouse architecture
- Lead the design, development, and deployment of robust, scalable Lakehouse leveraging Apache Spark, Apache Iceberg, and the REST Catalog for metadata management and interoperability
- Ensure the reliability, availability, and security of the Lakehouse infrastructure to support both analytical and operational workloads
- Collaborate with other engineering teams to integrate our data processing technologies with other Apple systems and services
- Build, mentor, and manage a high-performing team of software engineers with focus on Lakehouse, and related technologies
- Partner with data scientists, data engineers, product managers, and other stakeholders to understand their data processing needs and requirements
Other
- 8+ years of software development experience, with a focus on batch and streaming data processing at scale
- 4+ years of engineering people management experience; leading, mentoring, and growing teams
- Excellent communication, interpersonal, and leadership skills
- BS, MS, or Ph.D. in Computer Science or a related field of study
- Ability to work with cross-functional teams
