Apple Data Platform team powers analytics and machine learning across Apple. Our mission is to deliver cutting-edge, reliable, and user-friendly infrastructure for data ingestion, storage, processing, and interaction, enabling teams to build data-intensive applications with ease and success. This role sits within the Data Lakehouse team, a part of the Apple Data Platform. The team provides a fully managed Lakehouse solution powered by Apache Iceberg, Iceberg REST Catalog, and custom-built services to manage data at scale.
Requirements
- Good understanding of open source data processing frameworks like Apache Spark, Apache Flink, Apache Iceberg, and REST Catalog to operate Lakehouse
- Proven expertise in operating large Lakehouse leveraging Spark, Iceberg, REST Catalog
- 8+ years of software development experience, with a focus on batch and streaming data processing at scale.
- Good understanding of Lakehouse principles, with emphasis on Apache Spark, Iceberg, REST Catalog, along with experience in storage and compute architectures, metadata management, and scalable data processing frameworks to ensure high performance, reliability, and governance at scale.
Responsibilities
- Define and drive the technical roadmap for our Lakehouse, ensuring alignment with Apple’s overall data strategy and advancing the adoption of open formats and governance standards
- Drive innovation in data processing techniques by exploring emerging technologies and approaches to enhance performance, scalability, and efficiency across the Lakehouse architecture
- Lead the design, development, and deployment of robust, scalable Lakehouse leveraging Apache Spark, Apache Iceberg, and the REST Catalog for metadata management and interoperability
- Ensure the reliability, availability, and security of the Lakehouse infrastructure to support both analytical and operational workloads
- Collaborate with other engineering teams to integrate our data processing technologies with other Apple systems and services
- Build, mentor, and manage a high-performing team of software engineers with focus on Lakehouse, and related technologies
- Provide technical guidance and mentorship to team members, promoting their professional growth
Other
- Lead the data lakehouse team, guiding team members in both technical and professional growth.
- Collaborate closely with cross-functional teams to lead the planning, execution, and success of technical projects, with a focus on enhancing the experience for Apple’s internal customers.
- Build, mentor, and manage a high-performing team of software engineers with focus on Lakehouse, and related technologies
- Foster a culture of collaboration, innovation, and continuous improvement
- 4+ years of engineering people management experience; leading, mentoring, and growing teams.
