Pinterest is looking for a staff software engineer to help build and lead the next generation of batch processing platform and infrastructure, working on big data open source technologies at the scale of hundreds of petabytes of data.
Requirements
- 5+ years of experience of building and support large scalable big data infrastructure/platform
- 2+ years of experiences in contributing to open-source Spark
- Deep knowledge of big data technologies (e.g. Spark, Hadoop, Parquet/ORC, Flink)
- Proficiency in one or more programming languages (Java, Scala, Python)
- Experiences in Kubernetes and AWS technologies
Responsibilities
- Lead the strategy and technical direction of Pinterest’s Spark platform
- Improve and customize the internals of open source Spark to meet our challenges at scale and actively participate in open source community
- Build and scale batch processing frameworks and infrastructure to process petabytes-scale datasets
- Work with internal customers on critical business use cases that rely on batch processing
- Provide thought leadership to the entire company on how data should be processed and stored more reliably, quickly and efficiently at scale
- Contribute to the team’s technical vision and long-term roadmap
Other
- 8+ years of industry experience with a proven track record of technical excellence
- Experience in leading cross-team engineering efforts
- This role will need to be in the office for in-person collaboration 1-2 times/quarter and therefore can be situated anywhere in the country.
- This position is not eligible for relocation assistance.
- US based applicants only
