Filtr uses AI to scan 1000+ jobs and finds postings that perfectly matches your resume

At Cloudera, we empower people to transform complex data into clear and actionable insights, and we're looking for a Principal Engineer to work on the Cloudera distribution of Apache Spark to help build our enterprise-grade system, designed for customers running Spark on thousands of nodes and processing petabytes of data.

Requirements

• Experience with systems design and development specifically for large-scale distributed environments
• Strong understanding of Java and Scala
• Experience with large-scale, distributed systems design and development with an understanding of scaling, performance, and scheduling
• In-depth understanding of distributed query processing and planning
• Experience with using/developing Apache Spark or other related technologies.
• In-depth understanding of distributed systems concepts like consensus algorithms, distributed transactions, and fault tolerance
• Solid experience with at least one cloud service (AWS, Azure, GCP, OpenShift)

Responsibilities

• Architect, design, and implement resilient and scalable solutions for distributed data processing at massive scale, with a focus on fault tolerance, performance optimization, query planning, and resource management
• Take ownership of critical distributed systems components, solving complex challenges related to network communication, concurrency, data consistency, and system reliability across clusters of thousands of nodes
• Develop advanced monitoring, debugging, and performance analysis tools for large-scale distributed systems.
• Act as a tech lead for Cloudera’s Spark team
• Work with and contribute to the latest open source technologies, including Apache Spark, Iceberg, and Parquet
• Develop new features in Scala/Java a modern platforms
• Debug system level deployment issues, root cause analysis, perform system test analysis and resolve failures

Other

• Bachelor’s degree in Computer Science or equivalent, and 10+ years of experience; OR Master’s degree and 6+ years of experience; OR PhD and 4+ years of experience
• Strong oral and written communication skills.
• Strong ability to research and solve problems independently without constant supervision
• Passionate about programming, clean coding habits, attention to detail, and focus on quality
• This role is not eligible for immigration sponsorship