The ClickPipes Platform at ClickHouse needs to seamlessly onboard and integrate data from diverse data sources into ClickHouse, operating at petabyte scale and in high-velocity environments. This role aims to solve challenging technical problems related to real-time data ingestion and change data capture to ensure the success of clients and drive company growth.
Requirements
- Strong knowledge of Golang and experience with its ecosystem.
- Experience with distributed systems and microservices architecture
- The ability to design and build robust ETL data pipelines that can handle large volumes of data reliably and efficiently.
- Understanding data replication methodologies like CDC.
- Good knowledge of cloud-native architecture and practical experience with at least one major CSP.
- Experience with real-time data ingestion
- Experience with change data capture (CDC)
- Experience with distributed systems
- Experience with cloud services
Responsibilities
- Develop and enhance integrations with various data sources including streaming platforms, databases, data lakes, and object stores.
- Continuously improve our systems based on operational metrics, customer feedback, and evolving business requirements.
- Drive technical discussions and contribute to architectural decisions that impact our platform's scalability and resilience.
- Participate in on-call rotations to ensure system reliability and respond to production incidents.
Other
- 5+ years of relevant software development industry experience building data-intensive software solutions.
- You have excellent communication skills and the ability to work well within a team and across engineering teams.
- You are a strong problem solver and have solid production debugging skills.
- Capable of working independently, taking initiative, and driving projects forward with a high level of autonomy.
- Flexible work environment - ClickHouse is a globally distributed company and remote-friendly.
