Prodapt is looking for a GCP Data Engineer to design, build, and maintain scalable data pipelines, integrate various data sources, develop data models, and ensure data quality and security for their clients.
Requirements
- Proficiency in Google Cloud Platform services, especially BigQuery, Dataflow, Cloud Storage, and Pub/Sub.
- Strong programming skills in languages such as Python, Java, or SQL.
- Knowledge of data warehousing concepts and experience with data modeling techniques.
- Experience in designing and implementing ETL (Extract, Transform, Load) processes.
- Familiarity with big data tools and frameworks like Apache Spark, Hadoop, or similar technologies.
- Understanding of Continuous Integration and Continuous Deployment practices for data solutions
- GCP certifications (like Google Cloud Professional Data Engineer) preferred
Responsibilities
- Design, build, and maintain scalable data pipelines to process large datasets using GCP services like Dataflow, BigQuery, and Pub/Sub.
- Collaborate with data scientists and analysts to understand their data requirements and integrate various data sources.
- Develop and maintain data models and architecture that support business intelligence initiatives and analytics.
- Implement monitoring and logging of data processes to ensure data quality and optimize performance.
- Work closely with software engineers, data analysts, and other stakeholders to ensure seamless data flow and accessibility.
- Prepare technical documentation for data systems and processes to facilitate knowledge sharing and maintainability.
- Ensure data security and compliance with relevant regulations and best practices in data governance.
Other
- 6-10 years of experience in data engineering or related roles, with a focus on cloud environments.
- Bachelor’s degree in Computer Science, Information Technology, Data Science, or a related field.
- Strong analytical and problem-solving skills.
- Excellent communication and teamwork abilities.
- Adaptability and eagerness to learn new technologies and methodologies
