Capgemini is looking for a Senior AWS Data Engineer to lead the design and development of robust ETL/ELT data pipelines, architect scalable data architectures, and implement data quality and governance practices to enable data-driven decision-making.
Requirements
- Strong/expert Spark (PySpark) Using Jupyter Notebooks, Colab or DataBricks (preferred)
- Hands-on data pipeline development, ingest patterns in Azure
- Orchestration tools, ADF or Airflow
- SQL
- Denormalized Data modeling for big data systems
Responsibilities
- Lead the design and development of robust ETL/ELT data pipelines, ensuring efficient data ingestion, processing, and transformation from diverse sources into AWS data warehouses and data lakes. This includes designing and implementing solutions for batch and streaming data, handling various data formats like JSON, CSV, Parquet, and Avro.
- Architect, build, and optimize scalable data architectures, including data lakes (e.g., S3, Delta Lake, Iceberg) and data warehouses (e.g., Redshift, Snowflake) on AWS, ensuring optimal performance and data accessibility.
- Collaborate closely with data scientists, analysts, and other stakeholders to understand data requirements, design appropriate data models and schemas, and deliver tailored data solutions that enable data-driven decision-making.
- Implement advanced data quality and governance practices, ensuring data accuracy, consistency, and compliance with relevant regulations.
- Optimize data retrieval and develop dashboards and reports using various tools, leveraging deep understanding of data warehousing and analytics concepts.
- Proactively identify and resolve operational issues, troubleshoot complex data pipeline failures, and implement evolutionary recommendations for system improvements.
Other
- Bachelor’s degree in computer science, management information systems, or related discipline, or equivalent work experience
- Applicants for employment in the US must have valid work authorization that does not now and/or will not in the future require sponsorship of a visa for employment authorization in the US by Capgemini.
- Flexible work
- Healthcare including dental, vision, mental health, and well-being programs
- Financial well-being programs such as 401(k) and Employee Share Ownership Plan
