AECOM is seeking a Lead Data Engineer to guide the design, development, and optimization of enterprise-scale data pipelines and products, integrating and transforming large volumes of data from multiple enterprise systems into reliable, accessible, and high-quality data products that power analytics, reporting, and decision-making across the organization.
Requirements
- Deep expertise in AWS data and analytics services: e.g.; S3, Glue, Redshift, Athena, EMR/Spark, Lambda, IAM, and Lake Formation.
- Proficiency in Python/PySpark or Scala for data engineering, along with advanced SQL for warehousing and analytics workloads.
- Demonstrated success designing and operating large-scale ELT/ETL pipelines, data lakes, and dimensional/columnar data warehouses.
- Experience with workflow orchestration (e.g.; Airflow, Step Functions) and modern DevOps practices—CI/CD, automated testing, and infrastructure-as-code (e.g.; Terraform or CloudFormation).
- Experience with data lakehouse architecture and frameworks (e.g.; Apache Iceberg).
- Solid understanding of data modeling, data governance, security best practices (encryption, key management), and compliance requirements.
- Proficiency in modern data storage formats and table management systems, with a strong understanding of Apache Iceberg for managing large-scale datasets and Parquet for efficient, columnar data storage.
Responsibilities
- Lead the end-to-end design, development, and optimization of scalable data pipelines and products on AWS, leveraging services such as S3, Glue, Redshift, Athena, EMR, and Lambda.
- Provide day-to-day technical leadership and mentorship to a team of data engineers—setting coding standards, reviewing pull requests, and fostering a culture of engineering excellence.
- Partner with data architects to define target data models, integration patterns, and platform roadmaps that align with AECOM’s enterprise data strategy.
- Own project planning, estimation, resourcing, and sprint management for major data initiatives, ensuring on-time, on-budget delivery.
- Implement robust ELT/ETL frameworks, including orchestration (e.g., Airflow or AWS Step Functions), automated testing, and CI/CD pipelines to enable rapid, reliable deployments.
- Champion data quality, governance, and security; establish monitoring, alerting, and incident-response processes that keep data products highly available and trustworthy.
- Optimize performance and cost across storage, compute, and network layers; conduct periodic architecture reviews and tuning exercises.
Other
- Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related discipline plus at least 8 years of hands-on data engineering experience, or demonstrated equivalency of experience and/or education
- 3+ years in a technical-lead or team-lead capacity delivering enterprise-grade solutions.
- Strong communication, stakeholder-management, and documentation skills; aptitude for translating business needs into technical roadmaps.
- Relocation assistance is not available for this position
- Sponsorship for US work authorization is not available for this position, now or in the future.
