Guidehouse is seeking an experienced AWS Data Engineer to design, build, and maintain robust data pipelines and data processing solutions in the AWS cloud environment to migrate data from legacy applications to the AWS cloud platform.
Requirements
- Strong expertise in AWS data services, including S3, Glue, DMS, Athena, Redshift, EMR, Kinesis, and Lambda
- Proficiency in programming languages, including Python, Scala, or Java, for data processing and pipeline development
- Experience with SQL and working knowledge of both relational databases (PostgreSQL, Oracle) and NoSQL systems (DynamoDB, DocumentDB)
- Understanding of data modeling concepts for both transactional and analytical workloads
- Experience with Infrastructure as Code tools (Terraform, CloudFormation, CDK) and CI/CD pipelines for data engineering workflows
- Familiarity with big data frameworks such as Apache Spark, Apache Airflow, or AWS-native orchestration tools
- Knowledge of data formats (JSON, Avro, Parquet, ORC) and compression techniques for optimal storage and processing
Responsibilities
- Design and implement scalable data pipelines using AWS services including AWS Glue, Step Functions, Lambda, and Kinesis.
- Build ETL/ELT processes to ingest, transform, and load data from various sources into data warehouses and data lakes.
- Architect and maintain data lakes using S3, implement data cataloging with AWS Glue Catalog, and optimize data storage formats (Parquet, Delta, etc.).
- Design data warehouse solutions using Redshift and integrate with existing RDS/Aurora databases managed by the DBA team.
- Develop real-time and batch data processing solutions using Kinesis Data Streams, Kinesis Analytics, EMR, and AWS Batch.
- Create and maintain data models, schemas, and documentation.
- Build automated data quality checks, monitoring, and alerting systems.
Other
- US Citizenship or Green Card Holder
- Must be able to OBTAIN and MAINTAIN a Federal or DoD 'PUBLIC TRUST'
- Bachelor’s degree in computer science, Data Engineering, or related field. A Minimum of NINE (9) years of experience can be used in lieu of degree.
- Ability to travel up to 10%
- Ability to Obtain Public Trust
