Carnegie Mellon University’s Temporary Employment Services (TES) is seeking a Temporary Data Engineer to support the Delphi Group in the Machine Learning Department in its day-to-day functions, specifically building, maintaining, and optimizing data pipelines.
Requirements
- Airflow Experience: Confirmed experience building and managing Directed Acyclic Graphs (DAGs) in Apache Airflow 2.0 and 3.0.
- Technical Expertise: Strong proficiency in programming (Python preferred) and SQL.
- System Design: Ability to develop applications from the ground up based on complex specifications.
Responsibilities
- Orchestration: Design, develop, and run end-to-end workflows using Apache Airflow 2.0 and 3.0 to ensure efficient data processing.
- Pipeline Development: Modify and develop complex data applications and system programs based on detailed technical specifications.
- Quality Assurance: Code, test, and debug programs to ensure data integrity and system reliability.
- Documentation: Build and maintain comprehensive user documentation for system architectures and ETL processes.
- Continuous Improvement: Monitor system performance and perform related duties to ensure high availability of data assets.
Other
- Associate's degree required
- A combination of education and relevant experience from which comparable knowledge is demonstrated may be considered.
- Successful completion of a background check.
- Applicants for this position must be currently legally authorized to work for CMU in the United States.
