NTT DATA is looking to fill a Senior Data Engineer & Technical Lead (SDET Lead) role to address major data engineering initiatives within the Data & Advanced Analytics space. This involves building, deploying, and maintaining robust data pipelines and implementing CI/CD processes for data engineering projects.
Requirements
- Minimum 5+ years of practical experience building production-grade data pipelines using Python and PySpark.
- Proven track record of designing, deploying, and managing Airflow DAGs in enterprise environments. Require 5+ years of experience
- Ability to build and maintain CI/CD pipelines for data engineering workflows, including automated testing and deployment. Require 5+ years of experience
- Experience with containerization (Docker and cloud platforms (GCP) for data engineering workloads. Appreciation for twelve-factor design principles Require 5+ years of experience
- Ability to write object-oriented Python code manage dependencies and follow industry best practices . Require 5+ years of experience
- Proficiency with Git for source code management and collaboration (commits, branching, merging, GitHub/GitLab workflows). Require 5+ years of experience
- Strong command-line skills in Unix-like environments. Require 3+ years of experience
Responsibilities
- Design, develop, and optimize scalable data pipelines using Python and PySpark for batch and streaming workloads.
- Build, schedule, and monitor complex workflows using Airflow, ensuring reliability and maintainability.
- Architect and implement CI/CD pipelines for data engineering projects using GitHub, Docker, and cloud-native solutions.
- Apply test-driven development (TDD) practices and automate unit/integration tests for data pipelines.
- Implement secure coding best practices and design patterns throughout the development lifecycle.
- Create and maintain technical documentation, including process/data flow diagrams and system design artifacts.
- Analyze and resolve technical issues across the data stack, including pipeline failures and performance bottlenecks.
Other
- Hybrid/Onsite in Auburn Hills, MI (3-Days per Week Onsite)
- 6+ Months Duration
- W2 ONLY – Must be able to work directly with NTT Data | NO C2C
- Work closely with Data Architects, QA teams, and business stakeholders to translate requirements into technical solutions.
- Lead and mentor junior engineers, providing guidance on coding, testing, and deployment best practices.
