The partner company is looking for a Data Engineer III to design, implement, and maintain large-scale, secure, and high-performance data infrastructure and analytics systems, tackling complex data challenges and optimizing infrastructure for scalability and reliability.
Requirements
- Advanced proficiency in SQL and programming languages such as Python, Scala, or Java.
- Experience with big data frameworks (Apache Spark, Hadoop), workflow orchestration (Airflow, Step Functions), and streaming technologies (Kafka, Kinesis).
- Proficiency in cloud platforms (AWS, Azure, or GCP), containerization (Docker, Kubernetes), and Infrastructure-as-Code (Terraform).
- Expertise in relational databases, data warehousing (Redshift, Snowflake, BigQuery), and NoSQL systems (MongoDB, DynamoDB).
- Knowledge of data security, encryption, and compliance frameworks (NIST, FedRAMP).
- MLOps experience, real-time analytics, geospatial or graph databases, and government/intelligence data systems exposure.
Responsibilities
- Design and build automated data pipelines for ingestion, transformation, and standardization from diverse sources, including streaming platforms.
- Optimize pipeline performance and reliability, developing reusable components and frameworks.
- Manage cloud data infrastructure using Infrastructure-as-Code principles and ensure high availability, backup, and disaster recovery strategies.
- Design, implement, and optimize database schemas for transactional, analytical, and NoSQL workloads.
- Collaborate with analysts and data scientists to build analytical datasets and support ML model deployment.
- Implement automated testing, data quality checks, monitoring, and alerting systems to ensure data integrity and freshness.
- Mentor junior engineers and contribute to establishing best practices and technical standards.
Other
- Ability to obtain or hold a Secret security clearance (US citizenship required).
- 5–7 years of hands-on data engineering experience, with 2–3 years in a senior or technical lead capacity.
- Strong analytical, problem-solving, and communication skills, with a commitment to continuous learning.
- Fully remote work with flexible schedule options.
- Work in a diverse, inclusive, and mission-driven environment.
