Redhorse Corporation is seeking a Data Engineer to support the Defense Intelligence Agency (DIA) in advancing Combatant Command mission integration. This role aims to design and implement robust data solutions to enable data-driven decision-making within the Department of Defense (DoD), focusing on AI/ML and data-driven initiatives.
Requirements
- Experience with big data systems, including Apache Spark and Databricks.
- Experience with ETL processes.
- Experience with Amazon Web Services (AWS), Microsoft Azure, or MilCloud 2.0.
- Experience engineering data pipelines across multiple security domains/classified networks.
- Experience with multiple coding languages, including Python.
- Experience with GitLab and Jira.
- Experience with containerization technologies (Docker, Kubernetes).
Responsibilities
- Support the configuration and ingestion of structured, unstructured, and semi-structured data repositories into capabilities meeting mission partner requirements, supporting a data analytics and DevOps pipeline for rapid delivery of functionality.
- Maintain all operational aspects of data transfers, accounting for the security posture of the underlying infrastructure and supported systems and applications; monitor environmental health using various tracking capabilities.
- Automate configuration management, leverage appropriate tools, and stay current on ETL technologies and services.
- Review architecture and identify areas for automation, optimization, right-sizing, and cost reduction to improve environmental health.
- Apply comprehension of data engineering technologies and services, leveraging database expertise and various data structuring and retrieval approaches; comprehend cloud architectural constructs and support programmatic Cloud environment establishment and maintenance using vendor consoles.
- Engage with multiple functional groups to understand client challenges, prototype new ideas and technologies, create solutions to drive innovation, and design, implement, schedule, test, and deploy full features and components of solutions.
- Identify and implement scalable and efficient coding solutions.
Other
- Frequent travel is required.
- Bachelor’s degree plus 5+ years of experience, or a Master’s degree plus 3+ years of experience.
- Active Top Secret (TS) security clearance with Sensitive Compartmented Information (SCI) eligibility and CI polygraph.
- Familiarity with Agile methodologies.
