Agile Defense is looking for a Senior Data Scientist to design, develop, and implement scalable data pipelines and ETL processes, with a focus on data integration and AI applications, to support the nation's most important missions.
Requirements
- Demonstrated experience with data wrangling, data visualization, and analytics, including use of SuperSet and GIS tools.
- Hands-on experience with containerization and orchestration tools such as Docker and Kubernetes.
- Experience in R, C++, Python, JS, Gephi, sklearn, SQL, and other analytical or visualization frameworks.
- Experience with natural language processing, machine learning, network analysis concepts, and vector databases or embedding models for AI applications.
- Understanding of LLM prompt engineering and associated ETL applications.
- Expertise in workflow orchestration, data pipeline automation, and database optimization techniques.
- Working knowledge of Linux environments and version control with Git.
Responsibilities
- Design, develop, and implement scalable data pipelines and ETL/ELT processes using Apache Airflow, integrating batch and streaming techniques to enhance data availability and accessibility.
- Develop and optimize Python-based data processing solutions, with strong programming skills in Python and proficiency in multiple scripting languages.
- Develop messaging solutions utilizing Kafka to support real-time data streaming and event-driven architectures, including microservices development.
- Build and maintain high-performance data retrieval and indexing solutions using ElasticSearch/OpenSearch, ensuring efficient data storage, retrieval, and processing.
- Deploy and manage cloud-based infrastructure and services (e.g., AWS, MinIO) to support scalable and resilient data solutions.
- Work closely with cross-functional teams to define data strategies, develop technical solutions aligned with mission objectives, and ensure adherence to security and compliance requirements for classified data.
Other
- Active TS/SCI Clearance
- 7–10 years of relevant experience.
- Employees of Agile Defense are our number one priority, and the importance we place on our culture here is fundamental.
- Employees are valued as a family member, and we believe that we can accomplish great things together.
- Employees are expected to embody the 6Hs: Happy, Helpful, Honest, Humble, Hungry, Hustle.
