MetroStar is looking to build, deploy, and operate data pipelines and data engineering components to enable data brokering and exchange capabilities across a diverse range of producers and consumers.
Requirements
- 4+ years of proven experience in data engineering, with expertise in designing, developing, and maintaining data ingestion, transformation, and loading pipelines and components
- Demonstrated experience in designing and deploying data pipelines leveraging AWS cloud infrastructure across multiple classification domains (e.g., IL5 to IL6+)
- Experience with Infrastructure-as-Code (IaC) tools, including Terraform, CloudFormation, or Ansible, to automate deployment of data pipeline cloud infrastructure
- Understanding of RMF security principles and hands-on experience implementing security controls for data pipelines in cloud environments
- Strong scripting and programming skills in languages such as Go, Python, and Bash
- Experience with data pipeline tools and technologies such as Nifi, Hadoop, HDFS, and Kafka.
- Experience implementing data pipelines in the Cloudera Data Platform environment is highly preferred.
Responsibilities
- Work with team members to operationalize data pipelines and supporting cloud infrastructure
- Collaborate with external data producers and consumers to obtain and provide data through interfaces such as REST APIs and S3
- Provide day-to-day support of deploying Python-native data pipelines and performing data engineering tasks to enable data brokering and exchange capabilities
- Provide Tier 2/3 troubleshooting and incident resolution support for data pipelines in Production
Other
- Active TS/SCI with CI poly required
- Strong communication skills, with the ability to clearly convey complex technical concepts
- All qualified applicants will receive consideration for employment based on merit and without regard to sex, race, ethnicity, age, national origin, citizenship, religion, physical or mental disability, medical condition, genetic information, pregnancy, family structure, marital status, ancestry, domestic partner status, sexual orientation, gender identity or expression, veteran or military status, status as a protected veteran, or any other status protected by applicable federal, state, local, or international law.
- In compliance with federal law, all persons hired will be required to verify identity and eligibility to work in the United States and to complete the required employment eligibility verification form upon hire.
