The US federal government needs to improve its data engineering capabilities to better serve its mission and the American people.
Requirements
- 10 years of experience with data engineering
- Experience with data lifecycle engineering
- Experience with development and maintenance of extract, transform and load (ETL) tools and services
- Experience with cloud and on-prem data storage and processing solutions
- Experience with Python, SQL, Spark and other data engineering programming
- Experience with COTS and open source data engineering tools such as ElasticSearch and NiFi
- Experience with processing data within the Agile Lifecycle
Responsibilities
- Develop new tools, code, and services to execute data engineering activities
- Design and optimize Data Pipelines using tools such as Spark, Apache Iceberg, Trino, OpenSearch, EMR cloud services, NiFi and Kubernetes containers
- Ensure the pedigree and provenance of the data is maintained
- Clean and preprocess data to enable access for advanced analytics
- Collaborate with the engineering team, data stewards, and mission partners to aid in getting actionable value out of the data holdings
- Collaborate with software engineers to update, configure, and maintain data services based on the requirements
- Ensure data quality by working with the testing and data quality team to enhance standardization of data conditioning pipelines
Other
- Active TS/SCI with polygraph clearance
- Applicants for employment in the US must have work authorization that does not now or in the future require sponsorship of a visa for employment authorization in the United States
- Bachelor's degree or higher
- Travel may be required
