Duke Health is looking to advance health together by transforming, developing, and managing enterprise information technology solutions across Duke Health, and the Data Engineer role is crucial in achieving this goal by designing, developing, and optimizing data pipelines, managing servers, and integrating APIs.
Requirements
- basic understanding of local databases (Clarity; Caboodle) and underlying tables
- basic understanding of ETL technologies and scripting languages
- basic understanding of cloud data engineering technologies, including storage accounts, data pipelines, data lakes, structured vs. unstructured data
- knowledge of SQL and relational databases
- experience with Microsoft Azure Lakehouses and Delta Tables, Azure Data Factory, Azure Synapse, and medallion-style data architecture
- ability to write/modify accurate SQL queries
- ability to optimize SQL queries, stored procedures and scripts for performance reliability
- preferred certification in development technologies
Responsibilities
- designing, developing, and optimizing data pipelines
- managing servers
- integrating APIs
- collaborating closely with data scientists and analysts to meet data requirements
- ensuring robust data management solutions
- assisting in server administration tasks
- participating in code review and learning best practices in data engineering
- maintaining ongoing communications with manager and stakeholders
Other
- Bachelor's degree in a related field, or four years of equivalent technical experience required
- two years of directly related experience is required for Level 2
- four years of directly related experience is required for Level 3
- ability to work with stakeholders to understand business needs and perform business analysis
- ability to manage multiple projects to completion in a timely manner
- ability to develop and present topics as assigned/needed by customer and/or ACE team
