The Data Engineer II is responsible for contributing to team and department solution design architecture, and the development and implementation of appropriate standards and strategies for all sources of data.
Requirements
- A minimum of three (3) years of experience designing and developing ETL and/or BI solutions is required
- Demonstrable experience in all stages of definition, design, implementation, testing and deployment of complex database, ETL, BI and Analytical solutions is required
- Proven experience in Azure DataBricks, Azure Data Factory
- Proven experience in Python, PySpark
- Proven ability to write complex SQL queries
- Proven ability developing medium to complex data quality profiling or metadata analysis processes with an industry leading tool
- Solid understanding of data management and governance concepts, especially data quality and metadata management
Responsibilities
- Support and maintain operational availability and accuracy of enterprise data
- Work within an interdisciplinary group to develop, test, and deliver data solutions to business stakeholders
- Perform analysis and research to support and/or develop data transformation and workflows
- Collaborate with SMEs to develop business rules that measure and assure data quality; facilitating discussions with end users and data modelers
- Lead efforts to quantify and qualify reported data quality issues; develop and report program metrics to demonstrate progress and compliance
- Sets day-to-day objectives and delivers job responsibilities that may vary outside typical norms or practices.
- Work consists of tasks that are occasionally not routine and often works independently and is required to apply discretion within established operational boundaries and procedures.
Other
- Healthcare payer experience is preferred
- Ability to manage multiple medium to large scale projects while demonstrating a sense of urgency
- Proven understanding of relational database theory
- Ability to establish effective working relationships with stakeholders at all different levels
- Effective problem-solving skills with attention to detail
