The Data Engineer III - Databricks is responsible for engineering, architecting, developing, and implementing unique and innovative enterprise-wide solutions, with a primary responsibility of driving cloud solutions utilizing Azure DevOps Data Bricks.
Requirements
- Strong experience in Python and cloud-to-cloud data integration.
- Proficiency in ETL/ELT processes is essential.
- Familiarity with NoSQL databases is a plus
- Understanding of Data Vault and Kimball model
- Experience with the following: DataBricks
- Experience with the following: PythonPySpark
- Experience with the following: Azure Data Factory
Responsibilities
- Primary responsibility will be driving the cloud solution utilizing Azure DevOpsData Bricks.
- Key contributor for data stewardship workgroups and teams in data management and governance activities
- Be a champion or ambassador of metadata and data quality practices across the organization – leading lunch ‘n’ learn sessions, knowledge sharing sessions, etc.
- Gain an understanding of the stakeholder’s business culture and priorities, and the need for different modeling techniques when eliciting definitions and data quality requirements
- Perform analysis and research to support and/or develop glossary definitions, policies, programs, standards, guidelines and workflows
- Lead the development and maintenance of metadata catalogs
- Identify and quantify data related pain points within the organization and assist in the development of remediation plans
Other
- Bachelor’s degree in Computer Science or related field or equivalent years of relevant work experience is required
- A minimum of three (3) years of healthcare payer experience is required
- A minimum of five (5) years of experience designing and developing ETL solutions
- Demonstrable experience in all stages of definition, design, implementation, testing and deployment of complex database, ETL, BI and Analytical solutions is required
- Participation and experience working within a formal Data Warehouse and Governance environment is preferred
