Drive the development, maintenance, and optimization of Azure-based data platform
Requirements
- Extensive hands-on experience with Azure Synapse Analytics, Azure Data Lake (Gen2), SQL Server and Apache Spark
- Strong background in Dev Op’s culture with a deep understanding of source control and infrastructure as code
- Microsoft certifications in Azure data technologies (e.g., Microsoft Certified: Azure Data Engineer Associate, Azure Solutions Architect Expert)
- Expertise in Azure data services, including Azure Synapse Analytics / Azure Data Factory and Azure Data Lake Storage
- Strong proficiency in Apache Spark for big data processing, including Spark SQL, DataFrames, and RDDs, preferably using PySpark
Responsibilities
- Build and maintain efficient, secure, and reliable ELT pipelines using Azure Synapse Analytics and Apache Spark
- Develop and optimize data models for data warehousing and analytics
- Implement data governance frameworks, including data quality, lineage, and cataloging, using Azure Purview or similar tools
- Take ownership of existing Azure Dev Ops pipelines, source code repositories and branching strategies and produce new artefacts when required
- Monitor and optimize the performance of Azure Synapse Analytics, Data Lake, Spark jobs and on-premise SQL Servers
Other
- Bachelor's degree in computer science, Data Science, Information Technology, or a related field (or equivalent work experience)
- 7+ years of experience in data architecture, data engineering, or related roles, with at least 3 years focused on Azure cloud platforms
