Central Technology Services, a division of Central Bank, is looking to modernize its legacy data ingestion, ETL/ELT, and databases to cloud technologies (AWS/Azure) to drive analytics and business intelligence capabilities. They need to implement data access, classification, and security patterns that comply with regulatory standards and integrate data from multiple sources into cloud-based architectures.
Requirements
- 3+ years of experience in designing and implementing data warehouse and analytics solutions (on-premise and cloud).
- 3+ years of expertise in data warehousing concepts (ETL/ELT, data quality management, privacy/security, MDM) with hands-on experience using ADF, Data Factory, SSIS, and related tools.
- 3+ years of experience with cloud data and cloud-native data lakes/warehouses. Microsoft Azure services (Fabric Lakehouse, ADF, Data Factory, Synapse, etc.).
- 2+ years of experience in Python, Scala, or Java for use with distributed processing and analytics, such as Spark.
- Familiarity with CI/CD practices and tools such as Azure DevOps, Git, or Jenkins.
- Experience with Snowflake, Databricks, AWS
- Experience with containerization, microservices, streaming, and event-sourcing architecture patterns.
Responsibilities
- Provide technical leadership in modernizing legacy data ingestion, ETL/ELT, and databases to cloud technologies (AWS/Azure).
- Implement data access, classification, and security patterns that comply with regulatory standards (PII, locational data, contractual obligations, etc.).
- Integrate data from multiple sources into cloud-based architectures, collaborating with cross-functional teams.
- Work closely with data scientists, analysts, and stakeholders to meet data requirements with high-quality solutions.
- Design and map ETL/ELT pipelines for new or modified data streams, ensuring integration into on-prem or cloud-based data storage.
- Automate, validate and maintain ETL/ELT processes using technologies such as Databricks, ADF, SSIS, Spark, Python, and Scala.
- Conduct unit, system, and integration testing for ETL/ELT solutions, ensuring defects are resolved.
Other
- Demonstrate a self-driven, ownership mindset to navigate ambiguity, resolve constraints, and mitigate risks with minimal supervision.
- Build strong relationships with technical teams through effective communication, presentation, and collaboration skills.
- Collaborate with stakeholders, business analysts, and SMEs to translate business requirements into scalable solutions.
- Function within a matrixed team environment, sharing responsibilities across various teams.
- Proven ability to mentor team members and guide best practices for data engineering.
