Capco's Data Team helps clients transform their business by formulating data strategy, defining data management initiatives, and aligning roadmaps with business goals. This role focuses on designing and implementing data science and advanced analytics capabilities on Microsoft Azure Fabric and Databricks to unify fragmented data into a trusted, governed, and analytics-ready model.
Requirements
- 3+ years building and managing pipelines in Azure Databricks (PySpark, Delta Lake, MLflow).
- 2+ years hands-on experience with Microsoft Fabric (Data Factory, Dataflow Gen2, Data Warehouse).
- Power BI integration and data modeling
- Entity resolution and master data management (MDM) methods
- Statistical modeling, clustering, and record linkage algorithms
- Data governance, lineage tracking, and compliance (PII, HIPAA, etc)
- Strong background in SQL, Python, and large-scale data processing for analytics.
Responsibilities
- Design and develop data lakehouse and warehouse structures within Azure Databricks and Fabric environments.
- Build ETL and ELT pipelines to extract, cleanse, normalize, and enrich data from CRM, ERP, LMS, and financial systems .
- Develop reusable data transformation and validation frameworks leveraging PySpark, SQL, and Delta Live Tables.
- Implement entity resolution models to unify customer, member, or participant records across systems using deterministic and probabilistic matching techniques.
- Design and deploy matching algorithms utilizing Databricks MLflow, PySpark, and Azure Machine Learning for cross-system deduplication and linkage.
- Develop and schedule data ingestion pipelines in Azure Fabric and Databricks for recurring Excel, CSV, and structured PDF sources using Power Automate, Form Recognizer, and Fabric Dataflows.
- Provide curated and feature-engineered datasets for Power BI dashboards and machine learning use cases.
Other
- BA in Data Science, Computer Science, Applied Mathematics, or related discipline.
- 5+ years of experience in data engineering and applied data science on Azure platforms.
- Proven track record implementing identity resolution and entity linking frameworks.
- Microsoft Certified: Fabric Analytics Engineer Associate
- Microsoft Certified: Azure Data Scientist Associate
