Enabling the collection, organization, and accessibility of Competitive Intelligence data to help the team deliver key insights, and trends, in shorter periods of time through automation.
Requirements
- Extensive hands-on expertise with SSIS and Azure Data Factory, architecting ETL workflows with incremental loads, SCDs, and CDC pipelines
- Strong ability to design and enforce schema architectures, including star/snowflake models, surrogate keys, and conformed dimensions
- Demonstrated success in optimizing Synapse/SQL Server workloads through indexing, partitioning, and performance tuning
- Applied fluency in OCR technologies across static image and video media using Azure AI Video Indexer and/or open-source Python libraries
- Understanding of governance frameworks, including Microsoft Purview deployment for lineage, RBAC, metadata cataloguing, and retention compliance
- Experience with enforcing enterprise-grade data quality via SQL procedures, validation scripts, and automated pipeline alerts
- Capable of applying advanced web scraping techniques to expand datasets with market and competitive intelligence. For example, familiarity with web scraping & crawling using Python libraries like Beautiful Soup or Scrapy.
Responsibilities
- You will lead comprehensive data engineering initiatives to architect robust solutions aligned with client objectives.
- You'll apply OCR to static images (JPG, PNG, GIF) and video frames (MP4) via software, e.g. Azure AI Video Indexer, as well as Python libraries to extract, index, and tag metadata for integration into warehouses.
- You'll architect advanced ETL workflows in SQL Server Integration Services (SSIS) and/or Azure Data Factory, including incremental loads, slowly changing dimensions, and CDC (Change Data Capture).
- You will normalize disparate data sources by enforcing conformed dimensions, applying surrogate key management, and establishing star/snowflake schemas for downstream analytics.
- You'll integrate structured and unstructured datasets ensuring proper indexing, partitioning, and query optimization for workloads.
- You'll implement data quality monitoring using stored procedures, validation rules, and alerting pipelines to ensure referential integrity and schema compliance.
- Govern data assets, e.g. implementing Microsoft Purview, defining lineage, metadata cataloguing, retention schedules, and Role-Based Access Control aligned to compliance standards as applicable.
Other
- 8+ years of experience in data engineering, BI, or related domains
- Advanced academic qualification in Computer Science (B.S., M.Sc., or Ph.D.) or equivalent experience
- This will be a remote position reporting to the SVP of Strategy & Insight.
- Placement within the salary range is based on a variety of factors, including relevant experience, knowledge, skills, and other factors permitted by law.
- Dentsu is committed to providing equal employment opportunities to all applicants and employees.
