Condé Nast is seeking a seasoned Data Engineer to build and maintain dimensional data models across various business areas to enable data scientists, analysts, and business intelligence engineers to solve challenging use cases within the company's data infrastructure.
Requirements
- 4+ years experience of near Real Time (Streaming) & Batch Data Pipeline development in a large scale organization
- Experience in writing reusable/efficient code to automate analysis and data processes
- Experience in processing structured and unstructured data into a form suitable for analysis and reporting with integration with a variety of data metric providers ranging from web analytics, consumer analytics, and advertising
- Expertise in data modelling, batch data pipeline design and implementation
- Strong Experience in software development and engineering principles
- Experience implementing scalable, distributed, and highly available systems using AWS services such Kinesis, DynamoDB, S3
- High Proficiency in Python/PySpark, Scala or Java
- High Proficiency in SQL
- Experience with Databricks/Spark
- Experience with orchestration tools such as Airflow (we use Astronomer)
- Comfortable with CI/CD (we use GitHub Actions) Pipelines
- Experience with Git version control, and other software adjacent tools
- Terraform used as Infra as service tool.
Responsibilities
- Build and maintain dimensional data models across business areas such as subscriptions, video, clickstream, commerce, social and advertising within Condé Nast.
- Build, test, scale and maintain highly reliable dimensional data models from a variety of batch data sources and real-time streams
- Build and maintain highly validated data-marts with ensured clarity and correctness of key business metrics for BI reporting purposes
- Collaborate with other Data Engineers, Data Scientists, and BI Engineers to architect and implement a shared technical vision
- Contribute to the data infrastructure and platform used to build our data pipelines
- Follow agile processes with a focus on delivering production-ready, testable deliverables in an iterable fashion
- Serve as a senior technical contact for the data solutions engineering team
Other
- Understand the company data assets, and engage with business stakeholders, the analytics team, the Business Intelligence team and other data consumers to translate their data needs into efficient, effective and scalable dimensional data models
- Serve as a core member of the data engineering team and be proficient in assisting the business with understanding data attributes, data models, cardinality, granularity, etc
- Design and present recommendations to guide future business and data directions
- BS, MS, Ph.D., or equivalent industry experience in Computer Science, Software Engineering, or other related Science/Technology/Engineering/Math fields.
- 3+ of business/marketing analytics experience, preferably in a consumer-based organisation
- Experience successfully working on an independent project with very minimal supervision
- Exceptional communication skills, particularly in communicating and visualizing quantitative findings in a compelling and actionable manner for business stakeholders
- Experience in mentoring and supporting junior members of the team
