Comcast, NBCUniversal, and Sky's Media Group is looking to create a connected data ecosystem that unleashes the power of their streaming data, with a focus on international propositions, to enable fast feedback loops across territories and support intelligent customer and business decisions.
Requirements
- Experience of near Real Time & Batch Data Pipeline development in a similar Big Data Engineering role.
- Programming skills with an OOP language (e.g., Java, C++)
- Proficient with SQL
- Experience working in a cloud environment such as Google Cloud Platform or AWS
- Hands on programming experience of the following (or similar) technologies: Kubernetes, Docker, Apache Beam, Apache Flink, Apache Spark, Google BigQuery, Snowflake, Google BigTable, Google Pub/Sub, Kafka, Apache Airflow
- Experience implementing observability around data pipelines using SRE best practices.
- Experience in processing structured and unstructured data into a form suitable for analysis and reporting with integration with a variety of data metric providers ranging from advertising, web analytics, and consumer devices.
Responsibilities
- Develop and maintain batch and streaming data pipelines according to business and technical requirements.
- Deliver observable, reliable and secure software, embracing “you build it, you run it” mentality, and focus on automation.
- Continually work on improving the codebase and have active participation in all aspects of the team, including agile ceremonies.
- Take an active role in story definition, assisting business stakeholders with acceptance criteria.
- Work with Principal Engineers and Architects to share and contribute to the broader technical vision.
- Practice and champion best practices, striving towards excellence and raising the bar within the department.
- Operationalize data processing systems (DevOps) and system observability (SRE)
Other
- 1-3 years relevant experience in Data Engineering
- Bachelors’ degree with a specialization in Computer Science, Engineering, Physics, other quantitative field or equivalent industry experience.
- Strong Test-Driven Development background, with understanding of levels of testing required to continuously deliver value to production.
- Ability to work effectively across functions, disciplines, and levels
- Team-oriented and collaborative approach with a demonstrated aptitude, enthusiasm and willingness to learn new methods, tools, practices, and skills
