The company is looking to build and enhance its state of the art datalakehouse to deliver immediate value to customers by creating valuable, actionable insights from vast quantities of data.
Requirements
- Strong programming and technology knowledge in cloud data processing.
- Previous experience working in matured data lakes
- Strong data modelling skills for analytical workloads.
- Spark (or equivalent parallel processing framework) experience is needed, existing Databricks knowledge is a plus.
- Technical understanding of underlying systems and computation minutiae.
- Experience working with distributed systems and data processing on object stores.
- Highly proficient in Python (or another object orientated language)
Responsibilities
- Optimize data workloads at a software level by improving processing efficiency.
- Develop new data processing routes to remove redundancy or reduce transformation overhead.
- Perform complex data transformation in order to accomplish niche business use cases.
- Develop new frameworks for data development to supercharge delivery velocity.
- Monitor and maintain existing data workflows.
- Use observability best practices to ensure pipeline performance.
- Perform complex transformations on both real time and batch data assets.
Other
- Ability to work autonomously is a MUST.
- Interest and aptitude for cybersecurity; interest in identity security is highly preferred.
