SEKO Logistics is looking to solve the problem of designing and developing scalable data pipelines to collect, process, and store large volumes of data, while ensuring data quality and integrity.
Requirements
- 3 years’ experience building and supporting data lakehouse architectures using delta lake and change data feeds.
- 3 years’ experience with Spark and Python.
- 3 years’ experience writing code using OO programming.
- 3 years’ experience designing data warehouse table architecture such as star schema or Kimball method
- Experience developing and installing wheelhouse packages for managing dependencies and distributing code.
- Experience creating CI/CD pipelines for analytics solutions.
- Hands-on experience implementing data solutions using Microsoft Fabric.
Responsibilities
- Design and develop scalable data pipelines to collect, process, and store large volumes of data.
- Collaborate with data scientists and analysts to drive scope & requirements as well as deliver strong technical design of data analytics solutions.
- Ensure data quality and integrity through cleansing processes, validation, and automated testing.
- Develop and maintain requirements, design documentation and test plans.
- Implement data integration solutions to combine data from various sources.
- Optimize data workflows for performance and reliability and reduced cloud consumption.
- Monitor and troubleshoot data pipeline issues to ensure smooth operation.
Other
- Must be a self-starter capable of independently meeting objectives and interacting with members of various teams successfully
- Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent work experience).
- 5 years’ experience in a data engineering role.
- Participate in the development of a safe and healthy workplace. Comply with instructions given for their own safety and health and that of others, in adhering to safe work procedures.
- Strong team player with a continuous improvement mindset
