Develop and maintain data processing software to create and optimize data infrastructure, clean, prepare, and optimize data for further analysis and modelling, and lead complex data engineering projects to provide strategic guidance for the company
Requirements
- Deep understanding of distributed computing, data architecture, and scalability.
- Proficiency in designing and maintaining data warehouses and/or data lakes with big data technologies such as Spark/Databricks, or distributed databases, like Redshift and Snowflake
- Proficiency in building data pipelines and deploying/maintaining them following modern DE best practices (e.g., DBT, Airflow, Spark, Python OSS Data Ecosystem)
- Proficiency in Software Engineering fundamentals and software development tooling (e.g., Git, CI/CD, JIRA)
- Proficiency with cloud database technologies (e.g., Azure) and developing solutions on cloud computing services and infrastructure in the data and analytics space
- Basic familiarity with BI tools (e.g., Alteryx, Tableau, Power BI, Looker)
- Expertise in ELT and data analysis, SQL primarily
Responsibilities
- Designs, develops, optimizes, and maintains data architecture and pipelines that adhere to ELT principles and business goals.
- Solves complex data problems to delivers insights that helps business achieve its goals.
- Architect and implement end-to-end data solutions.
- Define data engineering best practices and standards.
- Mentor junior engineers and collaborate with cross-functional teams.
- Creates data products for engineer, analyst, and data scientist team members to accelerate their productivity.
- Engineer effective features for modelling in close collaboration with data scientists and businesses.
Other
- Bachelor’s degree in computer science, statistics, engineering, or a related field.
- 7 plus years of experience required.
- Leadership and project management abilities.
- Strong critical thinking skills and ability to address big data challenges.
- Ability to read, write, and understand English at a level sufficient to perform job-related tasks effectively and safely
