The data engineer is responsible for designing, building, and maintaining the systems and infrastructure needed for data storage, processing, and analysis. The data engineer will work with a multidisciplinary Agile team to build high-quality data pipelines, generate insights from connected data, and advance the company's data-driven decision-making capabilities.
Requirements
- Experience in building data pipelines and deploying/maintaining them following modern DE best practices -target tech stack: Python OSS Data Ecosystem (PySpark, pandas, Dash); knowledge of SQL, DBT is a plus
- Heavy experience with Azure Data Factory is a must.
- Experience with designing and maintaining data warehouses and/or data lakes with big data technologies such as Spark/Databricks, or distributed databases, like Redshift and Snowflake, and experience with housing,accessing, and transforming data in a variety of relational databases
- Software Engineering fundamentals and software development tooling (e.g., Git, CI/CD, JIRA) and familiarity with the Linux operating system and the Bash/Z shell
- Experience with cloud database technologies (e.g., Azure) and developing solutions on cloud computing services and infrastructure in the data and analytics space
- Basic familiarity with BI tools (e.g., Alteryx, Tableau, Power BI, Looker)
- Experience working in data engineering or data architect role
Responsibilities
- Designs, develops, optimizes, and maintains data architecture and pipelines that adhere to ELT principles and business goals.
- Solves complex data problems to delivers insights that helps business achieve its goals.
- Creates data products for engineer, analyst, and data scientist team members to accelerate their productivity.
- Engineer effective features for modelling in close collaboration with data scientists and businesses.
- Leads the evaluation, implementation and deployment of emerging tools and process for analytics data engineering in order to improve productivity as a team and quality.
- Partners with machine learning engineers, BI, and solutions architects to develop technical architectures for strategic enterprise projects and initiatives.
- Fosters a culture of sharing, re-use, design for scale stability, and operational efficiency of data and analytical solutions.
Other
- Hybrid 3x in office
- Contract-to-hire
- 6months-1year
- Strong understanding of agile methodologies and experience as a Data Engineer on a cross-functional agile team preferred
- 3+ years of experience as a Data Engineer, preferably in a large organization
