Transforming raw data into actionable insights, building robust data infrastructures, and enabling data-driven decision-making and innovation through advanced analytics and predictive modeling.
Requirements
- Proficiency in Apache Spark
- Proficiency in Python.
- Experience implementing ETL/ELT process and Data pipelines.
- Experience with Snowflake.
- Strong SQL scripting experience.
- Background and experience with cloud data technologies and tools.
- Familiar with data tools and technologies like: Spark, Hadoop, Apache beam, Dataproc or similar.
Responsibilities
- Design and build processes, data transformation, and metadata to meet business requirements and platform needs.
- Design and propose solutions for the Relational and Dimensional Model based on platform capabilities.
- Develop, maintain, test, and evaluate big data solutions.
- Focus on production status and data quality of the data environment.
- Pioneer initiatives around data quality, integrity, and security.
- Work closely with the Data Analyst/Data Scientist to understand evolving needs and define the data processing flow or interactive reports.
- Propose solutions for the cloud-based architecture and deployment flow.
Other
- 5+ years of experience in Data Engineering.
- Some experience leading IT projects and stakeholder management.
- Participation in both internal meetings and external meetings via video calls, as necessary.
- Ability to go into corporate or client offices to work onsite, as necessary.
- Prolonged periods of remaining stationary at a desk and working on a computer, as necessary.
