Data Engineer II is a role designed for individuals with a solid understanding and some hands-on experience in data engineering. This role involves taking ownership of larger data engineering tasks, designing solutions for moderately complex problems, and ensuring high-quality data pipelines and integrations.
Requirements
- Proficiency in SQL and experience with relational databases.
- Experience with cloud platforms (e.g., AWS) and data warehousing technologies (e.g., Snowflake).
- Strong understanding of data integration techniques and ETL/ELT processes.
- Ability to write complex code with a focus on optimization and scalability.
- Strong attention to detail and ability to ensure data quality and integrity.
- Experience with Python, big data technologies, and distributed data processing.
- Familiarity with data visualization tools like Power BI.
Responsibilities
- Independently design, develop, and maintain data pipelines and ETL processes for moderately complex data flows.
- Write complex, efficient, and maintainable code to integrate data from multiple sources, ensuring scalability and performance.
- Optimize and refactor existing code and pipelines to improve efficiency, reliability, and maintainability.
- Troubleshoot and resolve more complex technical issues with data quality, pipeline performance, or integration challenges.
- Ensure data quality, integrity, and security through monitoring, validation, and the application of best practices.
- Automate and document data processes to improve efficiency and repeatability.
- Collaborate on troubleshooting and resolving complex production data issues to ensure system reliability and availability.
Other
- 2-5 of experience in data engineering or a related field.
- Ability to work effectively in cross-functional teams and communicate technical concepts to non-technical stakeholders.
- Excellent problem-solving skills with a focus on troubleshooting complex data issues.
- Exposure to Agile development methodologies and tools (e.g., JIRA, Confluence).
- Experience mentoring or coaching junior engineers.
