OUC - The Reliable One is seeking a Data Engineer to structure, organize, and optimize scalable data pipelines, cloud architecture, and business intelligence to translate complex business needs into secure, high performance data solutions.
Requirements
- Expertise in SQL, Python, and Scala, ensuring optimized query performance for structured and unstructured data processing.
- Proficiency in two or more of the following languages for diverse technical implementations: JavaScript (Snowpark), DAX, Power Query (M), WDL, Power Fx, C-Sharp, Perl, VBA, R, Julia.
- Strong understanding of big data frameworks and distributed computing for scalable analytics.
- Proficiency in ETL/ELT tools, including Talend, Dataiku, Alteryx, Snowpipes, Azure Data Factory, SSIS.
- Experience in workflow automation using Power Automate and cloud-based integration platforms.
- Strong knowledge of cloud-native data solutions, including Snowflake, Databricks, AWS S3, Azure Synapse, and relational databases like Oracle, MySQL.
- Proficiency in VM-based deployments using Azure VMs, AWS EC2, Google Compute Engine for scalable data processing.
Responsibilities
- Gather and interpret business requirements, translating them into optimized data pipelines, models, and infrastructure solutions that support BI, analytics, and AI-driven decision- making.
- Perform data modeling using foundational and modern principles, methodologies, and automation strategies to structure data for predictive analytics, reporting, and AI-driven decision-making.
- Implement secure, high-performance integrations between on-premises and cloud-based storage solutions, optimizing data accessibility for BI, analytics, and operational insights.
- Develop and maintain scalable ETL/ELT pipelines for efficient data extraction, transformation, and loading, ensuring integration with AI/ML workflows or Data Analytics reporting layer.
- Ensure real-time and batch processing capabilities, supporting streaming, and event-driven architectures, while implementing robust data access management solutions to facilitate secure, compliant data accessibility for business intelligence (BI), analytics, and operational decision-making.
- Review and audit more complex SQL queries and ETL/ELT pipelines, ensuring efficiency, compliance, and cloud and AI-readiness, adhering to best practices and established standards.
- Collaborate with data scientists, reporting analysts, programmers, business systems analysts, solution architects, and IT teams to deliver reliable data solutions.
Other
- Bachelor’s Degree in Information Technology, Computer Science, Computer Engineering, Management Information Systems, Mathematics, Statistics, or a related field of study from an accredited college or university.
- Minimum of 3 years of experience in SQL or Python programming, ETL/ELT processes, data warehousing & cloud, data modeling and architecture, data security and governance, and visualization and reporting.
- Must be onsite Tuesdays and Thursdays, with two remote workdays per week.
- Ability to effectively communicate insights to non-technical stakeholders, gather and translate business requirements into scalable data architectures.
- Strong collaboration skills with data scientists, ML engineers, analysts, and IT teams.
