DNV Energy Management's Analytics & Data Science team needs to accelerate the transition toward a carbon-free future through software and analytics by building scalable, cloud-native data solutions that support demand side management, demand flexibility, and transportation electrification programs.
Requirements
- Azure Databricks (Apache Spark)
- Azure DevOps (pipelines, repos, CI/CD)
- Azure Service Bus
- PostgreSQL
- Python and SQL (T-SQL, PostgreSQL dialects)
- Experience with time-series data modeling, storage, and processing at scale.
- Familiarity with event-driven architectures and real-time/streaming data pipelines.
Responsibilities
- Develop & Optimize Data Pipelines: Build ETL/ELT pipelines using Azure Databricks (Spark), PostgreSQL, and Python to support large-scale data processing and analytics.
- Architect & Automate Workflows: Implement orchestration and messaging using Azure Service Bus and Databricks Workflows for automated, scalable operations.
- Enable ML & Advanced Analytics: Partner with ML engineers and software engineers to deliver clean, well-modeled data for feature engineering and operational ML pipelines.
- Ensure Data Quality & Reliability: Apply validation frameworks, observability tools, and monitoring using Azure Monitor and related services.
- Optimize for Performance & Scalability: Tune Spark jobs, queries, and pipelines for distributed computing efficiency.
- Implement DevOps & CI/CD: Build and maintain Azure DevOps pipelines for testing, version control, and automated deployment of data solutions.
- Foster Collaboration & Mentorship: Work with software developers, analytics engineers, and ML engineers to align data efforts with business goals, while mentoring junior engineers.
Other
- Bachelor's degree in Computer Science or related discipline, or equivalent work experience in data engineering.
- Minimum 5 years of professional data engineering experience, with at least 3 years on Azure-based solutions.
- Demonstrated ability to manage multiple projects and context switch effectively.
- Excellent written and verbal communication skills in English.
- Immigration-related employment benefits, for example visa sponsorship, are not available for this position.
