RWE Supply & Trading Americas, LLC is building its core AI Weather Lab team to explore creative applications of AI methods to challenges in the renewable energy industry. They are seeking an experienced senior software engineer to help innovate and build industry-leading AI models by architecting, developing, and optimizing end-to-end pipelines for machine learning models.
Requirements
- 7+ years of experience building, deploying, and monitoring complex pipelines that operate at an hours-level cadence
- Extensive experience in large scale data processing and distributed systems such as Apache Spark
- Extensive experience in serverless computing and event-driven architectures for building scalable and cost-effective pipelines, including stream processing frameworks for real-time data ingestion and processing
- Deep expertise in cloud computing platforms such as AWS, Azure, or Google Cloud Platform
- Strong understanding of containerization technologies (Docker, Kubernetes) and orchestration tools
- Proficiency with version control systems (e.g., Git), continuous integration/continuous deployment (CI/CD) pipelines, and infrastructure-as-code tools
- Proficiency in Python and SQL
Responsibilities
- Lead the design, development, and operation of complex machine learning and multi-faceted data fusion pipelines in a production environment
- Collaborate closely with data scientists, machine learning engineers, and scientists to integrate machine learning models into scalable production systems
- Develop and oversee reusable components, libraries, and frameworks to streamline pipeline development and maintenance
- Optimize pipeline performance, reliability, and resource utilization
- Stay on the forefront of advancements in machine learning technologies, cloud services, and software engineering best practices with an eye towards identifying and integrating exceptional new components into the AI lab’s production repertoire
- Develop and maintain the tech stack of the lab
Other
- To start as soon as possible, fulltime, permanent
- If you're passionate about building scalable pipelines for state-of-the-art production machine learning models and thrive in a fast-paced, collaborative environment, we'd love to hear from you!
- Forster a strong team culture where diverse viewpoints, backgrounds and expertise
- Excellent problem-solving skills, attention to detail, and a passion for learning and innovation
- Strong communication and leadership skills, with the ability to work effectively in cross-functional teams and contribute to a positive team climate
- Experience with Google Cloud Platform MLOps tools
- Familiarity with monitoring, logging, and debugging tools for distributed systems (e.g., Prometheus, ELK Stack)
- Understanding of security best practices for securing data pipelines
- Expertise working with large geospatial weather and climate datasets as well as the Pangeo stack
- What we value most is passion, collaboration, diversity of backgrounds and expertise, willingness to learn and innovate, and a determined and resilient work ethic. Even if you think you do not yet display all the skills above, we would still like to hear from you.
- Task oriented and hybrid working model
- Diverse and multicultural team in a highly dynamic and rapidly growing business
- Relocation expenses for moving to the greater Seattle area if you are not already local
- We value diversity and therefore welcome all applications, irrespective of sex, disability, nationality, ethnic and social background, religion and beliefs, age or sexual orientation and identity.
