GM Energy Data Engineering team is looking for a Senior Software Engineer to build and maintain large-scale data streaming platforms to support high-throughput data streams, data curation, storage, security, governance, and real-time AI-driven algorithms.
Requirements
- 8+ years software engineering experience, with a focus on data engineering and proficiency in java frameworks (Spring boot, Quarkus).
- Minimum of 5 years of experience with streaming technologies such as Kafka,Spark, etc.
- Proven cloud experience and strong familiarity with at least one cloud platform (Microsoft Azure - preferred, AWS, GCP).
- Experience with Databricks and/or Snowflake and/or other cloud-based data platforms required.
- Experience in Python and PySpark preferred.
- Good understanding and experience with CI/CD practices.
- Experience with data visualization tools to effectively communicate insights is preferred.
Responsibilities
- Creating and maintaining robust pipelines to collect, process, and store data from various enterprise source, IoT devices and sensors.
- Design and develop a scalable data platform to support growing data volume and complexity.
- Implementing logic to clean, filter, and transform raw device data into formats suitable for downstream analytics or storage.
- Maintain and enhance infrastructure for efficient data ingestion, transformation, and delivery.
- Implement rigorous data validation, governance, and quality assurance to ensure data integrity.
- Monitoring data flows, handling errors, and implementing retry mechanisms to ensure reliable and accurate data ingestion.
- Setting up monitoring tools and dashboards to track pipeline health, diagnose issues, and optimize performance.
Other
- This role is categorized as hybrid. This means the successful candidate is expected to report to GM Georgia IT Innovation Center or GM Global Technical Center or Austin Innovation Technical Center three times per week, at minimum.
- Mentor and support junior engineers through guidance, coaching, and learning opportunities.
- Stay current with industry trends and best practices in data management, ODS technologies, and API development.
- Solid problem-solving skills and the ability to translate business questions into data-driven solutions.
- GM DOES NOT PROVIDE IMMIGRATION-RELATED SPONSORSHIP FOR THIS ROLE.
