Stable Kernel is looking to solve the problem of developing and delivering innovative, transformational data initiatives for their portfolio of world-class clients, enabling them to extract actionable insights crucial to the health of their business.
Requirements
- Extensive experience using Python.
- Designing and building data pipelines: Developing and maintaining scalable ETL/ELT pipelines using, among other technologies, AWS services such as AWS Glue, Lambda, and Step Functions to extract, transform, and serve data from upstream sources.
- Custom extraction from non-SQL sources (e.g., APIs) using established industry patterns for incremental and full snapshot datasets using Python and standard data engineering/data science libs
- Implementing and enforcing data quality checks to ensure the accuracy, consistency, and reliability of data across systems.
- Create and manage CI/CD to deploy system and data changes
- GoLang
- Proven experience using additional programming languages is a plus.
Responsibilities
- Design and implement robust data pipelines, with an eye to the growth of the product and team, improving the speed of future work and ensuring high data quality and contributing to DataOps/DevOps initiatives to ensure complete and timely data flows.
- Improve performance of data pipelines and data lake queries used by data analysts.
- Apply your firm grasp of the concepts and practices of data engineering in the areas of ingestion, transformation, serving with a strong awareness of system and project needs for security, data management, DataOps, Data architecture, orchestration and standard software engineering practices.
- Providing expert feedback that improves the overall delivery of work.
- Defending the work against compromise and corner-cutting.
- Designing test strategies and involving yourself in CI/CD and automated testing for high system reliability.
- Mentoring others in best practices.
Other
- 7-10 years of development experience.
- Degree in Computer Science or relevant field or experience.
- Minimal travel is associated with this role.
- We are not sponsoring employment visas at this time. If you are currently in possession of an employment visa, it will need to be effective for at least 3 years from the date of your application for your candidacy for employment to be considered.
- Company-subsidized health, vision, and dental plan premium options beginning on the first day of employment
