The company is looking to build and maintain a scalable, secure, and highly available data platform (PaaS) to meet the growing demands of its users and facilitate efficient data ingestion, processing, storage, and analysis. This involves designing and developing microservices, managing cloud infrastructure, automating deployments, and ensuring data governance and security.
Requirements
- Proven experience managing and optimizing cloud infrastructure (AWS, Azure).
- Demonstrable experience with Infrastructure as Code (IaC) tools (e.g., Terraform, CloudFormation).
- Solid experience with containerization technologies (e.g., Docker, Kubernetes).
- Experience with CI/CD pipelines and DevOps best practices (GitHub Actions preferred)
- Familiarity with data storage and processing technologies (e.g., Kafka, Spark, Flink, Hadoop, Snowflake, MongoDB, S3/ADLS).
- Proficiency in one or more programming languages commonly used for platform development (e.g., Python, Go, Java).
- Strong understanding of networking, security, and performance principles in a cloud environment.
Responsibilities
- Collaborate with stakeholders to gather requirements and translate them into a comprehensive software system. This includes designing microservices, persistence layers, and implementing deployment workflows using tools like Terraform, Ansible, Puppet, or Salt.
- Design, develop, and implement highly available, scalable, and secure platform services for our data PaaS using programming languages (e.g., Python, Go, Java).
- Contribute to the architectural evolution of our data platform, ensuring it meets the growing demands of our users and adheres to best practices.
- Develop and maintain APIs, automation tools, and internal services to streamline platform operations and enhance developer experience.
- Implement and improve data pipelines, storage solutions, and processing frameworks within the PaaS.
- Manage and optimize cloud infrastructure in AWS and Azure cloud services to support our data platform, focusing on cost-efficiency, performance, and reliability.
- Implement Infrastructure as Code (IaC) principles using tools like Terraform for provisioning and managing resources.
Other
- Bachelor's degree in Computer Science, Software Engineering, or related field. OR demonstrable equivalent deep understanding, experience, and capability.
- Master or PhD degree in Computer Science (or related field) and 5+ years of experience in software engineering or 7+ years of experience in software engineering at a large scale. Experience in data system/platform development.
- Familiarity with database administration and optimization is a plus.
- Familiarity with application access control and automation
- Front-end engineering experience is advantageous.
