Abacus Insights is looking to solve the problem of unlocking the power of data for health plans to enable the right care at the right time, making life better for millions of people by integrating data into a state-of-the-art platform built on scalable, flexible, and resilient cloud architectures.
Requirements
- Proficiency in Python and SQL.
- Experience building and managing ETL pipelines using Databricks workflows or similar orchestration frameworks.
- Familiarity with structured and semi-structured data, and processing using PySpark.
- Working knowledge of AWS services (S3, SQS) or equivalent cloud services.
- Experience with Terraform and GitLab CI/CD.
- Skills in query tuning and performance optimization in SQL and/or SparkSQL.
- Familiarity with data warehousing solutions such as Snowflake or similar platforms.
Responsibilities
- Partner with clients and implementation teams to understand data distribution requirements.
- Design and develop data pipelines integrating with Databricks and Snowflake, ensuring accuracy and integrity.
- Lead architecture and implementation of solutions for health plan clients, optimizing cloud-based technologies.
- Translate business requirements into technical solutions in collaboration with cross-functional teams.
- Implement robust data security measures and ensure compliance with healthcare regulations such as HIPAA.
- Provide technical leadership, mentor junior engineers, and stay current with industry trends.
- Optimize data processes for efficiency, review architectures regularly, and maintain detailed documentation.
Other
- Bachelor’s degree in Computer Science, Computer Engineering, or related IT field.
- 5+ years of experience in data engineering.
- Strong communication skills with both technical and non-technical stakeholders.
- Unlimited paid time off – recharge when you need it
- Work from anywhere – flexibility to fit your life
