Convey Health Solutions needs a Lead Data Engineer to deliver and continuously improve healthcare data products and platform components, translating business strategy into data engineering execution while ensuring HIPAA and HITRUST compliance.
Requirements
- Python, PySpark, Scala (10+ years) – Directed large-scale data platform development with an emphasis on modularity and performance.
- Airflow (6+ years) – Defined standards for DAG design, observability, and reusability across teams.
- AWS (Glue, EMR, Lambda, Step Functions) (5+ years) – Designed hybrid orchestration strategies using serverless and EMR-based pipelines.
- Iceberg (4–5 years) – Led lakehouse strategy including governance, versioning, and cost optimization.
- Deep expertise in designing and optimizing data lake and lakehouse architectures at scale, including governance, schema evolution, cost efficiency, and access control.
- Docker/Kubernetes (6–8 years) – Managed Kubernetes-based orchestration for enterprise data pipelines and established Docker standards aligned with compliance and DevOps strategies.
- Extensive experience with large-scale data systems and cloud-based architectures (AWS required).
Responsibilities
- Architect and lead the adoption of serverless data platforms integrating AWS services such as Glue, Lambda, DynamoDB, and Lakehouse Formation.
- Lead GitOps strategy for infrastructure and pipeline deployment, ensuring compliance, rollback safety, and peer-reviewed workflows.
- Establish and enforce best practices for feature flag toggling strategies in critical data pipelines.
- Drive internal knowledge sharing and technical documentation standards across the engineering organization.
- Design scalable architectural solutions and effectively communicate complex technical concepts to both technical and non-technical audiences.
- Own technical environment and tooling strategy across the data engineering organization, ensuring standards, documentation, and collaboration practices are consistently applied.
- Maintain and oversee governance using: GitHub (repository standards and peer review), AWS Console (Glue, Iceberg, EMR, IAM governance), Jupyter/Python IDE (advanced notebook-based development), Lucidchart (platform-wide architecture diagrams), Microsoft Teams/Zoom (engineering leadership and stakeholder communications), Jira & Confluence (governance documentation and delivery tracking).
Other
- Proven track record of leading high-performing teams of 5–8 engineers.
- Acting as a bridge between engineering, data science, and product functions, this role supports alignment during sprint planning and delivery, emphasizing clear communication, documentation, and team empowerment.
- Mentor and guide junior engineers to promote professional growth, collaboration, and best practices.
- Excellent written, verbal, and interpersonal communication skills.
- Proven ability to motivate and lead teams, setting clear expectations and driving accountability.
