84.51° is seeking a hands-on Lead Data Engineer to guide a high-performing, hybrid scrum team of onshore and offshore engineers to architect, build, and operate their analytical lakehouse on a modern data stack, partnering closely with Product Management to own the data strategy and deliver scalable, high-quality data products.
Requirements
- 4+ years of hands-on experience with Spark (PySpark/SQL), including building and operating Delta Live Tables (DLT) pipelines.
- Strong proficiency with ADLS Gen2, Entra ID (Azure AD), Key Vault, and related services.
- Expert-level Python and SQL.
- Proven experience with Terraform.
- Strong understanding of data modeling (medallion, dimensional), data warehousing, and CI/CD best practices (Git, GitHub Actions).
- Experience with ELT tools (Fivetran), Snowflake, and real-time streaming data (Event Hubs, Kafka).
- Familiarity with AI-ready data strategies and AI development tools (e.g., GitHub Copilot).
Responsibilities
- Own the architecture and delivery of secure, scalable data pipelines on Databricks (PySpark, Delta Lake, DLT) and Azure.
- Lead technical strategy, establish engineering best practices (coding, reviews, CI/CD, IaC), and drive the adoption of modern tools and patterns like medallion architecture and Unity Catalog.
- Remain high-touch with design, coding (Python, SQL), and production operations.
- Implement robust monitoring, lead incident response, and optimize cloud performance and cost (FinOps).
- Build and maintain CI/CD pipelines (GitHub Actions) and manage infrastructure with Terraform.
- Translate business needs into technical roadmaps, data contracts, and SLAs.
- Own the team’s capacity planning and delivery commitments within an agile framework (Scrum/Cycle Planning), removing blockers and driving continuous improvement.
Other
- Manage, coach, and develop a hybrid team of 2-8 data engineers (FTEs and offshore contractors).
- Conduct 1:1s, performance reviews, and career planning while fostering an inclusive, high-ownership culture.
- Lead recruitment and hiring efforts.
- Demonstrated success leading hybrid (onshore/offshore) technical teams in an agile environment.
- Excellent communication skills and a proven ability to partner with Product and business stakeholders.
