Yobi is looking to solve the problem of ethically democratizing the benefits of data and artificial intelligence by building large-scale, consented behavioral datasets across the United States.
Requirements
- Proven experience in data engineering, with a strong understanding of data workflows and architecture
- Hands-on experience with Delta Lake, Scala Spark, Python Spark, and Airflow
- Deep knowledge of schema evolution, data contracts, dependency graphs, and metadata management
- Ability to design scalable, reliable, and efficient data pipelines
- Strong problem-solving skills and attention to detail
- Familiarity with various data pipeline patterns including push-based, pull-based, batch, streaming, and eventual consistency
Responsibilities
- Design, develop, and maintain scalable data pipelines and workflows to support Behavioral AI products
- Collaborate with cross-functional teams to define data schemas, models, and standards
- Implement data orchestration, workload management, and observability solutions to ensure reliability and performance
- Optimize data processes for cost efficiency and minimal wall-clock time
- Ensure data quality through schema validation, metadata management, and data contracts
- Lead efforts to influence best practices and standards across engineering teams
- Troubleshoot and resolve data pipeline issues promptly to minimize downtime
Other
- Excellent communication skills and ability to work collaboratively in a remote or hybrid environment
- A proactive attitude with a focus on delivering high-quality solutions
- Experience leading or influencing cross-functional engineering teams
- Annual bonus target based on personal and company performance
- Comprehensive health, dental, and vision insurance plans with minimal or no out-of-pocket costs
