At Lyft, data is the only way we make decisions. It is the core of our business, helping us create a transportation experience for our customers, and providing insights into the effectiveness of our product launch & features. As a Data Engineer at Lyft, you will be a part of an early-stage team that builds the data transport, collection, and storage, and exposes services that make data a first-class citizen at Lyft. We are looking for a Data Engineer to build a scalable data platform.
Requirements
- Strong experience with Spark and Airflow
- Experience with Hadoop (or similar) Ecosystem, S3, DynamoDB, MapReduce, Yarn, HDFS, Hive, Spark, Presto, Pig, HBase, Parquet
- Strong skills in a scripting language (Python, Bash, etc)
- Experience of driving building complex data models and pipelines
- Proficient in at least one of the SQL languages (MySQL, PostgreSQL, etc)
- Preferred experience of building and maintaining pay, identity, or integrity related data tables as a Data Engineer for large organizations
Responsibilities
- Develop, evolve, and support pipelines based on business and engineering needs
- Implement and adopt systems tracking data quality and consistency
- Propose and develop tools supporting self-service data pipeline management (ETL)
- SQL and MapReduce job tuning to improve data processing performance
- Contribute to the DE team tech roadmap and align it with stakeholders
- Build well-crafted, well-tested, readable, maintainable code with data infrastructure cost and scalability in mind
- Participate in code reviews to ensure code quality and distribute knowledge
Other
- 5+ years of relevant professional experience
- Experience of working directly with cross-functional data analytics, data scientists, and engineering teams to bridge Lyft's business goals with data engineering
- Participate in on-call rotations to ensure high availability and reliability of workflows and data
- Unblock, support and communicate with internal & external partners to achieve results
- This role will be in-office on a hybrid schedule — Team Members will be expected to work in the office 3 days per week on Mondays, Wednesdays, and Thursdays.
