Airbnb needs to build essential AI/ML data foundations powering all AI/ML use cases across the company to accelerate AI/ML innovation and enable rapid development and deployment of high-quality, impactful AI/ML solutions, including shaping cutting-edge Generative AI infrastructure.
Requirements
- Strong coding skills in Python, Java, or equivalent languages.
- Hands-on experience with distributed processing technologies (Spark, Kafka, Flink, Hadoop) and distributed storage (HDFS, S3).
- Solid knowledge of data warehousing concepts and databases (e.g. PostgreSQL, MySQL, Redshift, BigQuery, ClickHouse).
- Expertise building scalable ETL pipelines using schedulers like Airflow, Luigi, Oozie, or AWS Glue.
- Proven ability to analyze large datasets, identify insights, and drive impactful product solutions.
- Experience building end-to-end Machine Learning platforms and deploying ML models.
- Familiarity with Kubernetes, Docker, and modern infrastructure tools.
Responsibilities
- Design, build, automate, and maintain robust, scalable data pipelines using SparkSQL, Scala, and Airflow.
- Develop and optimize data models ensuring high-quality, consistent, and accurate data to support broad AI/ML product feature decisions.
- Collaborate closely with peer ML Infra teams to deliver automated data solutions driving AI/ML acceleration.
- Contribute to scalable GenAI infrastructure by leveraging foundational language and vision models to create high quality datasets that power cutting edge GenAI applications.
- Partner with key customer teams to deliver high-impact, high-quality datasets core to Airbnb's roadmap.
- Utilize leading open-source technologies including Spark, Airflow, Ray, MLFlow, TensorFlow, PyTorch, Docker, Kubernetes, and more.
- Experience building end-to-end Machine Learning platforms and deploying ML models.
Other
- 5+ years of relevant industry experience (BS/MS) or 2+ years with a PhD.
- Excellent written and verbal communication skills; comfortable collaborating cross-functionally.
- This position is US - Remote Eligible.
- While the position is Remote Eligible, you must live in a state where Airbnb, Inc. has a registered entity.
- All qualified individuals are encouraged to apply.
