eBay is undertaking a modernization initiative for the Batch Platform by transitioning from Broadcom’s Automic to Apache Airflow. We are seeking a Senior Software Engineer with expertise in Apache Airflow to join our Batch Operations team. This is a platform engineering role that presents an opportunity to provide Airflow as a Service and enable hundreds of internal users to coordinate diverse workflows across multiple marketplace domains! You will build foundational platform capabilities and craft how thousands of workflows provide business value across eBay’s global marketplace.
Requirements
- Demonstrated experience in providing Apache Airflow as a managed service within enterprise settings supporting extensive user groups.
- Expertise in architecting, managing, and scaling Airflow deployments in Kubernetes or cloud environments, including operational challenges, synchronization of DAG or equivalent experience, and multi-region failover.
- Deep familiarity with workflow automation patterns, enterprise scheduler migration, and bridging legacy-platform feature gaps.
- Proven success developing observability and operational tools (metrics, monitoring, dashboards, alerting, log analysis) for workflow orchestration platforms.
- Familiarity with Automic Enterprise Scheduler is a plus.
Responsibilities
- Develop, deploy, and maintain Airflow-as-a-service to fulfill eBay’s batch processing demands, ensuring consistent availability and security.
- Lead the deployment and management of Airflow clusters, focusing on reliability and scalability.
- Design and implement observability, monitoring, and operational solutions to offer deep insights into DAG/task executions and system health.
- Evaluate, spec, and implement feature gap between UC4 and Airflow.
- Automate and maintain operational standards across multiple environments.
- Support horizontal scaling and high availability across cloud infrastructure clusters.
Other
- Bachelor’s degree in Computer Science or related field plus 7 or more years of experience or Master’s degree in Computer Science or related field with 5 years of software engineering experience
- Excellent technical leadership, initiative, and collaboration skills; thrive in fast paced, high-impact modernization programs.
- If hired, employees will be in an “at-will position”
- All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, veteran status, and disability, or other legally protected status.
