eBay is modernizing its Batch Platform by transitioning from Broadcom's Automic to Apache Airflow, aiming to provide Airflow as a Service and enable hundreds of internal users to coordinate diverse workflows across multiple marketplace domains.
Requirements
- Demonstrated experience in providing Apache Airflow as a managed service within enterprise settings supporting extensive user groups.
- Expertise in architecting, managing, and scaling Airflow deployments in Kubernetes or cloud environments, including operational challenges, synchronization of DAG or equivalent experience, and multi-region failover.
- Deep familiarity with workflow automation patterns, enterprise scheduler migration, and bridging legacy-platform feature gaps.
- Proven success developing observability and operational tools (metrics, monitoring, dashboards, alerting, log analysis) for workflow orchestration platforms.
- Familiarity with Automic Enterprise Scheduler is a plus.
Responsibilities
- Develop, deploy, and maintain Airflow-as-a-service to fulfill eBay’s batch processing demands, ensuring consistent availability and security.
- Lead the deployment and management of Airflow clusters, focusing on reliability and scalability.
- Design and implement observability, monitoring, and operational solutions to offer deep insights into DAG/task executions and system health.
- Evaluate, spec, and implement feature gap between UC4 and Airflow.
- Automate and maintain operational standards across multiple environments.
- Support horizontal scaling and high availability across cloud infrastructure clusters.
Other
- Bachelor’s degree in Computer Science or related field plus 7 or more years of experience or Master’s degree in Computer Science or related field with 5 years of software engineering experience
- Excellent technical leadership, initiative, and collaboration skills; thrive in fast paced, high-impact modernization programs.
