McKesson is looking to design and operate reliable, scalable data workflows on the Databricks platform, with a strong focus on data process monitoring, job optimization, and data quality.
Requirements
- 4+ years of hands-on experience with Databricks and Apache Spark in a cloud or enterprise setting.
- Advanced knowledge of Databricks jobs, job monitoring, error handling, and performance metrics tools.
- Good understanding of database fundamentals, including SQL, table design, indexing, and troubleshooting (Oracle, PostgreSQL, MongoDB).
- Experience building, documenting, and supporting reliable production-grade data workflows.
- Proficient in Python and SQL for data engineering and automating monitoring/reporting tasks.
- Experience with data process monitoring tools, alerting automation, and dashboarding inside Databricks.
- Understanding of Delta Lake and lakehouse methodologies.
Responsibilities
- Build, optimize, and maintain batch and streaming data pipelines using Databricks, Apache Spark, and Delta Lake for cloud analytics workloads.
- Monitor, troubleshoot, and report on the status and health of data pipelines and processing jobs using Databricks-native tools, logs, and dashboards to ensure timely and reliable data delivery.
- Analyze and resolve job failures, resource bottlenecks, and data quality issues, escalating problems as needed and providing root-cause analysis.
- Apply strong SQL and data modeling knowledge (from Oracle, PostgreSQL, MongoDB) when creating, transforming, and validating large data sets to support a variety of business and analytics use cases.
- Implement and enforce data security controls, encryption, and access policies within Databricks, following industry best practices and healthcare compliance requirements.
- Work with data governance, compliance, and IT security teams to continuously evaluate and improve system security, privacy and regulatory alignment.
- Document pipeline architecture, monitoring processes, and standard operating procedures for the data engineering team and other stakeholders.
Other
- Degree or equivalent and typically requires 4+ years of relevant experience.
- Bachelor’s degree in computer science, Information Systems, Engineering, or related discipline.
- Candidates must be authorized to work in USA. Sponsorship is not available for this role.
- McKesson provides equal employment opportunities to applicants and employees and is committed to a diverse and inclusive environment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, disability, age or genetic information.
- Collaborate with business intelligence, analytics, and data operations teams to deliver high-quality data with consistent performance and availability.
