Building a Databricks-based data platform that powers machine learning and advanced analytics, focusing on scalable, production-grade feature engineering pipelines.
Requirements
- Strong proficiency in Python and relevant scripting languages, with experience in software development and scripting for Databricks.
- Expertise with libraries and frameworks (e.g., Pandas, Numpy, Scikit-Learn, TensorFlow, PyTorch, Databricks, MLFlow, dvc, dbt) and the ability to select the right tools for the use case.
- Experience building inference endpoints (APIs) and managing compute architecture for efficient model inference and data handling.
- Very good Azure and data & AI technology skills - specifically: Databricks / Python, Azure Datalake Store, Azure AI Search.
- Experience with DevOps practices, including Git, CI/CD using tools such as Azure Pipelines, or similar.
- Several years of experience in machine learning, data science, or a related field, with a strong understanding of statistics and data analysis.
- Experience with Azure especially with data and ML services, containerization (Docker).
Responsibilities
- Build and optimize databricks ETL/ELT pipelines for feature extraction, transformation, and loading from structured and unstructured data sources.
- Collaborate with ML engineers, data scientists, and software developers to deliver reliable, reusable, and versioned feature sets.
- Implement CI/CD pipelines, testing frameworks, and observability for data workflows.
- Develop feature stores, metadata tracking, and lineage tools to support data Ops.
- Ensure data quality, governance, and compliance across all data assets.
- Optimize performance and cost-efficiency of Databricks clusters and jobs for data workloads.
- Contribute to the architecture and design of the data platform and feature engineering framework.
Other
- Flexibility in work arrangements, including home office options and flexible working hours.
- A positive, team-oriented environment that fosters mutual trust, creativity, and initiative.
- Opportunities for career growth within a global, innovative framework.
- A diverse, multicultural workplace with a strong emphasis on team collaboration and professional development.
- Advanced Degree: Master’s degree in Computer Science, Engineering, Mathematics, with 5+ years of ML implementation experience or Ph.D. with 2+ years of hands-on ML Project experience.
