Vertical Bridge is looking to develop and maintain scalable data pipelines and contribute to the design of their enterprise data architecture to support various applications, analytics, and business intelligence platforms.
Requirements
- Experience with Azure cloud services, particularly Azure Databricks and related components
- Knowledge of data integration and ETL processes using Apache Spark
- Proficiency in SQL and familiarity with programming languages such as Python or Scala
- Experience with Power BI or similar BI tools for creating reports and dashboards
- Background in Machine Learning, Artificial Intelligence, and Data Science is strongly preferred for this role
- Solid understanding of Databricks, Lakehouse architecture, integrating Spark for data processing, and Power BI or Databricks One for data visualization
Responsibilities
- Work closely with a team of data engineers, business intelligence developers, and product owners to build scalable and efficient data pipelines, optimize data storage and retrieval processes, and deliver insightful analytics solutions to our stakeholders
- Collaborate with different functional teams to develop, tune, and produce ML models, enabling automated retraining, feature engineering, and monitoring within Databricks MLflow
- Develop and maintain scalable data pipelines to extract, transform, and load data from various sources into our data Lakehouse, focusing on performance optimization and efficiency
- Design and implement data models and schemas to support business intelligence and analytics
- Ensure data quality and integrity through data validation and cleansing processes
- Collaborate with data scientists, analysts, product owners, and business stakeholders to understand data requirements and translate them into technical solutions
- Explore and implement new data technologies and tools to enhance data capabilities
Other
- Position will work directly from the corporate office with minimum travel.
- Bachelor's degree in Computer Science, Data Science, Engineering, or a related field
- Strong analytical and problem-solving skills
- Excellent communication and teamwork abilities
