As a Big Data Lead, you will play a critical role in designing, developing, and maintaining our big data applications and pipelines to deliver on major business priorities within the Client Financial Platform.
Requirements
- Strong understanding of big data concepts, architectures, and technologies (e.g., Hadoop, Spark, Hive, Kafka, NoSQL databases).
- Proficiency in programming languages such as Java, Scala, or Python.
- Knowledge of cloud-based big data platforms (e.g., AWS EMR, Azure HDInsight, Google Cloud Dataproc).
- Certifications in relevant big data technologies.
- Experience with data warehousing and business intelligence tools.
- Experience with data visualization and reporting.
Responsibilities
- Design and implement scalable and robust big data architectures and pipelines to handle large volumes of data from various sources.
- Evaluate and select appropriate big data technologies and tools based on project requirements and industry best practices. Implement and integrate these technologies into our existing infrastructure.
- Develop and optimize data processing and analysis workflows using technologies such as Spark, Hadoop, Hive, and other relevant tools.
- Implement data quality checks and ensure adherence to data governance policies and procedures.
- Continuously monitor and optimize the performance of big data systems and pipelines to ensure efficient data processing and retrieval.
- Stay up-to-date with the latest advancements in big data technologies and explore new tools and techniques to improve our data infrastructure.
Other
- Lead and mentor a team of big data engineers, fostering a collaborative and high-performing environment. Provide technical guidance, code reviews, and support for professional development.
- Collaborate effectively with cross-functional teams, including data scientists, business analysts, and product managers, to understand their data needs and deliver impactful solutions.
- 8 - 10 years experience in leading and mentoring big data engineering teams.
- Excellent problem-solving and analytical skills.
- Strong presentation, communication and interpersonal skills.
- Ability to adjust priorities quickly as circumstances dictate
- Demonstrated leadership and project management skills
