Enhancing the functionality of the current data platform built on GCP for a global management consulting firm based in New York
Requirements
- Proficiency in Python, SQL, and Groovy
- Competency in Jenkins, GCP (BigQuery, Cloud Storage, Pub/Sub), and Cloud SQL
- Proven track record working with ETL tools such as Airflow
- Knowledge of JIRA, Confluence, and other collaboration tools
- Strong engineering process and hygiene skills e.g. source control, coding standards, code reviews
- Comfortable working in an agile team (Scrum)
- 4+ years of experience as a Data Engineer
Responsibilities
- Enhancing the functionality of the current data platform built on GCP (BigQuery, Cloud Storage, Pub/Sub). Typical tasks include adding new client data, processing it through the existing architecture, and securing final acceptance by product owners
- Developing new features and refactoring legacy solutions to align with modern data engineering best practices
- Handling data quality incidents, acting as the main point of contact between clients and the customer to resolve issues efficiently
- Managing the release process using Jenkins (writing in Groovy), implementing fixes, and delivering new functionality
- Building and maintaining ETL workflows with Apache Airflow, including adding new DAGs and optimizing existing ones
- Developing monitoring dashboards (e.g., in Looker, Data Studio, or Grafana) to ensure system quality and enable proactive issue resolution
- Working with GCP services (e.g., Secret Manager for credential storage, making configuration changes to systems, managing cloud resources)
Other
- Upper-intermediate English level
- Equal Opportunity Employer committed to inclusion across minority, gender identity, sexual orientation, disability, age, and more
- Reasonable accommodations are available to enable individuals with disabilities to perform essential functions
- International projects
- In-office, hybrid, or remote flexibility
