Travelers Data Engineering team needs to construct pipelines that contextualize and provide easy access to data by the entire enterprise, and grow and transform the analytics landscape.
Requirements
- Experience with building robust data pipelines and working with large-scale datasets.
- Highly proficient use of tools, techniques, and manipulation including Cloud platforms, programming languages, and a full understanding of modern engineering practices.
- Proficiency in tools like AWS, Databricks, Snowflake, Ab Initio, Terraform etc.
- Experience with some of the following tools & platforms (or similar): AWS (S3, Lambda, Kinesis, API Gateway, IAM, Glue, SNS, SQS, EventBridge, EKS, VPC, Step Functions, ECS/EKS, DynamoDB, etc.), Databricks, Python, JavaScript, Kafka, dbt, Terraform, Snowflake, SQL, Jenkins, GitHub, Airflow, Alation, Secrets Management (HashiCorp Vault, AWS Secrets Manager, or similar), Docker / OpenShift / Open Cloud Foundry, MongoDB, SonarQube
- Knowledge and experience with the some of the following concepts: Real-time & Batch Data Processing, Workload Orchestration, Cloud, Data lakes, Data Security, Networking, Serverless, Testing/Test Automation (Unit, Integration, Performance, etc.), DevOps, Logging, Monitoring, and Alerting, Containerization, Encryption / Decryption, Data Masking, Cost & Performance Optimization
- Experience with data engineering or equivalent experience.
- Strong problem solver who ensures systems are built with longevity and creates innovate ways to resolve issues.
Responsibilities
- Build and operationalize complex data solutions, correct problems, apply transformations, and recommending data cleansing/quality solutions.
- Design complex data solutions
- Perform analysis of complex sources to determine value and use and recommend data to include in analytical processes.
- Incorporate core data management competencies including data governance, data security and data quality.
- Collaborate within and across teams to support delivery and educate end users on complex data products/analytic environment.
- Perform data and system analysis, assessment and resolution for complex defects and incidents and correct as appropriate.
- Test data movement, transformation code, and data components.
Other
- Bachelor’s Degree in STEM related field or equivalent
- Eight years of related experience
- Strong written and verbal communication skills with the ability to work collaborate well with team members and business partners.
- Ability to lead team members and help create a safe environment for others to learn and grow as engineers.
- Four years of data engineering or equivalent experience.
