Veracyte is looking to hire a Senior Data Engineer to design, develop, and maintain scalable cloud data infrastructure and pipelines to support the company's data engineering needs, specifically for their Lakehouse built on AWS and Snowflake, to transform cancer care.
Requirements
- Hands-on experience with designing and deploying data pipelines in cloud environments, preferably AWS and/or GCP.
- Proficiency in programming languages such as Python, Java, or Scala.
- Experience with AWS services (S3, Glue, Lake Formation, SageMaker) and Snowflake for data warehousing, ELT processes, and data modeling.
- Familiarity with data cataloging tools, data lakes, and governance best practices.
- Knowledge of open formats like Apache Parquet and Iceberg is a plus.
- Experience with technologies such as Amazon S3, Snowflake, AWS Glue, Lake Formation, and SageMaker for data storage, processing, and analytics.
- Share expertise in data engineering best practices, emerging technologies, and tools like Apache Parquet, Iceberg, and Zero-ETL integrations.
Responsibilities
- Build and maintain scalable, efficient data pipelines and infrastructure for Lakehouse systems, including bronze, silver, and gold data layers.
- Work with technologies such as Amazon S3, Snowflake, AWS Glue, Lake Formation, and SageMaker for data storage, processing, and analytics.
- Partner with the Technical Program Manager (TPM), data scientists, and stakeholders to understand business requirements and translate them into technical data solutions.
- Participate in Scrum processes, including backlog grooming, sprint planning, and handling data set requests via Jira.
- Optimize data retrieval, processing, and ELT workflows for improved performance and reliability.
- Implement data security measures, governance policies, and compliance with PHI, consent, and regulatory requirements.
- Assist in identifying and assessing internal and external data sources for the data catalog.
Other
- Bachelor’s or Master’s degree in Engineering, Computer Science, or a related field.
- 6+ years of experience (BS) or 3+ years (MS) in data engineering or a similar role.
- Strong problem-solving and analytical skills.
- Excellent communication and collaboration abilities to work effectively in a cross-functional Scrum team.
- Ability to thrive in a fast-paced, dynamic environment focused on data as a service (DaaS).
