Vistra is seeking a Principal Data Engineer to architect, design, and implement scalable and robust data marketplace and data product solutions that enable efficient data processing, storage, and retrieval.
Requirements
- Strong programming skills in one or more languages and tools, such as Python, DBT, Airflow, GIT
- Strong proficiency in data engineering technologies, such as ETL/ELT frameworks, CI/CD, big data processing, and SQL/Snowflake and NoSQL databases.
- Deep understanding of database systems, data modeling, and data warehousing.
- Strong proficiency in building analytical dashboards using PowerBI/Tableau.
- Experience with cloud-based data storage and processing technologies, such as AWS, Azure, or Google Cloud.
- Proficient in Python, Airflow, DBT, Snowflake, Power BI
- Familiarity with SAP Datasphere is a plus.
Responsibilities
- Lead the design and development of scalable and reliable data pipelines, including data ingestion, processing, storage, and retrieval.
- Develop data models and schemas that support efficient data storage, retrieval, and analytics, employing optimization techniques to enhance query performance and scalability.
- Leverage big data technologies and frameworks (e.g., Snowflake, Data Fabric, SAP Datasphere) to process and analyze large volumes of data, enabling advanced analytics and machine learning initiatives.
- Manage and optimize data infrastructure, including cloud-based platforms, containerization technologies, and distributed computing environments.
- Develop and implement best practices for data modeling, storage, and retrieval.
- Ensure the security and privacy of our data and compliance with relevant regulations.
- Evaluate new technologies and tools for data processing, storage, and retrieval and recommend solutions to improve the efficiency and scalability of our data infrastructure.
Other
- 9-11 years of experience in data engineering, including experience in designing and building data pipelines.
- Ability to work collaboratively in a team environment.
- Strong communication and interpersonal skills.
- Ability to lead and manage projects.
- Experience in leading a team of data engineers and managing complex projects.
