Circana is seeking a Factory Data Engineer to lead the development of scalable data solutions in a cloud-native environment, involving designing data pipelines, optimizing data warehouse performance, and working with in-house tools for data operations and management.
Requirements
- Minimum 5 years of experience in Python, Perl, C programming languages
- Good understanding of Spark and pySpark
- Good understanding of databases such as Hive, Postgres or Oracle
- Should be familiar with scheduling tools such as Airflow or Control-M
Responsibilities
- Create Python, Spark, Perl, Shell Scripting programs to process data sets
- Maintain existing Python, Spark, Perl, Shell Scripts and C programs
- Learn and work extensively with in-house data tools and frameworks, contributing to their enhancement and integration.
- Ensure data quality, governance, and security across all data assets.
- Drive performance tuning, cost optimization, and automation of data workflows.
- Create required documentation and Provide interface documentation to other team members on interfaces
- Validate data generated to confirm that desired values are created
Other
- Collaborate with cross-functional teams to understand data requirements and deliver high-quality solutions.
- Mentor junior engineers and promote best practices in data engineering.
- Participate in Agile ceremonies, code reviews, and technical design discussions.
- Follow established SOPs and practices
- A passion for technology and insatiable appetite to learn, learn-it-all attitude.
- Strong communication skills, ability to write technical docs and present ideas
